finetune for new task

[ofirliba]

I would like to define a new vision task.
How can I do it?

another question , can I change the output format of the pretrained model ? For example, I want a structured output for the "more detailed caption". A sort of json format like this:

{
"backdround_description": str,
"number_of_people": int,
"people_description", str,
}

thanks

[andimarafioti]

Sure, you can train the model for a new task using this code. I recommend you to check out the blog we wrote about this: https://huggingface.co/blog/finetune-florence2

[ofirliba]

thanks for the fast respond. I will follow the blog.
Regarding the next question - can I change to output format to "json like format" without finetune ?

[andimarafioti]

I don't think so, florence is not that powerful.