Add support for Gemini Flash 2.0 Thinking model

[gasser707]

Hey Team! Thanks for this awesome project.

Gemini Flash 2.0 Thinking is released and its an awesome model in terms of speed and quality.
There is one issue with its response format that makes it impossible for BAML to parse its response. as explained here in the docs
The api returns content.parts where the first item in parts contains the model's thoughts and the second item has the actual response in the valid json format.

Here is a sample error:

baml_client/client.rb:114:in parsed_using_types': Failed to parse LLM response: (2 other previous tries) (RuntimeError) LLM call failed: LLMErrorResponse { client: "GEMINI_V2_FLASH_THINKING", model: None, prompt: Chat([RenderedChatMessage { role: "user", allow_duplicate_role: false, parts: [Text("Extract the resume from the following content:\nGman, [email protected], dev at Foo\n\nAnswer in JSON using this schema:\n{\n name: string,\n email: string,\n experience: string[],\n skills: string[],\n}")] }]), request_options: {"model": String("googlevertexai-us-central1:gemini-2.0-flash-thinking-exp-1219")}, start_time: SystemTime { tv_sec: 1734826574, tv_nsec: 413146099 }, latency: 163.465971ms, message: "Failed to parse into a response accepted by baml_runtime::internal::llm_client::primitive::openai::types::ChatCompletionGeneric<baml_runtime::internal::llm_client::primitive::openai::types::ChatCompletionChoice>: {\"choices\":[{\"finish_reason\":\"stop\",\"index\":0,\"logprobs\":null,\"message\":{\"content\":[{\"text\":\"Here's my thought process to arrive at the correct JSON output:\\n\\n1. **Identify the Core Request:** The primary goal is to extract resume information from the provided text and format it as a JSON object according to the given schema.\\n\\n2. **Analyze the Input:** The input is a single line of text: \\\"Gman, [email protected], dev at Foo Compant\\\".\\n\\n3. **Map Input to Schema Fields:** I need to map the information in the input string to the fields in the target JSON schema: name, email, experience, and skills.\\n\\n4. **Extract name:** The name is clearly \\\"Gman\\\". It appears at the beginning of the string and is likely separated by a comma.\\n\\n5. **Extract email:** The email address is \\\"[email protected]\\\". It follows the name and is separated by a comma. The presence of the \\\"@\\\" symbol is a strong indicator of an email address.\\n\\n6. **Extract experience:** The phrase \\\"dev at Foo\\\" clearly indicates a job role (\\\"dev\\\") and a company (\\\"Foo\\\"). This is a standard way to describe work experience. The schema expects an array of strings for experience. Therefore, \\\"dev at Foo\\\" becomes the single element in this array.\\n\\n7. **Extract skills:** The input string *doesn't explicitly list any skills*. There's no section labeled \\\"Skills\\\" or keywords typically associated with skills (e.g., \\\"Python\\\", \\\"Java\\\", \\\"SQL\\\"). According to the schema, skillsis also an array of strings. Since no skills are present, the correct representation is an empty array:[].\\n\\n8. **Format as JSON:** Finally, I assemble the extracted information into a JSON object that adheres to the provided schema. This involves using curly braces {}to define the object, key-value pairs with colons, and commas to separate the pairs. String values need to be enclosed in double quotes. Array values need to be enclosed in square brackets[].\\n\\n9. **Review and Verify:** I double-check that each piece of extracted information is placed correctly in the corresponding field and that the JSON syntax is correct. Specifically, I confirm that experienceandskills` are arrays, even if they contain only one or zero elements.\n\nThis step-by-step breakdown ensures that I correctly interpret the input, understand the schema requirements, and generate the accurate JSON output.","type":"text"},{"text":"json\\n{\\n \\\"name\\\": \\\"Gman\\\",\\n \\\"email\\\": \\\"[email protected]\\\",\\n \\\"experience\\\": [\\n \\\"dev at Foo\\\"\\n ],\\n \\\"skills\\\": []\\n}\\n","type":"text"}],"role":"assistant"}}],"created":1734826498,"id":"2024-12-21|16:14:58.363787-08|1.197.110.217|1363001519","model":"google/gemini-2.0-flash-thinking-exp-1219","object":"chat.completion","system_fingerprint":"","usage":{"completion_tokens":587,"prompt_tokens":62,"total_tokens":649}}\n\nCaused by:\n invalid type: sequence, expected a string", code: Other(2) }

from baml_client/client.rb:114:in 'ExtractResume'

[hellovai]

Thanks for flagging this! We'll be shipping this fix out on monday! (already have a patch out locally that works :) )

[gasser707]

Thanks a ton!

[hellovai]

this is now fixed in #1276