Azure OpenAI o1 model has no vision support?

[aadishj19]

What happened?

when I upload an image to Azure o1, it immediately replies in a second which led me to check the logs. I believe it Is it using 4o instead?

In the console I see this when I only use text:

LibreChat | id: "chatcmpl-AprNJUwupj6EdgSR5yoRRTnjJPVJT",
LibreChat | model: "o1-2024-12-17",
LibreChat | object: "chat.completion",

However when I insert an image:

LibreChat | id: "chatcmpl-AprLzmEtslgjQZjjupoiNYRsM2Esu",
LibreChat | model: "gpt-4o-2024-08-06",
LibreChat | object: "chat.completion",

I guess its using 4o? I thought with the o1 we can now also use vision with it, right? Do note that I have the updated version of the repo as I noticed a similar issue was raised for the openAI o1 model a few days ago. However, even after updating, the issue remains.

Steps to Reproduce

1. Choose Azure OpenAI o1 model
2. Upload an image and notice it'll immediately reply without spending any time reasoning.
3. Check the console debug logs.

What browsers are you seeing the problem on?

No response

Relevant log output

No response

Screenshots

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

    - group: "region-eastus"
      apiKey: "${EASTUS_API_KEY}"
      instanceName: "my_instance_name"
      version: "2024-12-01-preview" # latest version
      models:
        o1-mini:
          deploymentName: "o1-mini"
        o1-preview:
          deploymentName: "o1-preview"
    - group: "region-sweden"
      apiKey: "${SWEDEN_API_KEY}"
      instanceName: "my_instance_name"
      version: "2024-12-01-preview" # latest version
      models:
        o1-2024-12-17:
          deploymentName: "o1"

Currently, o1-mini is not considered a vision model so it will switch to gpt-4o. If I override this, using o1-mini, you'll get this error, which is from Azure:

2025-01-16T01:05:08.594Z error: [OpenAIClient.chatCompletion] Unhandled error type Error: 400 Invalid content type. image_url is only supported by certain models.

same with o1-preview:

2025-01-16T01:07:24.748Z error: [OpenAIClient.chatCompletion] Unhandled error type Error: 400 Invalid content type. image_url is only supported by certain models.

Same issue with o1-mini and o1-preview on OpenAI (not azure):

2025-01-16T01:08:51.189Z error: [handleAbortError] AI response error; aborting request: 400 Invalid content type. image_url is only supported by certain models.

--

TL;DR only o1 and not -preview or -mini will work

https://oai.azure.com/resource/models/o1/version/2024-12-17/registry/azure-openai

[danny-avila]

What region? API Version? Ensure you can attach images in the Azure playground. o1-mini and o1-preview don't have this capability in the playground:

[aadishj19]

  - group: "eastus2" # Azure resource group name
    apiKey: "${EASTUS2_API_KEY}"
    instanceName: "sua-eastus2" 
    version: "2024-12-01-preview"
    models:
      o1:
        deploymentName: "o1"

I'm able to use it in azure playground. and it takes a good minute or two to reply as well.

[danny-avila]

I meant if you can attach images in the playground. o1 is not available in eastus2:

only o1-preview and o1-mini

[aadishj19]

I think I found the issue. I had it enabled under the AI services category rather than Azure OpenAI category of resource group.

Also, is it possible to set the reasoning parameter yet? either in the yaml or env somehwre?

[danny-avila]

Not yet, track the issue here: #5067