Unsuported role system for openai-o1-mini

[AlexEnrique]

Got an error while using this:

await Agent(OPENAI_O1_MINI, system_prompt='you are a helpful assistant').run('hi')

Error from the model

BadRequestError: Error code: 400 - {'error': {'message': "Unsupported value: 'messages[0].role' does not support 'system' with this model.", 'type': 'invalid_request_error', 'param': 'messages[0].role', 'code': 'unsupported_value'}}

Looking at the docs of OpenAI, I saw that system prompts were replaced by developer prompts
(see https://platform.openai.com/docs/api-reference/chat/create)

[Kludex]

Which openai version are you running? pip show openai

[AlexEnrique]

Sorry for not providing on the first message.

Tested with openai versions 1.62.0 and 1.64.0 and pydantic-ai version 0.0.24

[Kludex]

Can you please share a MRE?

[AlexEnrique]

MRE

import os

from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIModel

model = OpenAIModel(
    model_name='o1-mini',
    api_key=os.getenv('OPENAI_API_KEY'),
)


agent = Agent(
    model=model,
    system_prompt='You are a helpful assistant.',
)

agent.run_sync('Hi')

[Kludex]

It seems that with:

from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIModel

model = OpenAIModel('o1-mini') # , system_prompt_role='developer')
agent = Agent(model=model, system_prompt='You are a helpful assistant.')

print(agent.run_sync('Hi'))

I have:

openai.BadRequestError: Error code: 400 - {'error': {'message': "Unsupported value: 'messages[0].role' does not support 'system' with this model.", 'type': 'invalid_request_error', 'param': 'messages[0].role', 'code': 'unsupported_value'}}

And... If I use the system_prompt_role:

from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIModel

model = OpenAIModel('o1-mini', system_prompt_role='developer')
agent = Agent(model=model, system_prompt='You are a helpful assistant.')

print(agent.run_sync('Hi'))

I have this error instead:

openai.BadRequestError: Error code: 400 - {'error': {'message': "Unsupported value: 'messages[0].role' does not support 'developer' with this model.", 'type': 'invalid_request_error', 'param': 'messages[0].role', 'code': 'unsupported_value'}}

So... 🤷‍♂

I've read on their docs that it's recommended to use o3-mini instead, which works as expected:

from pydantic_ai import Agent

agent = Agent(model='openai:o3-mini', system_prompt='You are a helpful assistant.')

print(agent.run_sync('Hi'))

[Kludex]

For what I've found in other forums, it just seems that o1-mini and similar don't support the system role.

[Kludex]

Anyway, since we have the parameter system_prompt_role, I'll be closing this issue.

The problem here is that the model doesn't support either developer or system role.

[Kludex]

You can use the user in system_prompt_role.

[drdavella]

@Kludex this seems like a reasonable workaround, but shouldn't it be possible to automatically set the system_prompt_role based on the known model name string so that users do not have to do it explicitly?

[Kludex]

I think it's a valid point. 👍