Do not add system prompt part when dynamic system prompt function returns empty value

[giacbrd]

The system_prompt decorator allows to dynamically add parts to the messages. However, a part generated by a decorated function is always added, even if the function output is useless, e.g. an empty string. Sometimes the dynamic behaviour we need is to just increment the system prompt, according to the context.

This PR add this behaviour by:

• avoiding adding a system prompt part to the messages if a system prompt function returns a value that evaluates to False
• defining the return type of these functions as str | None, so that it is explicit that these function can optionally return a value

Example of a dynamic system prompt where we add some information only if the retrieve call is successful

@agent.system_prompt
async def get_more_information(ctx: RunContext[MyDeps]) -> str | None:
    response = await ctx.deps.http_client.get(
        f'https://example.com/{ctx.deps.record_id}',
        headers={'Authorization': f'Bearer {ctx.deps.api_key}'},
    )
    if response.status_code == 200 and response.text.strip():
        return f'You also have this information: {response.text}'
    return None

[giacbrd]

The use of Union[str, None] instead of str | None in most of the type hints is due to compatibility with python 3.9, which otherwise fails to run the new code

[sydney-runkle]

@giacbrd,

Thanks for the contribution. I'll chat with the team about this idea tomorrow.

[dmontagu]

You should be fine to use str | None thanks to from __future__ import annotations. There may be some places where you need to use Union, but those would only be places where we analyzed the type-hint at runtime.

[dmontagu]

Yeah, this is a place where you need Union[str, None], though I think it would probably work to just use the string (i.e., 'str | None')

[dmontagu]

This seems wrong to me — with this change, if the function returns None (or '') we just don't update the part, rather than removing it. I think there are a few ways to do this more correctly:

• option 1: remove the part from the msg.parts list (which would need to be done carefully..)
• option 2: allow None in msg.parts, I don't love this
• option 3: set the msg.parts[i] to be a SystemPromptPart with empty string as the value, and modify the handling in the Model implementations to ignore SystemPromptParts with empty content.

[dmontagu]

I pushed a commit to undo some of the unnecessary typing changes.

I feel like the "right" way to fix the issue I pointed out might be to use '' as the value of dynamic system prompts when you want them excluded, and just have the Model implementations not emit an actual system prompt message in those cases.

Either way, we need to make sure that a dynamic system prompt that changes from returning something truthy to returning something non-truthy, or vice versa, doesn't lead to misbehavior.