Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: Python 3.10+ union-style param in function tool #22

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

RezaRahemtola
Copy link
Member

What does this PR do?

Adds a test for the chat template generation issue fixed here: huggingface/transformers#35103

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you read the contributor guideline,
    Pull Request section?
  • Was this discussed/approved via a Github issue or Telegram? Please add a link
    to it if that's the case.
  • Did you write any new necessary tests?

@RezaRahemtola RezaRahemtola self-assigned this Dec 19, 2024
Copy link

codecov bot commented Dec 19, 2024

❌ 1 Tests Failed:

Tests completed Failed Passed Skipped
14 1 13 0
View the top 1 failed tests by shortest run time
tests.tools.test_function_tools::test_function_new_union_style_param_tool
Stack Traces | 0.001s run time
fake_get_temperature_with_new_union_style_param_tool = <function fake_get_temperature_with_new_union_style_param_tool.<locals>.get_current_temperature at 0x7fac26b63400>

    def test_function_new_union_style_param_tool(
        fake_get_temperature_with_new_union_style_param_tool,
    ):
>       libertai_tool = Tool.from_function(
            fake_get_temperature_with_new_union_style_param_tool
        )

tests/tools/test_function_tools.py:12: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
libertai_agents/interfaces/tools.py:35: in from_function
    args_schema=get_json_schema(function),
../../../../..../pypoetry/virtualenvs/libertai-agents-cGTlWRjZ-py3.10/lib/python3.10.../transformers/utils/chat_template_utils.py:322: in get_json_schema
    json_schema = _convert_type_hints_to_json_schema(func)
../../../../..../pypoetry/virtualenvs/libertai-agents-cGTlWRjZ-py3.10/lib/python3.10.../transformers/utils/chat_template_utils.py:165: in _convert_type_hints_to_json_schema
    properties[param_name] = _parse_type_hint(param_type)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

hint = str | None

    def _parse_type_hint(hint: str) -> Dict:
        origin = get_origin(hint)
        args = get_args(hint)
    
        if origin is None:
            try:
                return _get_json_schema_type(hint)
            except KeyError:
                raise TypeHintParsingException(
                    "Couldn't parse this type hint, likely due to a custom class or object: ", hint
                )
    
        elif origin is Union:
            # Recurse into each of the subtypes in the Union, except None, which is handled separately at the end
            subtypes = [_parse_type_hint(t) for t in args if t is not type(None)]
            if len(subtypes) == 1:
                # A single non-null type can be expressed directly
                return_dict = subtypes[0]
            elif all(isinstance(subtype["type"], str) for subtype in subtypes):
                # A union of basic types can be expressed as a list in the schema
                return_dict = {"type": sorted([subtype["type"] for subtype in subtypes])}
            else:
                # A union of more complex types requires "anyOf"
                return_dict = {"anyOf": subtypes}
            if type(None) in args:
                return_dict["nullable"] = True
            return return_dict
    
        elif origin is list:
            if not args:
                return {"type": "array"}
            else:
                # Lists can only have a single type argument, so recurse into it
                return {"type": "array", "items": _parse_type_hint(args[0])}
    
        elif origin is tuple:
            if not args:
                return {"type": "array"}
            if len(args) == 1:
                raise TypeHintParsingException(
                    f"The type hint {str(hint).replace('typing.', '')} is a Tuple with a single element, which "
                    "we do not automatically convert to JSON schema as it is rarely necessary. If this input can contain "
                    "more than one element, we recommend "
                    "using a List[] type instead, or if it really is a single element, remove the Tuple[] wrapper and just "
                    "pass the element directly."
                )
            if ... in args:
                raise TypeHintParsingException(
                    "Conversion of '...' is not supported in Tuple type hints. "
                    "Use List[] types for variable-length"
                    " inputs instead."
                )
            return {"type": "array", "prefixItems": [_parse_type_hint(t) for t in args]}
    
        elif origin is dict:
            # The JSON equivalent to a dict is 'object', which mandates that all keys are strings
            # However, we can specify the type of the dict values with "additionalProperties"
            out = {"type": "object"}
            if len(args) == 2:
                out["additionalProperties"] = _parse_type_hint(args[1])
            return out
    
>       raise TypeHintParsingException("Couldn't parse this type hint, likely due to a custom class or object: ", hint)
E       transformers.utils.chat_template_utils.TypeHintParsingException: ("Couldn't parse this type hint, likely due to a custom class or object: ", str | None)

../../../../..../pypoetry/virtualenvs/libertai-agents-cGTlWRjZ-py3.10/lib/python3.10.../transformers/utils/chat_template_utils.py:150: TypeHintParsingException

To view more test analytics, go to the Test Analytics Dashboard
📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant