Flytekit cant raise exception for Pydantic v2 ValidationError

[jganbat-fn]

Hello Flyte folks, so I am updating our flytekit and we recently migrated to pydantic v2 on our model validation. however I am getting the following.
Test runs like below
• There is a unit test with pytest.raises that should raise ValidationError
• This unit test calls flyte task that includes the pydantic model validation
• However the test does not return the underlying ValidationError rather return the following TypeError: No constructor defined .
I can see the underlying ValidationError exception but also see During handling of the above exception, another exception occurred:

This TypeError happens at flytekit/exceptions/scopes.py:user_entry_point at this line216

                          raise type(exc)(f"Error encountered while executing '{fn_name}':\n  {exc}") from exc
E                   TypeError: No constructor defined

it looks like pydantic v2 we need to raise specific error like https://github.com/pydantic/pydantic/issues/8026|this.

I think we can just raise the original exception here right?

except Exception as exc:
    raise exc

[dosubot]

🤖

Thanks for bringing this to our attention. It seems like the issue is related to how Flytekit is handling exceptions raised by Pydantic v2. To better understand and address this issue, could you please provide some additional information?

1. Could you share the version of Flytekit you're currently using?
2. Could you provide a minimal reproducible example of the Flyte task that's causing this issue?
3. Could you share the full stack trace of the error you're encountering?

This information will help us understand the problem more clearly and find a solution. Thanks for your cooperation. Dosu

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[davidmirror-ops]

@jganbat-fn could you verify if it fails even using what's described here?

[dosubot]

I'm sorry, but I can't perform actions such as running commands or accessing external links, including GitHub repositories or pull requests. Therefore, I can't directly verify the issue mentioned or the contents of the pull request linked by the maintainer, davidmirror-ops.

However, based on the conversation and the typical workflow for addressing such issues, I can provide a synthesized response that might be helpful:

---

Hello @jganbat-fn,

Thank you for bringing this to our attention and for the detailed explanation of the issue you're encountering with Flytekit and Pydantic v2. It seems like the core of the problem lies in how exceptions are being re-raised within Flytekit, particularly in the context of Pydantic's validation errors.

As @davidmirror-ops mentioned, there's a pull request (#2217) that might address the issue you're experiencing. While I can't directly verify the contents of this PR or its applicability to your specific problem, it's worth checking if the changes proposed there align with the solution you're looking for. The PR might contain modifications that improve compatibility with Pydantic v2 or adjust how exceptions are handled within Flytekit.

If you haven't already, I recommend testing your code against the branch associated with that pull request to see if it resolves the TypeError you're encountering. This could provide a temporary workaround until an official fix is released.

In the meantime, your suggestion to simply re-raise the original exception is interesting and might be a viable short-term solution. However, it's important to consider any potential side effects this change could have, especially in terms of error handling and logging within Flytekit. I suggest opening a discussion or issue in the Flytekit GitHub repository to propose this change and gather feedback from the maintainers and community.

Thank you for your patience and for contributing to the improvement of Flytekit. Your involvement helps make the project better for everyone.

Best,
Dosu

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.