Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(agents-api): Add some custom error messages to base_evaluate #758

Merged
merged 9 commits into from
Oct 28, 2024

Conversation

Ahmad-mtos
Copy link
Contributor

@Ahmad-mtos Ahmad-mtos commented Oct 28, 2024

I added some error messages in base_evaluate to help users identify some specific errors.


Important

Adds custom error messages to base_evaluate in base_evaluate.py for Jinja template misuse and variable name misspellings, using thefuzz for suggestions.

  • Error Handling:
    • Introduces EvaluateError class in base_evaluate.py to provide custom error messages.
    • Detects Jinja template misuse and suggests using Python expressions.
    • Uses thefuzz to suggest correct variable names when NameNotDefined error occurs.
  • Dependencies:
    • Adds thefuzz to pyproject.toml for fuzzy string matching.

This description was created by Ellipsis for 5a25b84. It will automatically update as commits are pushed.

Copy link
Contributor

sweep-ai bot commented Oct 28, 2024

Hey @Ahmad-mtos, here is an example of how you can ask me to improve this pull request:

@Sweep Add unit tests for the `EvaluateError` class that verify:
1. The basic error message is preserved
2. The Jinja template suggestion is added when "unhashable" appears in the error
3. The variable name suggestion is added when a `NameNotDefined` error occurs with a similar variable name

📖 For more information on how to use Sweep, please read our documentation.

Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me! Reviewed everything up to 5a25b84 in 11 seconds

More details
  • Looked at 94 lines of code in 2 files
  • Skipped 1 files when reviewing.
  • Skipped posting 2 drafted comments based on config settings.
1. agents-api/agents_api/activities/task_steps/base_evaluate.py:17
  • Draft comment:
    Use f-strings for error message construction for consistency and clarity.
        error_message = f"{error.message}" if hasattr(error, "message") else str(error)
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The error message construction in EvaluateError should use f-strings for consistency and clarity.
2. agents-api/agents_api/activities/task_steps/base_evaluate.py:28
  • Draft comment:
    Consider lowering the fuzz.ratio threshold from 90.0 to a lower value for better detection of misspelled variable names.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The fuzz.ratio threshold of 90.0 might be too high for detecting misspelled variable names. A lower threshold could be more effective.

Workflow ID: wflow_3UDKIxxVzycbTP5s


You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

@creatorrr
Copy link
Contributor

@Ahmad-mtos take a look at the failing github action:

error: Your local changes to the following files would be overwritten by checkout:
	agents-api/agents_api/activities/task_steps/base_evaluate.py
Please commit your changes or stash them before you switch branches.
Aborting

This means that the linting process failed, can you please run poetry run poe check and commit the result?

@creatorrr creatorrr merged commit 7892cd7 into julep-ai:dev Oct 28, 2024
4 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants