Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bug: red traffic light dots for tests that passed #67

Open
jotelfor opened this issue May 29, 2024 · 0 comments
Open

bug: red traffic light dots for tests that passed #67

jotelfor opened this issue May 29, 2024 · 0 comments
Assignees
@tonykchen
Labels
bug Something isn't working

Comments

@jotelfor
Copy link

Testing the "main" branch (not the current release).

In agenteval_summary.md, the Pass rate is non-zero (some of the tests passed).
But... a red dot is shown for all of the tests.
Expected behaviour is that some of the tests will have green dots, since they passed.
@jotelfor jotelfor changed the title bug: traffic light dots red for tests that passed bug: red traffic light dots for tests that passed May 29, 2024
@tonykchen tonykchen added the bug Something isn't working label May 30, 2024
@tonykchen tonykchen self-assigned this May 30, 2024
rvernica added a commit to rvernica/agent-evaluation that referenced this issue Nov 3, 2024
@rvernica
Fix test result attribute in summary template
85a3554 
* Fix awslabs#67
* `success` is not an attribute of the [`TestResult`](https://github.com/awslabs/agent-evaluation/blob/83a51af4d6f84469c0081aaf28537857087f1bfd/src/agenteval/test/test_result.py#L26) object. Replace it with `passed`
@rvernica rvernica mentioned this issue Nov 3, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tonykchen tonykchen

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@tonykchen @jotelfor