Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

=== short test summary info === messages are not useful for ExceptionGroups #12943

Open
Zac-HD opened this issue Nov 6, 2024 · 4 comments · May be fixed by #12975
Open

=== short test summary info === messages are not useful for ExceptionGroups #12943

Zac-HD opened this issue Nov 6, 2024 · 4 comments · May be fixed by #12975
Labels
topic: reporting related to terminal output and user-facing messages and errors type: enhancement new feature or API change, should be merged into features branch

Comments

@Zac-HD
Copy link
Member

Zac-HD commented Nov 6, 2024

If there is only a single leaf exception, at any depth, I suggest we produce the short summary message for that leaf exception, and append [single exception in an ExceptionGroup] to the message.

@Zac-HD Zac-HD added type: enhancement new feature or API change, should be merged into features branch topic: reporting related to terminal output and user-facing messages and errors labels Nov 6, 2024
@nicoddemus
Copy link
Member

bikeshedding: [single exception in an ExceptionGroup] to make it clear it is a single exception, and avoid someone thinking "ok and about the other exceptions in that exception group"?

@jakkdl
Copy link
Member

jakkdl commented Nov 18, 2024

One problem with appending the [single exc...] is it getting lost if the error message is very long and gets truncated, making it impossible to distinguish a wrapped exception from a non-wrapped exception.

FAILED foo.py::test_bar - ValueError: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...

so unless we want custom logic for the truncation in _format_trimmed/_get_line_with_reprcrash_message it's probably better to prepend any message.

(you don't get truncation if you run in CI or with -vv, but day-to-day users will encounter it)

@Zac-HD
Copy link
Member Author

Zac-HD commented Nov 18, 2024

I think it's okay to truncate here; I'd rather lose the note than the actual message at least - and the group will still be visible if you go look at the specific test.

@jakkdl
Copy link
Member

jakkdl commented Nov 22, 2024

I think there's cases where losing the message would be preferably to losing the note, e.g. if you're writing anything equivalent to

with pytest.raises(ValueError):
    raise ExceptionGroup("", [ValueError("."*100])

then the reason it fails is because it's in a group. And if we make the note somewhat brief it's not going to take up much space, and the reader can quickly skip it when it's been read a million times.

and with a leading note we can easily get a format that allows printing multiple exceptions in a logical way:

[in ExceptionGroup]: ValueError("my message"), TypeError("my other message")

Though with -vv we might want to display the full exceptiongroup, nested and all.
While getting exactly >1 sub-exceptions might be less common than exactly one, the message ExceptionGroup: bbbbbbbbbb (2 sub-exceptions) is about as useless as with one sub-exception and we could probably print both sub-exceptions without any issues.

I suppose truncation introduces another issue here, where you could lose info about the number of exceptions, but we can add that back to the note

[2 in ExceptionGroup]: ValueError("my message"), TypeError("my other message")

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic: reporting related to terminal output and user-facing messages and errors type: enhancement new feature or API change, should be merged into features branch
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants