Allow fail to work in try/catch
#664
Conversation
|
|
Having skipped tests doesn't make much sense. We probably need some sort of integration test which spawns Jest or something for this case |
|
I've never done something like that before but am happy to put in the work to make it happen. Some initial ideas...
Which approach would you recommend I pursue? |
|
I'm also looking for this functionality but I don't think this solution works as-is.
I think what you wanted instead was If I checkout your PR and change the |
|
@liamjones The docs seem to suggest that
Since this PR directly reports a test as failing instead of throwing an exception for Jest to catch, Notice that by changing |
|
After digging into all this some more, I figured out a way to use a custom jest reporter to detect the expected test failures and flip them to passing. The custom reporter also marks these "expected failing" tests as actually failing if they unexpectedly pass. The solution is decently complex, so I'm happy to answer questions about how it works and implement suggestions to make the implementation more approachable. |
To verify that the new `fail` matcher works correctly, we need a way to check if the tests failed as expected or not. Typically one would use `test.failing` for this purpose, but that only works if the test threw an exception (e.g. JestException). The `fail` matcher does not do this so that it can still work inside `catch` blocks. Instead, we have to hook into the test reporting and mutate the test results for our expected test failures prior to usage by other test reporters. This commit creates a custom test reporter which detects tests run in the file `fail.test.js` (yes the name is hard-coded) and flips the results of tests inside a `.fail` describe block (i.e. our expected failures). - If the tests failed (expected) we flip the result to a pass. - If the tests passed (unexpected) we flip the result to a fail. The custom reporter also handles the logic for updating the counts for failing test suites so that later reporters reflect the actual test results correctly.
|
@SimenB This PR is ready for another maintainer review. P.S. I see per your GitHub status that you are currently "busy"/"slow to respond", so no rush! |
What
This PR allows calls to
expect().fail(message)to fail a test, even when theexpect().fail()is executed inside of a try/catch block.Why
closes #663
This functionality has been desired in Jest since the original
failfunction was removed from the imported Jasmine scope.Throwing an error is semantically different from failing a test, and attempting to equate the two makes it much harder to test code which uses try/catch blocks.
Notes
I am unaware of any way for Jest to report an "expected failure" similar topytestsxfaildecorator. The closest I could find was prefixing thetestasxtestto skip it. Deleting thosexprefixes allows one to manually inspect the test failure messages to verify they line up with the expected behavior.I am happy to edit the PR if there's a better way to record "expected failures".EDIT: The latest commit on the PR now validates that the tests fail as expected instead of skipping them.
Housekeeping