How to catch a Fatal

[dix75]

Hi all!
I have a code.
func xxx (x int) { if (x != 1) { log.Fatalf("Fatal msg") } }

Now I can't catch it, it'd be great to catch it like this.

func xxx () { assert.NotFatals(t, func(){ xxx(1) }) // and some other functions, like PanicsWithError }

[boyan-soubachov]

I'm not sure I understand, is the function being recursed?

Do you mind changing the example to be a bit more specific/realistic? I'd be happy to help out

[NeilW]

You can do it, but it is a bit involved - as Fatal calls os.Exit()

func testOsExit(t *testing.T, funcName string, testFunc func(*testing.T)) {
	if os.Getenv(funcName) == "1" {
		testFunc(t)
		return
	}
	cmd := exec.Command(os.Args[0], "-test.run="+funcName)
	cmd.Env = append(os.Environ(), funcName+"=1")
	err := cmd.Run()
	if e, ok := err.(*exec.ExitError); ok && !e.Success() {
		return
	}
	t.Fatal("subprocess ran successfully, want non-zero exit status")
}

func TestxxxFailure(t *testing.T) {
	testOsExit(t, "TestxxxFailure", func(t *testing.T) {
                xxx(2)
        })
}

What this does is patch the environment with a env variable matching the test function name, and then forks the test process running just that test function name again in the sub-process if the env variable is missing. If the env variable is there, then it knows it is running in the sub-proces and runs the actual test that should call Os.Exit with an error value.

It would be handy if testify could wrap this pattern somehow.

[glesica]

I'm not sure we want to make this a supported pattern. In my mind, if an error is "fatal", the program should terminate (which appears to be what the log package does), so it would be better to test those cases at a higher level (functional automation tests) which Testify isn't really designed for.

[NeilW]

That's into philosophical arguments about testing. Subprocess testing has been part of go test lore since the start. The code above is adapted from a 2014 presentation by Andrew Gerrand. https://talks.golang.org/2014/testing.slide#1

Having said that, the subprocess test doesn't update the coverage data - so you can never get the 100% coverage you're looking for.

[glesica]

I'm not saying I have any objection to that kind of testing, just that Testify isn't really otherwise designed to support it, so this would represent an expansion of Testify's "scope". If that's where people want to go, that's fine with me.

[NefixEstrada]

Hello! I also use this funcionality. What I do, is create a custom type and embed suite.Suite in it:

type Test struct {
	suite.Suite
}

After that, I just implement the functionality:

func (t *Test) Exits(f func()) {
	if os.Getenv("ASSERT_EXISTS_"+t.T().Name()) == "1" {
		f()
		return
	}

	cmd := exec.Command(os.Args[0], "-test.run="+t.T().Name())
	cmd.Env = append(os.Environ(), "ASSERT_EXISTS_"+t.T().Name()+"=1")
	err := cmd.Run()

	if e, ok := err.(*exec.ExitError); ok && !e.Success() {
		return
	}

	t.Fail("expecting unsuccessful exit")
}

I also thing it would be a really nice addition to testify. After all, I do use it for unit testing

[Goldziher]

This would be a very nice addition, I agree

[wkdwyerNIH]

although getting test coverage "credit" for unit tests that employ Gerrand's "BE_CRASHER" technique would be more satisfying, code annotations could help in code reviews, and it would be easy for reviewers to verify. Example:
func MustBeGT3(x int) {
if x < 4 {
log.Fatal().Msg("x is < 4!; see ya") // covered in Test_using_Gerrand_MustBeGT3
}
}

[Antonboom]

Usually application with nice architecture have single exit point, in package main:

func main() {
   if err := run(); err != nil {
       log.Fatal(err)
   }
}

The rest code could be easily [moved in separate package and] tested without "Fatal catching".

just that Testify isn't really otherwise designed to support it

👍

[wkdwyerNIH]

another helpful technique is to, in the app code, declare vars called logFatal and logWarn, and have the app code call logFatal instead of log.Fatal. So logFatal, which is normally in prod set to log.Fatal, can be set to log.Warn in unit tests. e.g.

// in X.go:
var logFatal  =  log.Fatal
var logWarn  =  log.Warn

func F() {
  logFatal(...
}

// in test_X.go:
func Test_F(t *testing.T) {
 old := logFatal
 logFatal = logWarn
 defer func() {
                logFatal = old
  }()
 // various tests for func F()...