fxtest: Add RequireRun helper

[aureliar8]

fxtest provides both a RequireStart() and RequireStop() helper to test that an fx application can start or stop without errors.
But there's currently no way to verify that an fx application can Run() without error.

This PR adds a RequireRun() method on fxtest.App that starts, runs and stops the app, and fails the test if either start or stop failed, or if run returned a non zero exit code

[CLAassistant]

All committers have signed the CLA.

[abhinav]

This feels like it would be easy to misuse.

For most normal applications, RequireRun will block indefinitely—until they try to cancel the test.

The idea of finishing running is true only for applications that use fx.Shutdowner to stop after doing something. Cases like that can currently be tested by something the following:

app := fxtest.New(t, ...)
defer app.RequireStart().RequireStop()
<-app.Wait()

This is equivalent to the proposed RequireRun function and only one extra line.
I'd like us to consider the risk of confusion of the indefinitely blocking RequireRun function versus one extra line with the current available APIs.

(If we were to add RequireRun, it would absolutely have to require a timeout argument so that there was an upper bound on the Wait().)

[aureliar8]

I don't think the misuse issue is that bad here.

Yes one could get confused and use RequireRun() on a never-ending fx app, but it would just make the test timeout and it'd be clear that RequireRun() makes the test timeout and that it was misused. I think that with a warning in the function documentation it's not too much of an issue

[sywhang]

but it would just make the test timeout and it'd be clear that RequireRun() makes the test timeout and that it was misused

I'm not sure if I agree with this -- how would it be "clear" that the test is timing out because of RequireRun?

[aureliar8]

but it would just make the test timeout and it'd be clear that RequireRun() makes the test timeout and that it was misused

I'm not sure if I agree with this -- how would it be "clear" that the test is timing out because of RequireRun?

I added the following code in fxtest/app_test.go to emulate a misuse of RequireSart():

t.Run("Will timeout", func(t *testing.T) {
		t.Parallel()
		spy := newTB()

		New(
			spy,
			fx.Invoke(func(lc fx.Lifecycle, s fx.Shutdowner) {}),
		).RequireRun() // <- This is misuse of RequireRun on a fx app that doesn't call Shutdown()
	})

Running the tests gives a stacktrace that points to a RequireRun (but I agree it requires a bit of effort to read)

❯ go test ./fxtest 
[Fx] HOOK OnStart		go.uber.org/fx/fxtest.TestLifecycle_OptionalT.func1.2() executing (caller: go.uber.org/fx/fxtest.TestLifecycle_OptionalT.func1)
[Fx] HOOK OnStart		go.uber.org/fx/fxtest.TestLifecycle_OptionalT.func1.2() called by go.uber.org/fx/fxtest.TestLifecycle_OptionalT.func1 ran successfully in 302ns
[Fx] HOOK OnStop		go.uber.org/fx/fxtest.TestLifecycle_OptionalT.func1.3() executing (caller: go.uber.org/fx/fxtest.TestLifecycle_OptionalT.func1)
[Fx] HOOK OnStop		go.uber.org/fx/fxtest.TestLifecycle_OptionalT.func1.3() called by go.uber.org/fx/fxtest.TestLifecycle_OptionalT.func1 ran successfully in 251ns
[Fx] HOOK OnStart		go.uber.org/fx/fxtest.TestLifecycle_OptionalT.func2.1() executing (caller: go.uber.org/fx/fxtest.TestLifecycle_OptionalT.func2)
[Fx] HOOK OnStart		go.uber.org/fx/fxtest.TestLifecycle_OptionalT.func2.1() called by go.uber.org/fx/fxtest.TestLifecycle_OptionalT.func2 failed in 217ns: great sadness
lifecycle didn't start cleanly: great sadness
panic: test timed out after 1m0s
	running tests:
		TestApp/Will_timeout (1m0s)

goroutine 102 [running]:
testing.(*M).startAlarm.func1()
	/usr/local/go/src/testing/testing.go:2373 +0x385
created by time.goFunc
	/usr/local/go/src/time/sleep.go:215 +0x2d

goroutine 1 [chan receive]:
testing.tRunner.func1()
	/usr/local/go/src/testing/testing.go:1651 +0x49b
testing.tRunner(0xc00012eb60, 0xc000177bb8)
	/usr/local/go/src/testing/testing.go:1696 +0x12d
testing.runTests(0xc000126450, {0xad63a0, 0x6, 0x6}, {0x416605?, 0x0?, 0xaddac0?})
	/usr/local/go/src/testing/testing.go:2166 +0x43d
testing.(*M).Run(0xc000135040)
	/usr/local/go/src/testing/testing.go:2034 +0x64a
go.uber.org/goleak.VerifyTestMain({0x87c560?, 0xc000135040?}, {0x0, 0x0, 0x0})
	/home/aurelien/go/pkg/mod/go.uber.org/goleak@v1.2.0/testmain.go:53 +0x5a
go.uber.org/fx/fxtest.TestMain(...)
	/home/aurelien/code/fx/fxtest/lifecycle_test.go:321
main.main()
	_testmain.go:57 +0xba

goroutine 20 [chan receive]:
testing.tRunner.func1()
	/usr/local/go/src/testing/testing.go:1651 +0x49b
testing.tRunner(0xc00012ed00, 0x8191e8)
	/usr/local/go/src/testing/testing.go:1696 +0x12d
created by testing.(*T).Run in goroutine 1
	/usr/local/go/src/testing/testing.go:1743 +0x390

goroutine 14 [select]:
go.uber.org/fx.(*signalReceivers).relayer(0xc000000140)
	/home/aurelien/code/fx/signal.go:84 +0xa5
created by go.uber.org/fx.(*signalReceivers).Start in goroutine 71
	/home/aurelien/code/fx/signal.go:112 +0x1b8

goroutine 41 [syscall]:
os/signal.signal_recv()
	/usr/local/go/src/runtime/sigqueue.go:152 +0x29
os/signal.loop()
	/usr/local/go/src/os/signal/signal_unix.go:23 +0x13
created by os/signal.Notify.func1.1 in goroutine 71
	/usr/local/go/src/os/signal/signal.go:151 +0x1f

goroutine 71 [chan receive]:
go.uber.org/fx/fxtest.(*App).RequireRun(0xc000012408)
	/home/aurelien/code/fx/fxtest/app.go:86 +0x137
go.uber.org/fx/fxtest.TestApp.func6(0xc000310820)
	/home/aurelien/code/fx/fxtest/app_test.go:178 +0x159
testing.tRunner(0xc000310820, 0x819450)
	/usr/local/go/src/testing/testing.go:1690 +0xf4
created by testing.(*T).Run in goroutine 20
	/usr/local/go/src/testing/testing.go:1743 +0x390
FAIL	go.uber.org/fx/fxtest	60.050s
FAIL