cmd/internal/test2json: test timeout panic trace is attributed to the running test, even if the test just passed
#57,305 opened on Dec 14, 2022
Repository metrics
- Stars
- (133,883 stars)
- PR merge metrics
- (No merged PRs in 30d)
Description
What version of Go are you using (go version)?
Does this issue reproduce with the latest release?
Yes.
What operating system and processor architecture are you using (go env)?
What did you do?
I ran go test with json output and a timeout which occurs right as one test is passing.
With the following test file:
Running go test . -json -timeout=1s reproduces the problem sporadically. The behavior is racy. Sometimes the test timeout panic trace is attributed to a test, other times not. And occasionally it's attributed to a test that passed.
What did you expect to see?
I wanted to see the test timeout panic output not attributed to any particular test (no {"Test":"TestHello"}) since there could be multiple tests running.
When adding t.Parallel() to both tests:
Note that I'm unsure if removing {"Test":"TestWorld"} would be problematic in this case, or break any existing tools, but it doesn't intuitively make sense to me that any one test would be attributed this output.
What did you see instead?
I saw two things:
- The test outputs
PASSbut the stack trace is attributed to it after the fact ({"Test":"TestHello"}) - The test outputs
PASSbut Go 1.20rc1 considers it to be running still.
The first case is problematic since tools like gotestsum rely on PASS to ignore future output from a test. The second is more of an inconvenience, however, if tools relied on the output to distinguish problematic tests, the statistics would be skewed.
When adding t.Parallel() to both tests: