golang/go

cmd/internal/test2json: test timeout panic trace is attributed to the running test, even if the test just passed

Open

#57,305 opened on Dec 14, 2022

View on GitHub
 (4 comments) (4 reactions) (0 assignees)Go (19,008 forks)batch import
NeedsInvestigationhelp wanted

Repository metrics

Stars
 (133,883 stars)
PR merge metrics
 (No merged PRs in 30d)

Description

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

What did you do?

I ran go test with json output and a timeout which occurs right as one test is passing.

With the following test file:

Running go test . -json -timeout=1s reproduces the problem sporadically. The behavior is racy. Sometimes the test timeout panic trace is attributed to a test, other times not. And occasionally it's attributed to a test that passed.

What did you expect to see?

I wanted to see the test timeout panic output not attributed to any particular test (no {"Test":"TestHello"}) since there could be multiple tests running.

When adding t.Parallel() to both tests:

Note that I'm unsure if removing {"Test":"TestWorld"} would be problematic in this case, or break any existing tools, but it doesn't intuitively make sense to me that any one test would be attributed this output.

What did you see instead?

I saw two things:

  1. The test outputs PASS but the stack trace is attributed to it after the fact ({"Test":"TestHello"})
  2. The test outputs PASS but Go 1.20rc1 considers it to be running still.

The first case is problematic since tools like gotestsum rely on PASS to ignore future output from a test. The second is more of an inconvenience, however, if tools relied on the output to distinguish problematic tests, the statistics would be skewed.

When adding t.Parallel() to both tests:

Contributor guide