golang/go

cmd/test2json: missing pass/fail action on individual benchmark

Open

#61,767 opened on Aug 4, 2023

View on GitHub
 (2 comments) (1 reaction) (0 assignees)Go (19,008 forks)batch import
NeedsInvestigationhelp wanted

Repository metrics

Stars
 (133,883 stars)
PR merge metrics
 (No merged PRs in 30d)

Description

What version of Go are you using (go version)?

What operating system and processor architecture are you using (go env)?

Not relevant - omitting

What did you do?

File foo_test.go:

package foo_x_test

import (
        "testing"
)

func TestFoo(t *testing.T) {
        t.Logf("hello")
}

func fib(n int) int {
        if n < 2 {
                return n
        }
        return fib(n-1) + fib(n-2)
}

func BenchmarkFib10(b *testing.B) {
        for n := 0; n < b.N; n++ {
                fib(10)
        }
}

Run go test -json -bench=. foo_test.go

Result:

{"Time":"2023-08-04T12:37:32.630848007-07:00","Action":"start","Package":"command-line-arguments"}
{"Time":"2023-08-04T12:37:32.674050117-07:00","Action":"run","Package":"command-line-arguments","Test":"TestFoo"}
{"Time":"2023-08-04T12:37:32.674089527-07:00","Action":"output","Package":"command-line-arguments","Test":"TestFoo","Output":"=== RUN   TestFoo\n"}
{"Time":"2023-08-04T12:37:32.674121806-07:00","Action":"output","Package":"command-line-arguments","Test":"TestFoo","Output":"    foo_test.go:8: hello\n"}
{"Time":"2023-08-04T12:37:32.674136934-07:00","Action":"output","Package":"command-line-arguments","Test":"TestFoo","Output":"--- PASS: TestFoo (0.00s)\n"}
{"Time":"2023-08-04T12:37:32.674152436-07:00","Action":"pass","Package":"command-line-arguments","Test":"TestFoo","Elapsed":0}
{"Time":"2023-08-04T12:37:32.67656493-07:00","Action":"output","Package":"command-line-arguments","Output":"goos: linux\n"}
{"Time":"2023-08-04T12:37:32.676585644-07:00","Action":"output","Package":"command-line-arguments","Output":"goarch: amd64\n"}
{"Time":"2023-08-04T12:37:32.676599219-07:00","Action":"output","Package":"command-line-arguments","Output":"cpu: Intel(R) Xeon(R) W-2135 CPU @ 3.70GHz\n"}
{"Time":"2023-08-04T12:37:32.676610204-07:00","Action":"run","Package":"command-line-arguments","Test":"BenchmarkFib10"}
{"Time":"2023-08-04T12:37:32.676619064-07:00","Action":"output","Package":"command-line-arguments","Test":"BenchmarkFib10","Output":"=== RUN   BenchmarkFib10\n"}
{"Time":"2023-08-04T12:37:32.676630118-07:00","Action":"output","Package":"command-line-arguments","Test":"BenchmarkFib10","Output":"BenchmarkFib10\n"}
{"Time":"2023-08-04T12:37:34.248814597-07:00","Action":"output","Package":"command-line-arguments","Test":"BenchmarkFib10","Output":"BenchmarkFib10-6   \t 1922632\t       602.9 ns/op\n"}
{"Time":"2023-08-04T12:37:34.248879794-07:00","Action":"output","Package":"command-line-arguments","Output":"PASS\n"}
{"Time":"2023-08-04T12:37:34.250175678-07:00","Action":"output","Package":"command-line-arguments","Output":"ok  \tcommand-line-arguments\t1.619s\n"}
{"Time":"2023-08-04T12:37:34.250212919-07:00","Action":"pass","Package":"command-line-arguments","Elapsed":1.619}

What did you expect to see?

Note that TestFoo gets a line with an individual test result --

{"Time":"2023-08-04T12:37:32.674152436-07:00","Action":"pass","Package":"command-line-arguments","Test":"TestFoo","Elapsed":0}

This is missing for BenchmarkFib10.

Expected to see:

{"Time":"<>","Action":"pass","Package":"command-line-arguments","Test":"BenchmarkFib10","Elapsed":0}

After Fib10 passes.

Really, this likely means that BenchmarkFib10 needs to be framed with a --- (PASS|SKIP|FAIL): BenchmarkFib10 (0.00s)\n at the end when -test.v=test2json is passed to the test binary as well.

(Previous work in this area fixed most of the test2json issues I've seen, cc @rsc -- https://go-review.googlesource.com/c/go/+/443596)

Contributor guide