golang/go

net/http: benchmarks fail on sierra with 'too many open files'

Open

#18,753 opened on Jan 22, 2017

View on GitHub
 (16 comments) (0 reactions) (0 assignees)Go (19,008 forks)batch import
NeedsFixTestinghelp wanted

Repository metrics

Stars
 (133,883 stars)
PR merge metrics
 (No merged PRs in 30d)

Description

Sample run excerpt:

BenchmarkCookieString-8                      	 2000000	       900 ns/op	     384 B/op	       3 allocs/op
BenchmarkReadSetCookies-8                    	  500000	      3703 ns/op	     960 B/op	      17 allocs/op
BenchmarkReadCookies-8                       	  300000	      4948 ns/op	    2008 B/op	      16 allocs/op
BenchmarkHeaderWriteSubset-8                 	 3000000	       545 ns/op	       0 B/op	       0 allocs/op
BenchmarkReadRequestChrome-8                 	  500000	      3187 ns/op	 191.68 MB/s	    1809 B/op	      14 allocs/op
BenchmarkReadRequestCurl-8                   	 1000000	      1663 ns/op	  46.90 MB/s	     880 B/op	       9 allocs/op
BenchmarkReadRequestApachebench-8            	 1000000	      1670 ns/op	  49.10 MB/s	     883 B/op	       9 allocs/op
BenchmarkReadRequestSiege-8                  	 1000000	      2161 ns/op	  69.87 MB/s	     965 B/op	      11 allocs/op
BenchmarkReadRequestWrk-8                    	 1000000	      1682 ns/op	  23.78 MB/s	     832 B/op	       7 allocs/op
BenchmarkClientServer-8                      	   20000	     71011 ns/op	    4962 B/op	      67 allocs/op
BenchmarkClientServerParallel4-8             	  100000	     15276 ns/op	    4937 B/op	      64 allocs/op
BenchmarkClientServerParallel64-8            	2017/01/22 11:19:46 http: Accept error: accept tcp 127.0.0.1:60387: accept: too many open files; retrying in 5ms
2017/01/22 11:19:46 http: Accept error: accept tcp 127.0.0.1:60387: accept: too many open files; retrying in 5ms
2017/01/22 11:19:46 http: Accept error: accept tcp 127.0.0.1:60387: accept: too many open files; retrying in 10ms
2017/01/22 11:19:46 http: Accept error: accept tcp 127.0.0.1:60387: accept: too many open files; retrying in 20ms
2017/01/22 11:19:46 http: Accept error: accept tcp 127.0.0.1:60387: accept: too many open files; retrying in 40ms
2017/01/22 11:19:46 http: Accept error: accept tcp 127.0.0.1:60387: accept: too many open files; retrying in 80ms
2017/01/22 11:19:46 http: Accept error: accept tcp 127.0.0.1:60387: accept: too many open files; retrying in 5ms
   10000	    135870 ns/op	    4749 B/op	      63 allocs/op
--- BENCH: BenchmarkClientServerParallel64-8
	serve_test.go:4494: Get: Get http://127.0.0.1:60387: dial tcp 127.0.0.1:60387: socket: too many open files
	serve_test.go:4494: Get: Get http://127.0.0.1:60387: dial tcp 127.0.0.1:60387: socket: too many open files
	serve_test.go:4494: Get: Get http://127.0.0.1:60387: dial tcp 127.0.0.1:60387: socket: too many open files
	serve_test.go:4494: Get: Get http://127.0.0.1:60387: dial tcp 127.0.0.1:60387: socket: too many open files
	serve_test.go:4494: Get: Get http://127.0.0.1:60387: dial tcp 127.0.0.1:60387: socket: too many open files
	serve_test.go:4494: Get: Get http://127.0.0.1:60387: dial tcp 127.0.0.1:60387: socket: too many open files
	serve_test.go:4494: Get: Get http://127.0.0.1:60387: dial tcp 127.0.0.1:60387: socket: too many open files
	serve_test.go:4494: Get: Get http://127.0.0.1:60387: dial tcp 127.0.0.1:60387: socket: too many open files
	serve_test.go:4494: Get: Get http://127.0.0.1:60387: dial tcp 127.0.0.1:60387: socket: too many open files
	serve_test.go:4494: Get: Get http://127.0.0.1:60387: dial tcp 127.0.0.1:60387: socket: too many open files
	... [output truncated]
BenchmarkClientServerParallelTLS4-8          	  100000	     15757 ns/op	    4945 B/op	      64 allocs/op
BenchmarkClientServerParallelTLS64-8         	2017/01/22 11:19:49 http: Accept error: accept tcp 127.0.0.1:60976: accept: too many open files; retrying in 5ms
2017/01/22 11:19:49 http: Accept error: accept tcp 127.0.0.1:60976: accept: too many open files; retrying in 5ms
2017/01/22 11:19:49 http: Accept error: accept tcp 127.0.0.1:60976: accept: too many open files; retrying in 10ms
2017/01/22 11:19:49 http: Accept error: accept tcp 127.0.0.1:60976: accept: too many open files; retrying in 20ms
2017/01/22 11:19:49 http: Accept error: accept tcp 127.0.0.1:60976: accept: too many open files; retrying in 40ms
2017/01/22 11:19:49 http: Accept error: accept tcp 127.0.0.1:60976: accept: too many open files; retrying in 5ms
2017/01/22 11:19:50 http: Accept error: accept tcp 127.0.0.1:60976: accept: too many open files; retrying in 5ms
2017/01/22 11:19:50 http: Accept error: accept tcp 127.0.0.1:60976: accept: too many open files; retrying in 5ms
2017/01/22 11:19:50 http: Accept error: accept tcp 127.0.0.1:60976: accept: too many open files; retrying in 5ms
2017/01/22 11:19:50 http: Accept error: accept tcp 127.0.0.1:60976: accept: too many open files; retrying in 5ms
2017/01/22 11:19:50 http: Accept error: accept tcp 127.0.0.1:60976: accept: too many open files; retrying in 5ms
2017/01/22 11:19:50 http: Accept error: accept tcp 127.0.0.1:60976: accept: too many open files; retrying in 5ms
    5000	    475368 ns/op	    6175 B/op	      80 allocs/op
--- BENCH: BenchmarkClientServerParallelTLS64-8
	serve_test.go:4494: Get: Get https://127.0.0.1:60976: dial tcp 127.0.0.1:60976: socket: too many open files
	serve_test.go:4494: Get: Get https://127.0.0.1:60976: dial tcp 127.0.0.1:60976: socket: too many open files
	serve_test.go:4494: Get: Get https://127.0.0.1:60976: dial tcp 127.0.0.1:60976: socket: too many open files
	serve_test.go:4494: Get: Get https://127.0.0.1:60976: dial tcp 127.0.0.1:60976: socket: too many open files
	serve_test.go:4494: Get: Get https://127.0.0.1:60976: dial tcp 127.0.0.1:60976: socket: too many open files
	serve_test.go:4494: Get: Get https://127.0.0.1:60976: dial tcp 127.0.0.1:60976: socket: too many open files
	serve_test.go:4494: Get: Get https://127.0.0.1:60976: dial tcp 127.0.0.1:60976: socket: too many open files
	serve_test.go:4494: Get: Get https://127.0.0.1:60976: dial tcp 127.0.0.1:60976: socket: too many open files
	serve_test.go:4494: Get: Get https://127.0.0.1:60976: dial tcp 127.0.0.1:60976: socket: too many open files
	serve_test.go:4494: Get: Get https://127.0.0.1:60976: dial tcp 127.0.0.1:60976: socket: too many open files
	... [output truncated]
BenchmarkServer-8                            	   20000	     75523 ns/op	    2341 B/op	      21 allocs/op
BenchmarkClient-8                            	   20000	     72795 ns/op	    3351 B/op	      50 allocs/op

Contributor guide

Research direction
Investigate why the benchmarks are hitting the system's open file limit on macOS Sierra. Check if the test creates too many connections without proper cleanup, or if the system's ulimit is too low. Consider increasing the file descriptor limit or ensuring connections are closed properly.
Tech stack
go
Domain
backend
Issue type
Bug
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
2
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-3 hours
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
Go
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
40