golang/go

bytes: trivial to beat ContainsAny in easy cases

Open

#31,321 opened on Apr 7, 2019

View on GitHub
 (5 comments) (0 reactions) (0 assignees)Go (19,008 forks)batch import
NeedsInvestigationPerformancehelp wanted

Repository metrics

Stars
 (133,883 stars)
PR merge metrics
 (No merged PRs in 30d)

Description

See this CL: https://go-review.googlesource.com/c/go/+/155965

In short, code like bytes.ContainsAny(data, " \t") is easy to read and write, but a simple manual implementation as follows is considerably faster:

any := false
for _, b := range data {
	if b == ' ' || b == '\t' {
		any = true
		break
	}
}

ContainsAny uses IndexAny, which already has three paths; it does nothing for empty chars, it builds an ASCII set if chars is all ASCII and over eight bytes, and otherwise it does a naive double-loop search.

I'm not sure what the right solution here would be. I think adding more special cases to IndexAny would probably be overkill, and even slow down the other cases because of the added code. Making the compiler treat ContainsAny or IndexAny in a special way is also undesirable.

I guess if the compiler was super smart in the future, inlined all the code and realised that our chars are just " \t", it would simply inline that last naive search code. But I imagine the current compiler is far away from that possibility.

This is an unfortunate outcome, because the bytes and strings packages are very intuitive and often the fastest. I'd like to continue relying on them, without having to write my own loops by hand when they start showing up on CPU profiles.

/cc @josharian @randall77 @martisch

Contributor guide