blake2b, blake2s: limit blocks processed at once in assembly - #368
AskAlexSharov wants to merge 2 commits into
Conversation
Problem
GC stop-the-world is slow if a large input is passed to Write or Sum512.
Root cause: hashBlocks() is handed unbounded input, and assembly is not
preemptible.
Stop-the-world (improved)
/sched/pauses/stopping/gc:seconds, the time the runtime spends waiting for every
P to halt, worst pause while hashing a 64 MiB buffer. linux/amd64, EPYC 4344P,
GOMAXPROCS=2. Values are histogram bucket upper bounds, so read them as "<=":
```
before after pauses >1ms
blake2b 58.72ms 66us 3 -> 0
blake2s 83.886ms 164us 3 -> 0
```
```
GOMAXPROCS=2 go test -run='^$' -bench=BenchmarkSTW -count=6 -benchtime=2s ./blake2b/ ./blake2s/
benchstat before.txt after.txt
blake2b │ gcwait-sec/op │ gcwait-sec/op vs base │
STW/Write-2 50625.4µ ± 0% 436.0µ ± 6% -99.14% (p=0.002 n=6)
STW/Sum-2 50440.6µ ± 0% 438.1µ ± 7% -99.13% (p=0.002 n=6)
blake2s │ gcwait-sec/op │ gcwait-sec/op vs base │
STW/Write-2 74260.5µ ± 0% 525.8µ ± 5% -99.29% (p=0.002 n=6)
STW/Sum-2 74176.0µ ± 0% 525.6µ ± 6% -99.29% (p=0.002 n=6)
```
Throughput (no degradations)
```
for i in $(seq 8); do
GOMAXPROCS=2 go test -run='^$' -bench='BenchmarkWrite|BenchmarkSum' -count=3 -benchtime=200ms ./blake2b/ ./blake2s/
done
benchstat before.txt after.txt
blake2b sec/op sec/op vs base
Write128-2 101.2n ± 0% 101.1n ± 0% -0.15% (p=0.017 n=24)
Write1M-2 784.7µ ± 0% 785.0µ ± 0% +0.04% (p=0.032 n=24)
Write1K-2 774.5n ± 0% 774.0n ± 0% ~ (p=0.370 n=24)
Sum128-2 106.8n ± 0% 106.7n ± 0% ~ (p=0.093 n=24)
Sum1K-2 779.7n ± 0% 781.8n ± 0% +0.27% (p=0.017 n=24)
geomean 1.387µ 1.387µ +0.00%
blake2s sec/op sec/op vs base
Write64-2 76.65n ± 0% 76.55n ± 0% ~ (p=0.415 n=24)
Write1M-2 1.154m ± 0% 1.154m ± 0% ~ (p=0.178 n=24)
Write1K-2 1.141µ ± 0% 1.142µ ± 0% ~ (p=0.527 n=24)
Sum64-2 84.86n ± 0% 83.61n ± 0% -1.47% (p=0.000 n=24)
Sum1K-2 1.150µ ± 0% 1.149µ ± 0% -0.09% (p=0.001 n=24)
geomean 1.580µ 1.575µ -0.32%
```
References:
- Go stdlib sha256, md5: golang/go#64417
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
The hash goroutine was not synchronized before runtime.GC(), so a collection could complete before hashing had started and that iteration measured nothing. It overlapped every time in practice only because the hash is tens of milliseconds against a sub-millisecond collection, which is luck rather than design. The goroutine now signals before it hashes and the timing waits for that signal. Numbers are unchanged: STW/Write 50765.1us -> 444.2us and STW/Sum 50759.7us -> 434.3us for blake2b, 74686.5us -> 504.5us and 74578.3us -> 480.7us for blake2s.
|
This PR (HEAD: 9f1f6bb) has been imported to Gerrit for code review. Please visit Gerrit at https://go-review.googlesource.com/c/crypto/+/816400. Important tips:
|
|
Message from Gopher Robot: Patch Set 1: (1 comment) Please don’t reply on this GitHub thread. Visit golang.org/cl/816400. |
|
Message from Gopher Robot: Patch Set 1: Congratulations on opening your first change. Thank you for your contribution! Next steps: Most changes in the Go project go through a few rounds of revision. This can be During May-July and Nov-Jan the Go project is in a code freeze, during which Please don’t reply on this GitHub thread. Visit golang.org/cl/816400. |
Problem: GC stop-the-world is slow if a large input is passed to Write or Sum512.
Root cause: hashBlocks() is handed unbounded input, and assembly is not
preemptible, so one call keeps every P in stop-the-world for as long as it
runs. CL 671098 bounded crypto/md5 and crypto/sha256 the same way.
Stop-the-world (improved)
Worst stop-the-world stopping pause, /sched/pauses/stopping/gc:seconds, which
is the time the runtime spends waiting for every P to halt, while hashing a
64 MiB buffer. linux/amd64, EPYC 4344P, GOMAXPROCS=2. Values are histogram
bucket upper bounds, so read them as "at most":
BenchmarkSTW, added here, reports the same effect as the wall time of one
runtime.GC() that overlaps the hash, for both paths:
The before column is flat at 0% because it is not measuring the collector at
all, it is measuring the hash. The variance that appears afterwards is the
collector's own work.
Throughput (no degradations)
Updates golang/go#64417