Performance comparison between assembly and cgo

[yingfeng]

Hi folks,
I noticed that parts of encoding/decoding snippets are accelerated using assembly. Have you made any performance comparison of using cgo for corresponding acceleration? Although cgo itself has a remarkable overhead around 100ns for each call, it might be negligible given larger batch inputs. On the other hand, assembly itself also has the shortcomings such that it could not be inlined, so it would be interesting if performance comparison is available. Thank you~

[WojciechMula]

The comment from my side: I wanted to avoid cgo or bindings to C libraries. Either pure go or assembly.

Speaking of performance comparison, I checked zstd decompression speed against bindings to a C library - both https://github.com/DataDog/zstd and https://github.com/valyala/gozstd. The compiled C code is faster in most cases, but for highly compressible files I noticed that performance was pretty close.

Inlining: the assembly implementations for zstd are huge, they do everything in one call.

[yingfeng]

Hmm, so adopting assembly instead of cgo is just a choice of taste... right ?

[klauspost]

@yingfeng cgo is not Go is still relevant. Many software projects will not allow cgo. It complicates the build chain - mostly making crossbuilds hard, compiles slowly.

This package will always be pure Go and Go assembly. That is an active goal.

[yingfeng]

Got it, thanks for your explanation~