S2 at least 60% slower than snappy on Apple Silicon

[academe-01]

Hello,

I am experiencing significantly slower speeds compared to Snappy on Apple M3.

Does anyone else have similar issues?
I am currently using s2.Encode and snappy.Encode.

[klauspost]

What is your payload size?

I don't have arm silicon, so you will have to be a bit more specific than this.

[klauspost]

The hash table for finding matches is twice the size (64KB vs 32KB). Since you are encoding small payloads the "penalty" of zeroing the hash table is the vast majority of the time taken.

I will make a "special" version for small payloads that has a smaller table - probably either 32 or 16KB - and use that for small inputs. The amd64 assembly versions already does this.

[academe-01]

Many thanks! Much appreciated!

[academe-01]

@klauspost maybe it can be made as configurable parameter ? I mean if s2.Encode used and not stream, why to make 16 kb table on let's say 32 byte input ?

[klauspost]

It cannot be configurable since table sizes needs to be fixed at compile time.

But it is pretty easily fixed by just having a separate encoder for small inputs... like this...

func encodeBlockSnappyGo(dst, src []byte) (d int) {
	if len(src) <= 64<<10 {
		return encodeBlockSnappyGo64K(dst, src)
	}

[klauspost]

#1048