Elevated CPI rate in flate/stateless.go

[marselester]

Hi! You've built an awesome library which helped to shave 20% of CPU consumption in Parca Agent parca-dev/parca-agent#1065.

Out of curiosity I profiled the agent with Intel Vtune and it showed CPI rate 124.4 on var dst tokens line https://github.com/klauspost/compress/blob/master/flate/stateless.go#L68.

Cycles per Instruction Retired, or CPI, is a fundamental performance metric indicating approximately how much time each executed instruction took, in units of cycles. Modern superscalar processors issue up to four instructions per cycle, suggesting a theoretical best CPI of 0.25. But various effects (long-latency memory, floating-point, or SIMD operations; non-retired instructions due to branch mispredictions; instruction starvation in the front-end) tend to pull the observed CPI up. A CPI < 1 is typical for instruction bound code, while a CPI > 1 may show up for a stall cycle bound application, also likely memory bound. CPI is an excellent metric for judging an overall potential for application performance tuning.

I am not sure how to interpret the "store misses", it's my first time using Vtune :).
Perhaps the initialization of dst causes ~0.25MB of writes (the size of the struct) to CPU caches (L2 is 256KB, L3 is 3MB) which propagate to the main memory. Hence the elevated store latency in the report.
What do you think?

type tokens struct {
	extraHist [32]uint16  // codes 256->maxnumlit
	offHist   [32]uint16  // offset codes
	litHist   [256]uint16 // codes 0->255
	nFilled   int
	n         uint16 // Must be able to contain maxStoreBlockSize
	tokens    [maxStoreBlockSize + 1]token
}

[klauspost]

Yes, var dst tokens is zeroing.

As the docs state using Stateless compression is only for a very specific use cases, and it doesn't seem your use case covers that. Stateless does not mean no allocations - it means no buffers are kept between writes, so it will allocate more totally, since it has to allocate for every Write.

You are sacrificing both speed and compression by using a stateless compressor. Likely you will see a 2-3x speed up by using a sync.Pool with a gzip encoder at level 1.

I'd recommend the default compression you will not lose as much compression as you currently do. It is still likely to be 2x faster than the old.

[marselester]

Thank you for the explanation! Stateless compression helped to avoid excessive heap allocs which reduced GC pressure.
From what I understand a new flate.Writer was created each time z.compressor, _ = flate.NewWriter(z.w, z.level), see the profile below.

ROUTINE ======================== compress/gzip.(*Writer).Write in compress/gzip/gzip.go
         0     1.98GB (flat, cum) 40.10% of Total
         .          .    163:			z.buf[8] = 2
         .          .    164:		} else if z.level == BestSpeed {
         .          .    165:			z.buf[8] = 4
         .          .    166:		}
         .          .    167:		z.buf[9] = z.OS
         .   512.03kB    168:		_, z.err = z.w.Write(z.buf[:10])
         .          .    169:		if z.err != nil {
         .          .    170:			return 0, z.err
         .          .    171:		}
         .          .    172:		if z.Extra != nil {
         .          .    173:			z.err = z.writeBytes(z.Extra)
         .          .    174:			if z.err != nil {
         .          .    175:				return 0, z.err
         .          .    176:			}
         .          .    177:		}
         .          .    178:		if z.Name != "" {
         .          .    179:			z.err = z.writeString(z.Name)
         .          .    180:			if z.err != nil {
         .          .    181:				return 0, z.err
         .          .    182:			}
         .          .    183:		}
         .          .    184:		if z.Comment != "" {
         .          .    185:			z.err = z.writeString(z.Comment)
         .          .    186:			if z.err != nil {
         .          .    187:				return 0, z.err
         .          .    188:			}
         .          .    189:		}
         .          .    190:		if z.compressor == nil {
         .     1.98GB    191:			z.compressor, _ = flate.NewWriter(z.w, z.level)
         .          .    192:		}
         .          .    193:	}
         .          .    194:	z.size += uint32(len(p))
         .          .    195:	z.digest = crc32.Update(z.digest, crc32.IEEETable, p)
         .          .    196:	n, z.err = z.compressor.Write(p)
ROUTINE ======================== compress/flate.NewWriter in compress/flate/deflate.go
    1.64GB     1.98GB (flat, cum) 40.09% of Total
         .          .    665:// compression efficiency.
         .          .    666://
         .          .    667:// If level is in the range [-2, 9] then the error returned will be nil.
         .          .    668:// Otherwise the error returned will be non-nil.
         .          .    669:func NewWriter(w io.Writer, level int) (*Writer, error) {
    1.64GB     1.64GB    670:	var dw Writer
         .   347.72MB    671:	if err := dw.d.init(w, level); err != nil {
         .          .    672:		return nil, err
         .          .    673:	}
         .          .    674:	return &dw, nil
         .          .    675:}

Initially I thought that it's possible to reuse a gzip.Writer with Reset(), but I was wrong because it's mandatory to call Close() in order to write a gzip footer.

le.PutUint32(z.buf[:4], z.digest)
le.PutUint32(z.buf[4:8], z.size)
_, z.err = z.w.Write(z.buf[:8])

Do you think it's feasible to decouple footer writing from Close() method?

[klauspost]

You can Reset after Close

[marselester]

Oh, I didn't realize that :) Thank you!