Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

client: Optimize signature buffer pool #666

Open
cthulhu-rider opened this issue Dec 11, 2024 · 1 comment
Open

client: Optimize signature buffer pool #666

cthulhu-rider opened this issue Dec 11, 2024 · 1 comment
Labels
blocked Can't be done because of something client Issue related to the client discussion Open discussion of some problem enhancement Improving existing functionality

Comments

@cthulhu-rider
Copy link
Contributor

Is your feature request related to a problem? Please describe.

Client allows to specify buffer pool for signing defaulting to

size := prm.signMessageBufferSizes
if size == 0 {
size = defaultBufferSize
}
c.buffers = &sync.Pool{}
c.buffers.New = func() any {
b := make([]byte, size)
return &b
}

to sign any request, client selects an arbitrary buffer from the pool. If this buffer has sufficient size to encode the signed request parts, they are written into it. Otherwise, temp buffer of needed size is allocated for each part via make

i see following drawbacks of this approach:

  1. buffers of different size are mixed in the pool. Some signed entities are empty (like NetmapService request bodies), some are pretty small <=1K (meta headers, bodies of almost all requests), some can be up to 4M (object PUT payload chunks)
  2. make-allocated buffers aint pooled

2 just does not allow reusing the buffer. 1 may worsen caching in some cases:

  • when next 4M object payload chunk is to-be-signed, we are likely to select small buffer which leads . At the same time, pool can still have big buffers from previous chunks
  • when small entity is signed, it can acquire big buffer, however, most of its len will not be used. This also affects 1

Describe the solution you'd like

  1. use size-based buffer pool. Size of any signed part is known in advance. We can precalc it and select the buffer of the closest size
  2. if buffer is dynamically allocated for signing anyway, pool it

Describe alternatives you've considered

no except keep as it is

Additional context

@cthulhu-rider cthulhu-rider added enhancement Improving existing functionality blocked Can't be done because of something client Issue related to the client discussion Open discussion of some problem labels Dec 11, 2024
@cthulhu-rider
Copy link
Contributor Author

also wanna mention that when verifying responses, the buffers are always dynamic. They can also be pooled. This deserves a separate issue, but i suggest to create it only if current one is unblocked

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
blocked Can't be done because of something client Issue related to the client discussion Open discussion of some problem enhancement Improving existing functionality
Projects
None yet
Development

No branches or pull requests

1 participant