Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[2/N] Chunk encoding optimization: bench mark the gnark based encoding #624

Merged
merged 1 commit into from
Jul 11, 2024

Conversation

jianoaix
Copy link
Contributor

@jianoaix jianoaix commented Jul 1, 2024

Why are these changes needed?

To understand the performance of new chunk serialization and deserialization compared to existing approach.

Encoding (BenchmarkFrameGnarkSerialization) is 7x faster, decoding (BenchmarkFrameGnarkDeserialization) is 3.6x faster.

goos: linux
goarch: amd64
pkg: github.com/Layr-Labs/eigenda/encoding
cpu: Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
BenchmarkFrameGobSerialization-8       	   71583	     16657 ns/op
BenchmarkFrameGobSerialization-8       	   71329	     16743 ns/op
BenchmarkFrameGobSerialization-8       	   70735	     16912 ns/op
BenchmarkFrameGnarkSerialization-8     	  478899	      2402 ns/op
BenchmarkFrameGnarkSerialization-8     	  467806	      2390 ns/op
BenchmarkFrameGnarkSerialization-8     	  467905	      2402 ns/op
BenchmarkFrameGobDeserialization-8     	   32188	     34923 ns/op
BenchmarkFrameGobDeserialization-8     	   34275	     34864 ns/op
BenchmarkFrameGobDeserialization-8     	   33954	     34775 ns/op
BenchmarkFrameGnarkDeserialization-8   	  121190	      9715 ns/op
BenchmarkFrameGnarkDeserialization-8   	  122356	      9832 ns/op
BenchmarkFrameGnarkDeserialization-8   	  120322	      9718 ns/op

Checks

  • I've made sure the lint is passing in this PR.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, in that case, please comment that they are not relevant.
  • Testing Strategy
    • Unit tests
    • Integration tests
    • This PR is not tested :(

@jianoaix jianoaix requested a review from bxue-l2 July 1, 2024 16:35
@jianoaix
Copy link
Contributor Author

jianoaix commented Jul 3, 2024

ping?

@jianoaix
Copy link
Contributor Author

jianoaix commented Jul 8, 2024

ping again?

@jianoaix
Copy link
Contributor Author

Ping for review!

@jianoaix jianoaix merged commit 63c1d21 into Layr-Labs:master Jul 11, 2024
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants