diff --git a/benchmark/poseidon2/README.md b/benchmark/poseidon2/README.md index ca57c5f0e..2a0a5c943 100644 --- a/benchmark/poseidon2/README.md +++ b/benchmark/poseidon2/README.md @@ -2,6 +2,7 @@ ``` Run on 13th Gen Intel(R) Core(TM) i9-13900K (32 X 5500 MHz CPU s) +Compiler: clang-15 CPU Caches: L1 Data 48 KiB (x16) L1 Instruction 32 KiB (x16) @@ -15,29 +16,30 @@ CPU Caches: L2 Unified 4096 KiB (x12) ``` +Note: Run with `build --@rules_rust//:extra_rustc_flags="-Ctarget-cpu=native"` in your .bazelrc.user Note that Poseidon2 runs 10000x per test due to some time results being too small when running a single iteration. ## BN254 ```shell -bazel run --config opt --//:has_matplotlib //benchmark/poseidon2:poseidon2_benchmark -- -p bn254_fr --vendor horizen --vendor plonky3 --check_results +GOMP_SPINCOUNT=0 bazel run --config maxopt --//:has_matplotlib //benchmark/poseidon2:poseidon2_benchmark -- -p bn254_fr --vendor horizen --vendor plonky3 --check_results ``` ### On Intel i9-13900K | Trial Number | Tachyon | Horizen | Plonky3 | | :----------- | -------- | ------------ | -------- | -| 0 | 0.064726 | **0.050047** | 0.082464 | -| 1 | 0.061723 | **0.049997** | 0.082598 | -| 2 | 0.060917 | **0.050063** | 0.08257 | -| 3 | 0.06086 | **0.049952** | 0.082493 | -| 4 | 0.060655 | **0.050173** | 0.082409 | -| 5 | 0.060768 | **0.050683** | 0.08268 | -| 6 | 0.060843 | **0.050278** | 0.082675 | -| 7 | 0.060696 | **0.050062** | 0.082579 | -| 8 | 0.060688 | **0.05004** | 0.082592 | -| 9 | 0.060677 | **0.050128** | 0.083144 | -| avg | 0.061255 | **0.050142** | 0.08262 | +| 0 | 0.069228 | **0.050903** | 0.085679 | +| 1 | 0.062046 | **0.050892** | 0.085772 | +| 2 | 0.06053 | **0.050848** | 0.08553 | +| 3 | 0.060648 | **0.050825** | 0.085643 | +| 4 | 0.060553 | **0.051126** | 0.08583 | +| 5 | 0.060592 | **0.05362** | 0.085475 | +| 6 | 0.060576 | **0.050936** | 0.085731 | +| 7 | 0.06051 | **0.05081** | 0.085613 | +| 8 | 0.060561 | **0.050889** | 0.086382 | +| 9 | 0.060558 | **0.050896** | 0.086557 | +| avg | 0.06158 | **0.051174** | 0.085821 | ![image](/benchmark/poseidon2/poseidon2_benchmark_bn254_ubuntu_i9.png) @@ -45,17 +47,17 @@ bazel run --config opt --//:has_matplotlib //benchmark/poseidon2:poseidon2_bench | Trial Number | Tachyon | Horizen | Plonky3 | | :----------- | -------- | ------------ | -------- | -| 0 | 0.072979 | **0.055031** | 0.081624 | -| 1 | 0.072478 | **0.054731** | 0.081525 | -| 2 | 0.071973 | **0.055768** | 0.081633 | -| 3 | 0.071969 | **0.054835** | 0.081638 | -| 4 | 0.072009 | **0.054884** | 0.081545 | -| 5 | 0.071933 | **0.055** | 0.081572 | -| 6 | 0.07201 | **0.054946** | 0.081521 | -| 7 | 0.072033 | **0.054883** | 0.081539 | -| 8 | 0.071967 | **0.054989** | 0.081626 | -| 9 | 0.071934 | **0.054942** | 0.081556 | -| avg | 0.072128 | **0.055** | 0.081577 | +| 0 | 0.068967 | **0.058728** | 0.086994 | +| 1 | 0.068786 | **0.05839** | 0.086825 | +| 2 | 0.068658 | **0.058245** | 0.086779 | +| 3 | 0.068673 | **0.058189** | 0.086693 | +| 4 | 0.068675 | **0.058303** | 0.08674 | +| 5 | 0.068693 | **0.058109** | 0.08681 | +| 6 | 0.068621 | **0.058405** | 0.086816 | +| 7 | 0.068747 | **0.058247** | 0.086871 | +| 8 | 0.068637 | **0.058383** | 0.086842 | +| 9 | 0.068665 | **0.058162** | 0.086846 | +| avg | 0.068712 | **0.058316** | 0.086821 | ![image](/benchmark/poseidon2/poseidon2_benchmark_bn254_mac_m3.png) @@ -66,24 +68,24 @@ Note: Horizen and Plonky3 compute values with a different internal matrix, requi ### Horizen ```shell -bazel run --config opt --//:has_matplotlib //benchmark/poseidon2:poseidon2_benchmark -- -p baby_bear --vendor horizen --check_results +GOMP_SPINCOUNT=0 bazel run --config maxopt --//:has_matplotlib //benchmark/poseidon2:poseidon2_benchmark -- -p baby_bear --vendor horizen --check_results ``` #### On Intel i9-13900K | Trial Number | Tachyon | Horizen | | :----------- | ------------ | -------- | -| 0 | **0.011549** | 0.034751 | -| 1 | **0.011439** | 0.034627 | -| 2 | **0.011475** | 0.034581 | -| 3 | **0.011543** | 0.035442 | -| 4 | **0.011455** | 0.03632 | -| 5 | **0.011372** | 0.034545 | -| 6 | **0.011381** | 0.034538 | -| 7 | **0.011142** | 0.03459 | -| 8 | **0.010845** | 0.034522 | -| 9 | **0.010819** | 0.034589 | -| avg | **0.011302** | 0.03485 | +| 0 | **0.011424** | 0.034238 | +| 1 | **0.011975** | 0.034214 | +| 2 | **0.011505** | 0.034245 | +| 3 | **0.011304** | 0.03418 | +| 4 | **0.011313** | 0.034231 | +| 5 | **0.011354** | 0.034234 | +| 6 | **0.010743** | 0.034487 | +| 7 | **0.01071** | 0.034259 | +| 8 | **0.010706** | 0.034229 | +| 9 | **0.010708** | 0.034246 | +| avg | **0.011174** | 0.034256 | ![image](/benchmark/poseidon2/poseidon2_benchmark_baby_bear_horizen_ubuntu_i9.png) @@ -91,41 +93,41 @@ bazel run --config opt --//:has_matplotlib //benchmark/poseidon2:poseidon2_bench | Trial Number | Tachyon | Horizen | | :----------- | ------------ | -------- | -| 0 | **0.010979** | 0.013892 | -| 1 | **0.010574** | 0.013858 | -| 2 | **0.010544** | 0.014098 | -| 3 | **0.010642** | 0.013843 | -| 4 | **0.010517** | 0.013842 | -| 5 | **0.010599** | 0.013938 | -| 6 | **0.010519** | 0.013913 | -| 7 | **0.010474** | 0.013889 | -| 8 | **0.010572** | 0.013892 | -| 9 | **0.010533** | 0.013979 | -| avg | **0.010595** | 0.013914 | +| 0 | **0.010679** | 0.014511 | +| 1 | **0.010448** | 0.014653 | +| 2 | **0.010286** | 0.014961 | +| 3 | **0.01024** | 0.014769 | +| 4 | **0.010233** | 0.014717 | +| 5 | **0.010267** | 0.014761 | +| 6 | **0.010226** | 0.01514 | +| 7 | **0.010303** | 0.01475 | +| 8 | **0.010253** | 0.014693 | +| 9 | **0.010326** | 0.014533 | +| avg | **0.010326** | 0.014748 | ![image](/benchmark/poseidon2/poseidon2_benchmark_baby_bear_horizen_mac_m3.png) ### Plonky3 ```shell -bazel run --config opt --//:has_matplotlib //benchmark/poseidon2:poseidon2_benchmark -- -p baby_bear --vendor plonky3 --check_results +GOMP_SPINCOUNT=0 bazel run --config maxopt --//:has_matplotlib //benchmark/poseidon2:poseidon2_benchmark -- -p baby_bear --vendor plonky3 --check_results ``` #### On Intel i9-13900K | Trial Number | Tachyon | Plonky3 | | :----------- | -------- | ------------ | -| 0 | 0.009848 | **0.006655** | -| 1 | 0.00992 | **0.006582** | -| 2 | 0.009955 | **0.006616** | -| 3 | 0.009811 | **0.006572** | -| 4 | 0.009851 | **0.006537** | -| 5 | 0.009776 | **0.006645** | -| 6 | 0.009822 | **0.006548** | -| 7 | 0.009738 | **0.006586** | -| 8 | 0.009757 | **0.006594** | -| 9 | 0.009717 | **0.006619** | -| avg | 0.009819 | **0.006595** | +| 0 | 0.00999 | **0.005391** | +| 1 | 0.009882 | **0.005298** | +| 2 | 0.009848 | **0.00513** | +| 3 | 0.009772 | **0.005157** | +| 4 | 0.00977 | **0.005072** | +| 5 | 0.009774 | **0.005032** | +| 6 | 0.009783 | **0.005062** | +| 7 | 0.009878 | **0.005077** | +| 8 | 0.009778 | **0.005014** | +| 9 | 0.009762 | **0.005016** | +| avg | 0.009823 | **0.005124** | ![image](/benchmark/poseidon2/poseidon2_benchmark_baby_bear_plonky3_ubuntu_i9.png) @@ -133,16 +135,16 @@ bazel run --config opt --//:has_matplotlib //benchmark/poseidon2:poseidon2_bench | Trial Number | Tachyon | Plonky3 | | :----------- | -------- | ------------ | -| 0 | 0.009201 | **0.00732** | -| 1 | 0.008927 | **0.007346** | -| 2 | 0.008922 | **0.007309** | -| 3 | 0.008735 | **0.007324** | -| 4 | 0.008765 | **0.0076** | -| 5 | 0.008715 | **0.007335** | -| 6 | 0.008704 | **0.007439** | -| 7 | 0.008664 | **0.007369** | -| 8 | 0.008679 | **0.007347** | -| 9 | 0.008657 | **0.007353** | -| avg | 0.008796 | **0.007374** | +| 0 | 0.009116 | **0.007311** | +| 1 | 0.008967 | **0.007352** | +| 2 | 0.008805 | **0.007312** | +| 3 | 0.008748 | **0.007315** | +| 4 | 0.008742 | **0.007339** | +| 5 | 0.008741 | **0.007309** | +| 6 | 0.008774 | **0.00732** | +| 7 | 0.00873 | **0.007696** | +| 8 | 0.008791 | **0.007342** | +| 9 | 0.008741 | **0.007353** | +| avg | 0.008815 | **0.007364** | ![image](/benchmark/poseidon2/poseidon2_benchmark_baby_bear_plonky3_mac_m3.png)\*\*\*\* diff --git a/benchmark/poseidon2/poseidon2_benchmark_baby_bear_horizen_mac_m3.png b/benchmark/poseidon2/poseidon2_benchmark_baby_bear_horizen_mac_m3.png index 1499d73ad..863a78143 100644 Binary files a/benchmark/poseidon2/poseidon2_benchmark_baby_bear_horizen_mac_m3.png and b/benchmark/poseidon2/poseidon2_benchmark_baby_bear_horizen_mac_m3.png differ diff --git a/benchmark/poseidon2/poseidon2_benchmark_baby_bear_horizen_ubuntu_i9.png b/benchmark/poseidon2/poseidon2_benchmark_baby_bear_horizen_ubuntu_i9.png index 502f3b277..a4304a880 100644 Binary files a/benchmark/poseidon2/poseidon2_benchmark_baby_bear_horizen_ubuntu_i9.png and b/benchmark/poseidon2/poseidon2_benchmark_baby_bear_horizen_ubuntu_i9.png differ diff --git a/benchmark/poseidon2/poseidon2_benchmark_baby_bear_plonky3_mac_m3.png b/benchmark/poseidon2/poseidon2_benchmark_baby_bear_plonky3_mac_m3.png index fbf658958..1452cfb38 100644 Binary files a/benchmark/poseidon2/poseidon2_benchmark_baby_bear_plonky3_mac_m3.png and b/benchmark/poseidon2/poseidon2_benchmark_baby_bear_plonky3_mac_m3.png differ diff --git a/benchmark/poseidon2/poseidon2_benchmark_baby_bear_plonky3_ubuntu_i9.png b/benchmark/poseidon2/poseidon2_benchmark_baby_bear_plonky3_ubuntu_i9.png index 107e66a96..edc8db8d2 100644 Binary files a/benchmark/poseidon2/poseidon2_benchmark_baby_bear_plonky3_ubuntu_i9.png and b/benchmark/poseidon2/poseidon2_benchmark_baby_bear_plonky3_ubuntu_i9.png differ diff --git a/benchmark/poseidon2/poseidon2_benchmark_bn254_mac_m3.png b/benchmark/poseidon2/poseidon2_benchmark_bn254_mac_m3.png index dbabac2b1..eae247fd2 100644 Binary files a/benchmark/poseidon2/poseidon2_benchmark_bn254_mac_m3.png and b/benchmark/poseidon2/poseidon2_benchmark_bn254_mac_m3.png differ diff --git a/benchmark/poseidon2/poseidon2_benchmark_bn254_ubuntu_i9.png b/benchmark/poseidon2/poseidon2_benchmark_bn254_ubuntu_i9.png index 7b2a8d3f2..f65e64a5d 100644 Binary files a/benchmark/poseidon2/poseidon2_benchmark_bn254_ubuntu_i9.png and b/benchmark/poseidon2/poseidon2_benchmark_bn254_ubuntu_i9.png differ