docs(benchmark): update FFT batch benchmarks with new flags

kroma-network · Oct 23, 2024 · c5d6e07 · c5d6e07
1 parent 5e37b03
commit c5d6e07
Show file tree

Hide file tree

Showing 5 changed files with 47 additions and 37 deletions.
diff --git a/benchmark/fft_batch/README.md b/benchmark/fft_batch/README.md
@@ -4,6 +4,7 @@
 
 ```
 Run on 13th Gen Intel(R) Core(TM) i9-13900K (32 X 5500 MHz CPU s)
+Compiler: clang-15
 CPU Caches:
   L1 Data 48 KiB (x16)
   L1 Instruction 32 KiB (x16)
@@ -17,70 +18,79 @@ CPU Caches:
   L2 Unified 4096 KiB (x12)
 ```
 
-### FFTBatch
+Note: Run with `build --@rules_rust//:extra_rustc_flags="-Ctarget-cpu=native"` in your .bazelrc.user
 
-```shell
-bazel run --config opt --//:has_matplotlib //benchmark/fft_batch:fft_batch_benchmark -- -k 20 -k 21 -k 22 -k 23 -k 24 -k 25 -k 26 --vendor plonky3 -p baby_bear --check_results
-```
+### FFTBatch
 
 WARNING: On Mac M3, tests beyond degree 24 are not feasible due to memory constraints.
 
 #### On Intel i9-13900K
 
+```shell
+GOMP_SPINCOUNT=0 bazel run --config maxopt --//:has_matplotlib //benchmark/fft_batch:fft_batch_benchmark -- -k 20 -k 21 -k 22 -k 23 -k 24 -k 25 -k 26 --vendor plonky3 -p baby_bear --check_results
+```
+
 | Exponent | Tachyon      | Plonky3      |
 | :------- | ------------ | ------------ |
-| 20       | 0.117925     | **0.110098** |
-| 21       | 0.222959     | **0.218505** |
-| 22       | 0.459209     | **0.447758** |
-| 23       | 0.97874      | **0.964644** |
-| 24       | 2.09675      | **2.092210** |
-| 25       | **6.20441**  | 6.98453      |
-| 26       | **18.6084**  | 20.7476      |
+| 20       | **0.092595** | 0.094762     |
+| 21       | **0.191168** | 0.193567     |
+| 22       | 0.406239     | **0.384377** |
+| 23       | 0.892501     | **0.842694** |
+| 24       | 1.91177      | **1.90586**  |
+| 25       | **5.82862**  | 7.34128      |
+| 26       | **17.1807**  | 20.3968      |
 
 ![image](/benchmark/fft_batch/fft_batch_benchmark_ubuntu_i9.png)
 
 #### On Mac M3 Pro
 
-| Exponent | Tachyon   | Plonky3      |
-| :------- | --------- | ------------ |
-| 20       | 0.132521  | **0.072505** |
-| 21       | 0.287744  | **0.140527** |
-| 22       | 0.588894  | **0.280177** |
-| 23       | 1.17446   | **0.621024** |
-| 24       | 3.17213   | **2.399220** |
+```shell
+GOMP_SPINCOUNT=0 bazel run --config maxopt --//:has_matplotlib //benchmark/fft_batch:fft_batch_benchmark -- -k 20 -k 21 -k 22 -k 23 -k 24  --vendor plonky3 -p baby_bear --check_results
+```
+
+| Exponent | Tachyon  | Plonky3      |
+| :------- | -------- | ------------ |
+| 20       | 0.083416 | **0.066952** |
+| 21       | 0.194191 | **0.138168** |
+| 22       | 0.408045 | **0.299547** |
+| 23       | 0.955439 | **0.679252** |
+| 24       | 11.8495  | **6.47188**  |
 
 ![image](/benchmark/fft_batch/fft_batch_benchmark_mac_m3.png)
 
 ### CosetLDEBatch
 
-```shell
-bazel run --config opt --//:has_matplotlib //benchmark/fft_batch:fft_batch_benchmark -- -k 20 -k 21 -k 22 -k 23 -k 24 -k 25 --vendor plonky3 -p baby_bear --run_coset_lde --check_results
-```
-
-WARNING: On Mac M3, tests beyond degree 24 are not feasible due to memory constraints.
+WARNING: On Intel i9-13900K, tests beyond degree 25 are not feasible due to memory constraints, and on Mac M3, tests beyond degree 24 are not feasible due to memory constraints.
 
 #### On Intel i9-13900K
 
-| Exponent | Tachyon      | Plonky3  |
-| :------- | ------------ | -------- |
-| 20       | **0.414096** | 0.783275 |
-| 21       | **0.828539** | 1.47701  |
-| 22       | **1.784080** | 3.06198  |
-| 23       | **3.673930** | 6.49181  |
-| 24       | **9.325390** | 16.2383  |
-| 25       | **25.66560** | 41.3335  |
+```shell
+GOMP_SPINCOUNT=0 bazel run --config maxopt --//:has_matplotlib //benchmark/fft_batch:fft_batch_benchmark -- -k 20 -k 21 -k 22 -k 23 -k 24 -k 25 --vendor plonky3 -p baby_bear --run_coset_lde --check_results
+```
+
+| Exponent | Tachyon     | Plonky3  |
+| :------- | ----------- | -------- |
+| 20       | **0.46917** | 0.639744 |
+| 21       | **0.92528** | 1.2923   |
+| 22       | **1.87363** | 2.68427  |
+| 23       | **4.06008** | 5.67987  |
+| 24       | **9.6627**  | 14.6164  |
+| 25       | **25.7953** | 39.5498  |
 
 ![image](/benchmark/fft_batch/coset_lde_batch_benchmark_ubuntu_i9.png)
 
 #### On Mac M3 Pro
 
+```shell
+GOMP_SPINCOUNT=0 bazel run --config maxopt --//:has_matplotlib //benchmark/fft_batch:fft_batch_benchmark -- -k 20 -k 21 -k 22 -k 23 -k 24 --vendor plonky3 -p baby_bear --run_coset_lde --check_results
+```
+
 | Exponent | Tachyon      | Plonky3      |
 | :------- | ------------ | ------------ |
-| 18       | 0.100942     | **0.086087** |
-| 19       | 0.214471     | **0.182212** |
-| 20       | 0.481229     | **0.359246** |
-| 21       | **0.981806** | 1.518190     |
-| 22       | 3.86094      | **3.244580** |
-| 23       | 7.50879      | **6.052250** |
+| 20       | **0.318485** | 0.323865     |
+| 21       | 0.667106     | **0.660975** |
+| 22       | **1.44873**  | 3.40795      |
+| 23       | 8.27201      | **5.91238**  |
+| 24       | 39.9678      | **23.1033**  |
 
 ![image](/benchmark/fft_batch/coset_lde_batch_benchmark_mac_m3.png)
diff --git a/benchmark/fft_batch/coset_lde_batch_benchmark_mac_m3.png b/benchmark/fft_batch/coset_lde_batch_benchmark_mac_m3.png
diff --git a/benchmark/fft_batch/coset_lde_batch_benchmark_ubuntu_i9.png b/benchmark/fft_batch/coset_lde_batch_benchmark_ubuntu_i9.png
diff --git a/benchmark/fft_batch/fft_batch_benchmark_mac_m3.png b/benchmark/fft_batch/fft_batch_benchmark_mac_m3.png
diff --git a/benchmark/fft_batch/fft_batch_benchmark_ubuntu_i9.png b/benchmark/fft_batch/fft_batch_benchmark_ubuntu_i9.png