diff --git a/benchmark/fft_batch/README.md b/benchmark/fft_batch/README.md index 1528c196c..6c6832030 100644 --- a/benchmark/fft_batch/README.md +++ b/benchmark/fft_batch/README.md @@ -4,6 +4,7 @@ ``` Run on 13th Gen Intel(R) Core(TM) i9-13900K (32 X 5500 MHz CPU s) +Compiler: clang-15 CPU Caches: L1 Data 48 KiB (x16) L1 Instruction 32 KiB (x16) @@ -17,70 +18,79 @@ CPU Caches: L2 Unified 4096 KiB (x12) ``` -### FFTBatch +Note: Run with `build --@rules_rust//:extra_rustc_flags="-Ctarget-cpu=native"` in your .bazelrc.user -```shell -bazel run --config opt --//:has_matplotlib //benchmark/fft_batch:fft_batch_benchmark -- -k 20 -k 21 -k 22 -k 23 -k 24 -k 25 -k 26 --vendor plonky3 -p baby_bear --check_results -``` +### FFTBatch WARNING: On Mac M3, tests beyond degree 24 are not feasible due to memory constraints. #### On Intel i9-13900K +```shell +GOMP_SPINCOUNT=0 bazel run --config maxopt --//:has_matplotlib //benchmark/fft_batch:fft_batch_benchmark -- -k 20 -k 21 -k 22 -k 23 -k 24 -k 25 -k 26 --vendor plonky3 -p baby_bear --check_results +``` + | Exponent | Tachyon | Plonky3 | | :------- | ------------ | ------------ | -| 20 | 0.117925 | **0.110098** | -| 21 | 0.222959 | **0.218505** | -| 22 | 0.459209 | **0.447758** | -| 23 | 0.97874 | **0.964644** | -| 24 | 2.09675 | **2.092210** | -| 25 | **6.20441** | 6.98453 | -| 26 | **18.6084** | 20.7476 | +| 20 | **0.092595** | 0.094762 | +| 21 | **0.191168** | 0.193567 | +| 22 | 0.406239 | **0.384377** | +| 23 | 0.892501 | **0.842694** | +| 24 | 1.91177 | **1.90586** | +| 25 | **5.82862** | 7.34128 | +| 26 | **17.1807** | 20.3968 | ![image](/benchmark/fft_batch/fft_batch_benchmark_ubuntu_i9.png) #### On Mac M3 Pro -| Exponent | Tachyon | Plonky3 | -| :------- | --------- | ------------ | -| 20 | 0.132521 | **0.072505** | -| 21 | 0.287744 | **0.140527** | -| 22 | 0.588894 | **0.280177** | -| 23 | 1.17446 | **0.621024** | -| 24 | 3.17213 | **2.399220** | +```shell +GOMP_SPINCOUNT=0 bazel run --config maxopt --//:has_matplotlib //benchmark/fft_batch:fft_batch_benchmark -- -k 20 -k 21 -k 22 -k 23 -k 24 --vendor plonky3 -p baby_bear --check_results +``` + +| Exponent | Tachyon | Plonky3 | +| :------- | -------- | ------------ | +| 20 | 0.083416 | **0.066952** | +| 21 | 0.194191 | **0.138168** | +| 22 | 0.408045 | **0.299547** | +| 23 | 0.955439 | **0.679252** | +| 24 | 11.8495 | **6.47188** | ![image](/benchmark/fft_batch/fft_batch_benchmark_mac_m3.png) ### CosetLDEBatch -```shell -bazel run --config opt --//:has_matplotlib //benchmark/fft_batch:fft_batch_benchmark -- -k 20 -k 21 -k 22 -k 23 -k 24 -k 25 --vendor plonky3 -p baby_bear --run_coset_lde --check_results -``` - -WARNING: On Mac M3, tests beyond degree 24 are not feasible due to memory constraints. +WARNING: On Intel i9-13900K, tests beyond degree 25 are not feasible due to memory constraints, and on Mac M3, tests beyond degree 24 are not feasible due to memory constraints. #### On Intel i9-13900K -| Exponent | Tachyon | Plonky3 | -| :------- | ------------ | -------- | -| 20 | **0.414096** | 0.783275 | -| 21 | **0.828539** | 1.47701 | -| 22 | **1.784080** | 3.06198 | -| 23 | **3.673930** | 6.49181 | -| 24 | **9.325390** | 16.2383 | -| 25 | **25.66560** | 41.3335 | +```shell +GOMP_SPINCOUNT=0 bazel run --config maxopt --//:has_matplotlib //benchmark/fft_batch:fft_batch_benchmark -- -k 20 -k 21 -k 22 -k 23 -k 24 -k 25 --vendor plonky3 -p baby_bear --run_coset_lde --check_results +``` + +| Exponent | Tachyon | Plonky3 | +| :------- | ----------- | -------- | +| 20 | **0.46917** | 0.639744 | +| 21 | **0.92528** | 1.2923 | +| 22 | **1.87363** | 2.68427 | +| 23 | **4.06008** | 5.67987 | +| 24 | **9.6627** | 14.6164 | +| 25 | **25.7953** | 39.5498 | ![image](/benchmark/fft_batch/coset_lde_batch_benchmark_ubuntu_i9.png) #### On Mac M3 Pro +```shell +GOMP_SPINCOUNT=0 bazel run --config maxopt --//:has_matplotlib //benchmark/fft_batch:fft_batch_benchmark -- -k 20 -k 21 -k 22 -k 23 -k 24 --vendor plonky3 -p baby_bear --run_coset_lde --check_results +``` + | Exponent | Tachyon | Plonky3 | | :------- | ------------ | ------------ | -| 18 | 0.100942 | **0.086087** | -| 19 | 0.214471 | **0.182212** | -| 20 | 0.481229 | **0.359246** | -| 21 | **0.981806** | 1.518190 | -| 22 | 3.86094 | **3.244580** | -| 23 | 7.50879 | **6.052250** | +| 20 | **0.318485** | 0.323865 | +| 21 | 0.667106 | **0.660975** | +| 22 | **1.44873** | 3.40795 | +| 23 | 8.27201 | **5.91238** | +| 24 | 39.9678 | **23.1033** | ![image](/benchmark/fft_batch/coset_lde_batch_benchmark_mac_m3.png) diff --git a/benchmark/fft_batch/coset_lde_batch_benchmark_mac_m3.png b/benchmark/fft_batch/coset_lde_batch_benchmark_mac_m3.png index d2e076547..bc2d12d42 100644 Binary files a/benchmark/fft_batch/coset_lde_batch_benchmark_mac_m3.png and b/benchmark/fft_batch/coset_lde_batch_benchmark_mac_m3.png differ diff --git a/benchmark/fft_batch/coset_lde_batch_benchmark_ubuntu_i9.png b/benchmark/fft_batch/coset_lde_batch_benchmark_ubuntu_i9.png index 538f22908..3142cf2f5 100644 Binary files a/benchmark/fft_batch/coset_lde_batch_benchmark_ubuntu_i9.png and b/benchmark/fft_batch/coset_lde_batch_benchmark_ubuntu_i9.png differ diff --git a/benchmark/fft_batch/fft_batch_benchmark_mac_m3.png b/benchmark/fft_batch/fft_batch_benchmark_mac_m3.png index 16235970a..ebe611711 100644 Binary files a/benchmark/fft_batch/fft_batch_benchmark_mac_m3.png and b/benchmark/fft_batch/fft_batch_benchmark_mac_m3.png differ diff --git a/benchmark/fft_batch/fft_batch_benchmark_ubuntu_i9.png b/benchmark/fft_batch/fft_batch_benchmark_ubuntu_i9.png index e1f07bf3d..17dc46f6e 100644 Binary files a/benchmark/fft_batch/fft_batch_benchmark_ubuntu_i9.png and b/benchmark/fft_batch/fft_batch_benchmark_ubuntu_i9.png differ