Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
BuiltinFunction: Use a generated DAFSA instead of a generated enum
A DAFSA (deterministic acyclic finite state automaton) is essentially a trie flattened into an array, but that also uses techniques to minimize redundant nodes. This provides fast lookups while minimizing the required data size, but normally does not allow for associating data related to each word. However, by adding a count of the number of possible words from each node, it becomes possible to also use it to achieve minimal perfect hashing for the set of words (which allows going from word -> unique index as well as unique index -> word). This allows us to store a second array of data so that the DAFSA can be used as a lookup for e.g. BuiltinFunction. Some resources: - https://en.wikipedia.org/wiki/Deterministic_acyclic_finite_state_automaton - https://web.archive.org/web/20220722224703/http://pages.pathcom.com/~vadco/dawg.html - https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.56.5272 (this is the origin of the minimal perfect hashing technique) - http://stevehanov.ca/blog/?id=115 What all is going on here: - `process_builtins.py` was removed in favor of a zig implementation that generates the DAFSA - The new zig implementation uses the latest .def files from the LLVM repository, so the set of builtins is different and there are some new macro definitions that are not yet supported (e.g. TARGET_BUILTIN), but otherwise the logic matches the python script wherever possible - The new zig implementation imports `src/builtins/TypeDescription.zig` directly so the calculated max parameter count will always be in sync The results: Microbenchmark of lookup time: # Does a builtin name exist?: ------------------- dafsa ------------------ always found: 88ns per lookup not found (random bytes): 33ns per lookup not found (1 char diff): 86ns per lookup ------------------- enum ------------------- always found: 3648ns per lookup not found (random bytes): 6310ns per lookup not found (1 char diff): 3604ns per lookup dafsa is 41.5x / 191.2x / 41.9x faster # Name -> BuiltinFunction lookup: ------------------- dafsa ------------------ always found: 154ns per lookup not found (random bytes): 32ns per lookup not found (1 char diff): 154ns per lookup ------------------- enum ------------------- always found: 3649ns per lookup not found (random bytes): 6317ns per lookup not found (1 char diff): 3649ns per lookup dafsa is 23.7x / 197.4x / 23.7x faster About 500KiB removed from a ReleaseSmall build of aro: 1.8M arocc-master 1.3M arocc-dafsa $ bloaty arocc-dafsa -- arocc-master FILE SIZE VM SIZE -------------- -------------- +8.1% +22.9Ki +8.1% +22.9Ki .data.rel.ro -0.1% -8 -0.1% -8 .eh_frame_hdr -1.4% -2.41Ki -1.4% -2.41Ki .data -35.5% -49.4Ki -35.5% -49.4Ki .eh_frame -56.8% -201Ki -56.8% -201Ki .rodata -30.4% -258Ki -30.4% -258Ki .text -26.9% -488Ki -26.9% -488Ki TOTAL Compile time of `zig build` is reduced by around a second: $ hyperfine "git checkout master; zig build" "git checkout dafsa; zig build" --prepare "rm -rf zig-cache" --runs 3 --warmup 1 Benchmark #1: git checkout master; zig build Time (mean ± σ): 11.760 s ± 0.024 s [User: 11.387 s, System: 0.605 s] Range (min … max): 11.745 s … 11.788 s 3 runs Benchmark #2: git checkout dafsa; zig build Time (mean ± σ): 10.523 s ± 0.016 s [User: 10.121 s, System: 0.608 s] Range (min … max): 10.505 s … 10.534 s 3 runs Summary 'git checkout dafsa; zig build' ran 1.12 ± 0.00 times faster than 'git checkout master; zig build' Unit tests run faster: Benchmark #1: ./unit-tests-master Time (mean ± σ): 224.9 ms ± 2.6 ms [User: 210.2 ms, System: 14.5 ms] Range (min … max): 220.7 ms … 230.1 ms 13 runs Benchmark #2: ./unit-tests-dafsa Time (mean ± σ): 97.4 ms ± 2.6 ms [User: 83.1 ms, System: 14.2 ms] Range (min … max): 94.2 ms … 103.5 ms 30 runs Summary './unit-tests-dafsa' ran 2.31 ± 0.07 times faster than './unit-tests-master' Record tests run slightly slower (not sure why): Benchmark #1: ./record-runner-master test/records Time (mean ± σ): 10.283 s ± 0.037 s [User: 118.281 s, System: 0.261 s] Range (min … max): 10.262 s … 10.327 s 3 runs Benchmark #2: ./record-runner-dafsa test/records Time (mean ± σ): 10.950 s ± 0.209 s [User: 123.950 s, System: 0.284 s] Range (min … max): 10.796 s … 11.187 s 3 runs Summary './record-runner-master test/records' ran 1.06 ± 0.02 times faster than './record-runner-dafsa test/records' Integration test times are ~unchanged: Benchmark #1: ./test-runner-master /home/ryan/Programming/zig/arocc/test/cases zig Time (mean ± σ): 382.8 ms ± 7.2 ms [User: 300.9 ms, System: 88.0 ms] Range (min … max): 373.2 ms … 393.2 ms 10 runs Benchmark #2: ./test-runner-dafsa /home/ryan/Programming/zig/arocc/test/cases zig Time (mean ± σ): 392.9 ms ± 3.5 ms [User: 308.5 ms, System: 90.8 ms] Range (min … max): 388.9 ms … 399.7 ms 10 runs Summary './test-runner-master /home/ryan/Programming/zig/arocc/test/cases zig' ran 1.03 ± 0.02 times faster than './test-runner-dafsa /home/ryan/Programming/zig/arocc/test/cases zig' enum + fast path
- Loading branch information