Split benchmark packages #1276
Split benchmark packages #1276
Conversation
benchmarks/package.lisp
Outdated
| benchmark-clauses | ||
| &rest native-clauses) | ||
| (let ((benchmark-package (intern (format nil "BENCHMARK-~S" name) 'keyword)) | ||
| (native-package (intern (format nil "BENCHMARK-~S/NATIVE" name) 'keyword))) |
There was a problem hiding this comment.
Can these be COALTON-BENCHMARK/...? It's a little wordier, but worth it.
Maybe also add a default package local nickname "NATIVE" for the native package. That way the actual benchmarks can always refer to native:....
In general I don't like symbol generating macros, but for this limited use case I think it's fine.
|
Just a quick note: Issue #1204 |
|
Any recommendation for alternative benchmarking package? |
|
Are your benchmarks using mostly Coalton or would you ideally like equal support for Common Lisp benchmarks? |
|
I aim at specific use patterns that are expected to be optimized by the Coalton compiler, but it includes Common Lisp counterpart as base cases, so the framework needs to measure both cases. |
|
I wrote an initial draft of a potential implementation in Coalton, does this look like it might be able to support what you want in your benchmarks? |
|
Yes. At this moment, the essential feature is to have a standard way to define benchmarks and run them per-package. I can call plain-CL version of benchmark code from Coalton anyway. If #1286 will be merged soon, I'll adapt my other benchmarks to it. |
This change allows each group of benchmarks to be run selectively, as well as running all of them at once. A new macro `define-coalton-benchmark` provides boilerplate of package definitions. See README.md for the instructions.
shirok
force-pushed
the
split-package-benchmarks
branch
from
3079688 to
3b1d6fe
Compare
|
Reflected @stylewarning's comment and rebased. If the new Coalton-based benchmark suite may take some time to mature, I'd suggest this to be an intermediate step. |
As we have more benchmarks, it is more convenient to allow running not only the entire benchmarks, but the specific group of benchmarks of concern. Trivial-benchmarks treats each benchmark package as a group. This patch establishes a convention to create multiple benchmark packages.
These commits try to keep changes small, for easier transition.