Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Features to measure the constraints of libs #6

Open
iAmMichaelConnor opened this issue Nov 15, 2024 · 3 comments
Open

Features to measure the constraints of libs #6

iAmMichaelConnor opened this issue Nov 15, 2024 · 3 comments

Comments

@iAmMichaelConnor
Copy link

@kashbrti @jtriley-eth @TomAFrench

Some suggestions:

It doesn't look like I can create a tasklist, otherwise I'd have neatly made one for the 4 things below:

  • Change this template's filing structure to house test binaries which can consume the lib
  • Add CI to catch constraint regressions to this template
  • Figure out how to measure constraints of each lib function separately
  • Add CI to give a constraint diff report in PRs

I've noticed that I and other people tend to follow a pattern like this:

  • Put the actual library in a ./lib/ dir.
  • Put a test binary - which depends on the library - in a ./examples/ dir (the name can be debated).
  • Write a script which compiles the lib and then calls bb gates

Should we update this template to follow that kind of approach?

Some further suggestions:

Ideally, CI for libraries should catch regressions in constraint counts.

A single main function isn't the best for debugging which of the many functions in your lib might have regressed, constraint-wise. It would be better if we had some way of writing many main functions, each of which we could measure independently as part of CI.
I encountered this problem yesterday, and it was painful to figure out which function had regressed.

aztec-packages CI goes even further, and for every PR outputs a report of increases / decreases in constraint counts and opcodes. Pretty cool.

@TomAFrench
Copy link
Member

I've noticed that I and other people tend to follow a pattern like this...

I'm not a fan of that workflow for a few reasons:

  • It basically requires all libraries to be workspaces rather than just a simple package.which affects consumers
  • huge blowup in the number of binary packages for if you want to benchmark a decent number of things
  • extra complexity in compiling the benchmarks as we need to go hunting across a full workspace.

It would be better if we had some way of writing many main functions

We have this already with the #[export] tag which allows compiling non-main functions with the nargo export command. I've thrown together an example of using this in #7

@iAmMichaelConnor
Copy link
Author

We have this already with the #[export] tag

Oh cool! Does this work for methods of structs, or just for free functions?

I've thrown together an example

Delightful!

@TomAFrench
Copy link
Member

Does this work for methods of structs, or just for free functions

It might work for struct methods but it shouldn't. I'd recommend wrapping these in a free function if you want to benchmark it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants