Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: skip tests having oracle calls #6666

Open
wants to merge 17 commits into
base: master
Choose a base branch
from
Open

feat: skip tests having oracle calls #6666

wants to merge 17 commits into from

Conversation

guipublic
Copy link
Contributor

Description

Problem*

Resolves #6643

Summary*

Add a 'skip-oracle' flag for the nargo test command.

Additional Context

Documentation*

Check one:

  • No documentation needed.
  • Documentation included in this PR.
  • [For Experimental Features] Documentation to be submitted in a separate PR.

PR Checklist*

  • I have tested the changes locally.
  • I have formatted the changes with Prettier and/or cargo fmt on default settings.

@github-actions github-actions bot added the documentation Improvements or additions to documentation label Dec 2, 2024
@guipublic guipublic changed the title skip tests having oracle calls feat: skip tests having oracle calls Dec 2, 2024
Copy link
Contributor

github-actions bot commented Dec 2, 2024

Copy link
Contributor

github-actions bot commented Dec 3, 2024

Peak Memory Sample

Program Peak Memory
keccak256 81.50M
workspace 122.13M
regression_4709 333.71M
ram_blowup_regression 2.55G

@guipublic guipublic added the run-external-checks Trigger CI job to run tests on external repos label Dec 4, 2024
Comment on lines +75 to +76
if name == ForeignCall::CreateMock.name() {
OracleResult::Mocked
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why aren't all the other mock functions considered to be Mocked status?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to identify mocked oracles by looking at the create_mock() function, but it is not easy. Instead I used an heuristic that consider that all oracles are mocked if there is one create_mock.

@@ -49,6 +49,10 @@ pub(crate) struct TestCommand {
/// JSON RPC url to solve oracle calls
#[clap(long)]
oracle_resolver: Option<String>,

/// Flag to skip tests using oracles.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// Flag to skip tests using oracles.
/// Flag to skip tests using unhandled oracles without mocks.

The way it's phrased sounds like any test with an oracle will be skipped.

for brillig_function in &compiled_program.program.unconstrained_functions {
match brillig_function.get_oracle_status(ForeignCall::check_oracle_status) {
OracleResult::Mocked => {
has_oracle = false;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
has_oracle = false;
// Assume that mocks will handle any call.
has_oracle = false;

Copy link
Contributor

@aakoshh aakoshh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me, just a couple of suggestions for additional docs/comments. I'll defer to @TomAFrench whether this should be the default on CI or the env var he did that detects a failure after attempting to run the test.

Copy link
Contributor

github-actions bot commented Dec 6, 2024

FYI @noir-lang/developerrelations on Noir doc changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation run-external-checks Trigger CI job to run tests on external repos
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add flag to nargo test to ignore failures due to missing foreign calls
2 participants