Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement runtime with dynamic dispatch #1051

Open
wants to merge 10 commits into
base: main
Choose a base branch
from
Open

Conversation

JulianGCalderon
Copy link
Contributor

@JulianGCalderon JulianGCalderon commented Jan 14, 2025

Implements runtime with dynamic dispatch.

Depends on #1052

Previously, runtime function were defined in a different crate cairo_native_runtime, compiled as a static library (.a), and linked to the starknet contract.

This PR gets rid of the runtime static library completely, by defining a global for each runtime function. On execution, it will access the global to find the runtime function. This global is originally empty, but when loading the executor, the caller sets the globals, making them point to the correct functions (located in the callers binary).

This PR is a POC and should be tested thoroughly. As this new approach adds an additional level of indirection when calling the runtime, we should check that it doesn't introduce benchmark regressions.

Implementation Details

The build_function method declares the global if not yet declared, accesses it, and calls the function.

The setup_runtime function sets the value of all defined globals with their corresponding location. It is called when loading the executors.

Since we are using rust's runtime, instead of contract's runtime, we can call set_costs_builtin directly, instead of looking for the symbol.

I removed the hyperfine standalone AOT benches, as adapting it to the new dynamic runtime is difficult, and probably not worth it.

Roadmap

  • Adapt all runtime bindings
  • Remove original runtime link (AOT)
  • Remove original runtime register (JIT)
  • Compare execution with VM on many blocks
    • From 900000 to 900484 without diffs
    • From 1000000 to 1000500 without diffs
  • Measure performance: 5% to 10% improvement
    • From 900000 to 900010:
      • With static Runtime: 1.78s
      • With dynamic Runtime: 1.64s
    • From 900010 to 900020:
      • With static Runtime: 2.81s
      • With dynamic Runtime: 2.57s
  • Measure compiled programs size: 10X improvement
    • Average complied library with static runtime: 13MiB
    • Average compiled library with dynamic runtime: 450KiB

Why is it faster? We should take a closer look at it.

Checklist

  • Linked to Github Issue
  • Unit tests added
  • Integration tests added.
  • This change requires new documentation.
    • Documentation has been added/updated.

Copy link

github-actions bot commented Jan 14, 2025

Benchmark results Main vs HEAD.

Command Mean [s] Min [s] Max [s] Relative
base dict_insert.cairo (JIT) 3.908 ± 0.041 3.822 3.963 1.00
base dict_insert.cairo (AOT) 4.041 ± 0.051 3.985 4.136 1.03 ± 0.02
Command Mean [s] Min [s] Max [s] Relative
:--- ---: ---: ---: ---:
head dict_insert.cairo (JIT) 3.846 ± 0.021 3.814 3.881 1.02 ± 0.01
head dict_insert.cairo (AOT) 3.782 ± 0.021 3.745 3.829 1.00
Command Mean [s] Min [s] Max [s] Relative
base dict_snapshot.cairo (JIT) 3.774 ± 0.045 3.710 3.851 1.00
base dict_snapshot.cairo (AOT) 3.865 ± 0.018 3.841 3.894 1.02 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
:--- ---: ---: ---: ---:
head dict_snapshot.cairo (JIT) 3.803 ± 0.026 3.763 3.845 1.03 ± 0.01
head dict_snapshot.cairo (AOT) 3.688 ± 0.023 3.649 3.714 1.00
Command Mean [s] Min [s] Max [s] Relative
base factorial_2M.cairo (JIT) 4.062 ± 0.017 4.039 4.083 1.00
base factorial_2M.cairo (AOT) 4.275 ± 0.033 4.230 4.339 1.05 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
:--- ---: ---: ---: ---:
head factorial_2M.cairo (JIT) 4.159 ± 0.019 4.130 4.186 1.02 ± 0.01
head factorial_2M.cairo (AOT) 4.077 ± 0.036 4.045 4.164 1.00
Command Mean [s] Min [s] Max [s] Relative
base fib_2M.cairo (JIT) 3.609 ± 0.025 3.557 3.647 1.00
base fib_2M.cairo (AOT) 3.845 ± 0.031 3.788 3.903 1.07 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
:--- ---: ---: ---: ---:
head fib_2M.cairo (JIT) 3.675 ± 0.058 3.597 3.747 1.02 ± 0.02
head fib_2M.cairo (AOT) 3.599 ± 0.017 3.578 3.633 1.00
Command Mean [s] Min [s] Max [s] Relative
base linear_search.cairo (JIT) 3.839 ± 0.025 3.808 3.885 1.00
base linear_search.cairo (AOT) 3.942 ± 0.044 3.887 4.041 1.03 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
:--- ---: ---: ---: ---:
head linear_search.cairo (JIT) 3.813 ± 0.019 3.786 3.840 1.02 ± 0.01
head linear_search.cairo (AOT) 3.726 ± 0.025 3.692 3.767 1.00
Command Mean [s] Min [s] Max [s] Relative
base logistic_map.cairo (JIT) 3.912 ± 0.036 3.863 3.965 1.00
base logistic_map.cairo (AOT) 3.960 ± 0.015 3.934 3.983 1.01 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
:--- ---: ---: ---: ---:
head logistic_map.cairo (JIT) 3.957 ± 0.024 3.921 3.994 1.04 ± 0.01
head logistic_map.cairo (AOT) 3.809 ± 0.027 3.774 3.860 1.00

@codecov-commenter
Copy link

codecov-commenter commented Jan 14, 2025

Codecov Report

Attention: Patch coverage is 98.13433% with 5 lines in your changes missing coverage. Please review.

Project coverage is 80.82%. Comparing base (1191808) to head (1880fbe).

Files with missing lines Patch % Lines
src/executor/contract.rs 71.42% 4 Missing ⚠️
src/metadata/runtime_bindings.rs 99.57% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1051      +/-   ##
==========================================
- Coverage   80.95%   80.82%   -0.14%     
==========================================
  Files         108      108              
  Lines       29938    29517     -421     
==========================================
- Hits        24237    23856     -381     
+ Misses       5701     5661      -40     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link

github-actions bot commented Jan 14, 2025

Benchmarking results

Benchmark for program dict_insert

Open benchmarks
Command Mean [s] Min [s] Max [s] Relative
Cairo-vm (Rust, Cairo 1) 20.981 ± 0.208 20.790 21.512 5.53 ± 0.12
cairo-native (embedded AOT) 3.795 ± 0.072 3.699 3.899 1.00
cairo-native (embedded JIT using LLVM's ORC Engine) 3.826 ± 0.024 3.785 3.871 1.01 ± 0.02

Benchmark for program dict_snapshot

Open benchmarks
Command Mean [s] Min [s] Max [s] Relative
Cairo-vm (Rust, Cairo 1) 5.877 ± 0.090 5.793 6.041 1.56 ± 0.03
cairo-native (embedded AOT) 3.798 ± 0.062 3.694 3.901 1.01 ± 0.02
cairo-native (embedded JIT using LLVM's ORC Engine) 3.768 ± 0.045 3.721 3.847 1.00

Benchmark for program factorial_2M

Open benchmarks
Command Mean [s] Min [s] Max [s] Relative
Cairo-vm (Rust, Cairo 1) 15.201 ± 0.141 14.958 15.345 3.75 ± 0.05
cairo-native (embedded AOT) 4.053 ± 0.037 4.002 4.102 1.00
cairo-native (embedded JIT using LLVM's ORC Engine) 4.186 ± 0.064 4.099 4.274 1.03 ± 0.02

Benchmark for program fib_2M

Open benchmarks
Command Mean [s] Min [s] Max [s] Relative
Cairo-vm (Rust, Cairo 1) 14.916 ± 0.103 14.791 15.083 4.20 ± 0.06
cairo-native (embedded AOT) 3.552 ± 0.041 3.506 3.619 1.00
cairo-native (embedded JIT using LLVM's ORC Engine) 3.632 ± 0.055 3.549 3.714 1.02 ± 0.02

Benchmark for program linear_search

Open benchmarks
Command Mean [s] Min [s] Max [s] Relative
Cairo-vm (Rust, Cairo 1) 5.896 ± 0.069 5.781 5.995 1.60 ± 0.03
cairo-native (embedded AOT) 3.694 ± 0.053 3.642 3.783 1.00
cairo-native (embedded JIT using LLVM's ORC Engine) 3.830 ± 0.035 3.778 3.892 1.04 ± 0.02

Benchmark for program logistic_map

Open benchmarks
Command Mean [s] Min [s] Max [s] Relative
Cairo-vm (Rust, Cairo 1) 5.853 ± 0.087 5.745 6.004 1.48 ± 0.03
cairo-native (embedded AOT) 3.954 ± 0.046 3.871 4.021 1.00
cairo-native (embedded JIT using LLVM's ORC Engine) 4.097 ± 0.031 4.060 4.156 1.04 ± 0.01

@JulianGCalderon JulianGCalderon force-pushed the dynamic-runtime branch 11 times, most recently from 94ca074 to be15f36 Compare January 15, 2025 18:49
@JulianGCalderon JulianGCalderon changed the base branch from main to fix-ci January 15, 2025 18:49
@JulianGCalderon JulianGCalderon force-pushed the dynamic-runtime branch 5 times, most recently from 23138a4 to 5005809 Compare January 15, 2025 19:33
@JulianGCalderon JulianGCalderon changed the base branch from fix-ci to main January 15, 2025 19:35
@JulianGCalderon JulianGCalderon force-pushed the dynamic-runtime branch 3 times, most recently from f62b6d7 to 46872ae Compare January 15, 2025 21:11
src/executor/contract.rs Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants