Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support new exec sol compat init API for debugging #93

Merged
merged 1 commit into from
Oct 22, 2024
Merged

Support new exec sol compat init API for debugging #93

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 16 additions & 12 deletions commands.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,13 +44,14 @@ $ solana-test-suite create-fixtures [OPTIONS]
**Options**:

* `-i, --input-dir PATH`: Either a file or directory containing InstrContext messages [default: corpus8]
* `-s, --solana-target PATH`: Solana (or ground truth) shared object (.so) target file path [default: /home/kbhargava/repos/solfuzz-agave/target/release/libsolfuzz_agave.so]
* `-s, --solana-target PATH`: Solana (or ground truth) shared object (.so) target file path [default: impl/lib/libsolfuzz_agave_v2.0.so]
* `-t, --target PATH`: Shared object (.so) target file paths (pairs with --keep-passing). Targets must have sol_compat_instr_execute_v1 defined
* `-o, --output-dir PATH`: Output directory for fixtures [default: test_fixtures]
* `-p, --num-processes INTEGER`: Number of processes to use [default: 4]
* `-r, --readable`: Output fixtures in human-readable format
* `-k, --keep-passing`: Only keep passing test cases
* `-g, --group-by-program`: Group fixture output by program type
* `-l, --log-level INTEGER`: FD logging level [default: 5]
* `--help`: Show this message and exit.

## `solana-test-suite debug-instr`
Expand Down Expand Up @@ -82,12 +83,12 @@ $ solana-test-suite debug-mismatches [OPTIONS]

**Options**:

* `-s, --solana-target PATH`: Solana (or ground truth) shared object (.so) target file path [default: /home/kbhargava/repos/solfuzz-agave/target/release/libsolfuzz_agave.so]
* `-t, --target PATH`: Shared object (.so) target file paths (pairs with --keep-passing). Targets must have sol_compat_instr_execute_v1 defined [default: /home/kbhargava/repos/firedancer/build/native/gcc/lib/libfd_exec_sol_compat.so]
* `-s, --solana-target PATH`: Solana (or ground truth) shared object (.so) target file path [default: impl/lib/libsolfuzz_agave_v2.0.so]
* `-t, --target PATH`: Shared object (.so) target file paths (pairs with --keep-passing). Targets must have sol_compat_instr_execute_v1 defined [default: impl/lib/libsolfuzz_firedancer.so]
* `-o, --output-dir PATH`: Output directory for InstrContext messages [default: debug_mismatch]
* `-u, --repro-urls TEXT`: Comma-delimited list of FuzzCorp mismatch links
* `-s, --section-names TEXT`: Comma-delimited list of FuzzCorp section names
* `-f, --fuzzcorp-url TEXT`: Comma-delimited list of FuzzCorp section names [default: https://api.dev.fuzzcorp.asymmetric.re/uglyweb/firedancer-io/solfuzz/bugs/]
* `-f, --fuzzcorp-url TEXT`: Comma-delimited list of FuzzCorp section names
* `--help`: Show this message and exit.

## `solana-test-suite debug-non-repros`
Expand All @@ -104,12 +105,12 @@ $ solana-test-suite debug-non-repros [OPTIONS]

**Options**:

* `-s, --solana-target PATH`: Solana (or ground truth) shared object (.so) target file path [default: /home/kbhargava/repos/solfuzz-agave/target/release/libsolfuzz_agave.so]
* `-t, --target PATH`: Shared object (.so) target file paths (pairs with --keep-passing). Targets must have sol_compat_instr_execute_v1 defined [default: /home/kbhargava/repos/firedancer/build/native/gcc/lib/libfd_exec_sol_compat.so]
* `-s, --solana-target PATH`: Solana (or ground truth) shared object (.so) target file path [default: impl/lib/libsolfuzz_agave_v2.0.so]
* `-t, --target PATH`: Shared object (.so) target file paths (pairs with --keep-passing). Targets must have sol_compat_instr_execute_v1 defined [default: impl/lib/libsolfuzz_firedancer.so]
* `-o, --output-dir PATH`: Output directory for InstrContext messages [default: debug_mismatch]
* `-u, --repro-urls TEXT`: Comma-delimited list of FuzzCorp mismatch links
* `-s, --section-names TEXT`: Comma-delimited list of FuzzCorp section names
* `-f, --fuzzcorp-url TEXT`: Comma-delimited list of FuzzCorp section names [default: https://api.dev.fuzzcorp.asymmetric.re/uglyweb/firedancer-io/solfuzz/bugs/]
* `-f, --fuzzcorp-url TEXT`: Comma-delimited list of FuzzCorp section names
* `--help`: Show this message and exit.

## `solana-test-suite decode-protobuf`
Expand Down Expand Up @@ -144,6 +145,7 @@ $ solana-test-suite exec-instr [OPTIONS]
* `-i, --input PATH`: Input InstrContext file or directory of files
* `-t, --target PATH`: Shared object (.so) target file path to execute [default: impl/firedancer/build/native/clang/lib/libfd_exec_sol_compat.so]
* `-r, --randomize-output-buffer`: Randomizes bytes in output buffer before shared library execution
* `-l, --log-level INTEGER`: FD logging level [default: 2]
* `--help`: Show this message and exit.

## `solana-test-suite instr-from-fixtures`
Expand Down Expand Up @@ -191,8 +193,8 @@ $ solana-test-suite regenerate-all-fixtures [OPTIONS]

* `-i, --input-dir PATH`: Input test-vectors directory [default: corpus8]
* `-o, --output-dir PATH`: Output directory for regenerated fixtures [default: /tmp/regenerated_fixtures]
* `-t, --target PATH`: Shared object (.so) target file path to execute [default: /home/kbhargava/repos/solfuzz-agave/target/release/libsolfuzz_agave.so]
* `-s, --stubbed-target PATH`: Stubbed shared object (.so) target file path to execute [default: /home/kbhargava/repos/solfuzz-agave/target/x86_64-unknown-linux-gnu/release/libsolfuzz_agave_stubbed.so]
* `-t, --target PATH`: Shared object (.so) target file path to execute [default: impl/lib/libsolfuzz_agave_v2.0.so]
* `-s, --stubbed-target PATH`: Stubbed shared object (.so) target file path to execute [default: impl/lib/libsolfuzz_firedancer.so]
* `--help`: Show this message and exit.

## `solana-test-suite regenerate-fixtures`
Expand All @@ -209,10 +211,11 @@ $ solana-test-suite regenerate-fixtures [OPTIONS]
**Options**:

* `-i, --input-dir PATH`: Either a file or directory containing InstrFixture messages [default: corpus8]
* `-t, --target PATH`: Shared object (.so) target file path to execute [default: /home/kbhargava/repos/solfuzz-agave/target/release/libsolfuzz_agave.so]
* `-t, --target PATH`: Shared object (.so) target file path to execute [default: impl/lib/libsolfuzz_agave_v2.0.so]
* `-o, --output-dir PATH`: Output directory for regenerated fixtures [default: regenerated_fixtures]
* `-d, --dry-run`: Only print the fixtures that would be regenerated
* `-a, --all-fixtures`: Regenerate all fixtures, regardless of FeatureSet compatibility. Will apply minimum compatible features.
* `-l, --log-level INTEGER`: FD logging level [default: 5]
* `--help`: Show this message and exit.

## `solana-test-suite run-tests`
Expand All @@ -231,8 +234,8 @@ $ solana-test-suite run-tests [OPTIONS]
**Options**:

* `-i, --input PATH`: Single input file or input directory containing InstrContext or InstrFixture messages [default: corpus8]
* `-s, --solana-target PATH`: Solana (or ground truth) shared object (.so) target file path [default: /home/kbhargava/repos/solfuzz-agave/target/release/libsolfuzz_agave.so]
* `-t, --target PATH`: Shared object (.so) target file paths [default: /home/kbhargava/repos/firedancer/build/native/gcc/lib/libfd_exec_sol_compat.so]
* `-s, --solana-target PATH`: Solana (or ground truth) shared object (.so) target file path [default: impl/lib/libsolfuzz_agave_v2.0.so]
* `-t, --target PATH`: Shared object (.so) target file paths [default: impl/lib/libsolfuzz_firedancer.so]
* `-o, --output-dir PATH`: Output directory for test results [default: test_results]
* `-p, --num-processes INTEGER`: Number of processes to use [default: 4]
* `-r, --randomize-output-buffer`: Randomizes bytes in output buffer before shared library execution
Expand All @@ -241,4 +244,5 @@ $ solana-test-suite run-tests [OPTIONS]
* `-c, --consensus-mode`: Only fail on consensus failures. One such effect is to normalize error codes when comparing results
* `-f, --failures-only`: Only log failed test cases
* `-sf, --save-failures`: Saves failed test cases to results directory
* `-l, --log-level INTEGER`: FD logging level [default: 5]
* `--help`: Show this message and exit.
32 changes: 28 additions & 4 deletions src/test_suite/test_suite.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,14 +78,20 @@ def exec_instr(
"-r",
help="Randomizes bytes in output buffer before shared library execution",
),
log_level: int = typer.Option(
2,
"--log-level",
"-l",
help="FD logging level",
),
):
# Initialize output buffers and shared library
initialize_process_output_buffers(randomize_output_buffer=randomize_output_buffer)
try:
lib = ctypes.CDLL(shared_library)
except:
set_ld_preload_asan()
lib.sol_compat_init()
lib.sol_compat_init(log_level)

files_to_exec = file_or_dir.iterdir() if file_or_dir.is_dir() else [file_or_dir]
for file in files_to_exec:
Expand Down Expand Up @@ -236,6 +242,12 @@ def create_fixtures(
organize_fixture_dir: bool = typer.Option(
False, "--group-by-program", "-g", help="Group fixture output by program type"
),
log_level: int = typer.Option(
5,
"--log-level",
"-l",
help="FD logging level",
),
):
# Add Solana library to shared libraries
shared_libraries = [reference_shared_library] + shared_libraries
Expand All @@ -256,7 +268,7 @@ def create_fixtures(
for target in shared_libraries:
# Load in and initialize shared libraries
lib = ctypes.CDLL(target)
lib.sol_compat_init()
lib.sol_compat_init(log_level)
globals.target_libraries[target] = lib

test_cases = [input_path] if input_path.is_file() else list(input_path.iterdir())
Expand Down Expand Up @@ -357,6 +369,12 @@ def run_tests(
"-sf",
help="Saves failed test cases to results directory",
),
log_level: int = typer.Option(
5,
"--log-level",
"-l",
help="FD logging level",
),
):
# Add Solana library to shared libraries
shared_libraries = [reference_shared_library] + shared_libraries
Expand All @@ -380,7 +398,7 @@ def run_tests(
for target in shared_libraries:
# Load in and initialize shared libraries
lib = ctypes.CDLL(target)
lib.sol_compat_init()
lib.sol_compat_init(log_level)
globals.target_libraries[target] = lib

# Make log output directories for each shared library
Expand Down Expand Up @@ -827,6 +845,12 @@ def regenerate_fixtures(
"-a",
help="Regenerate all fixtures, regardless of FeatureSet compatibility. Will apply minimum compatible features.",
),
log_level: int = typer.Option(
5,
"--log-level",
"-l",
help="FD logging level",
),
):
globals.output_dir = output_dir
globals.reference_shared_library = shared_library
Expand All @@ -836,7 +860,7 @@ def regenerate_fixtures(
globals.output_dir.mkdir(parents=True, exist_ok=True)

lib: ctypes.CDLL = ctypes.CDLL(shared_library)
lib.sol_compat_init()
lib.sol_compat_init(log_level)
globals.target_libraries[shared_library] = lib
initialize_process_output_buffers()

Expand Down
Loading