Support for Python 3.12

[tonybaloney]

This adds support for Python 3.12 (so far, the release is months away).

PRECALL and LOAD_METHOD have been removed. So the if-macro that says version >= 3.11 would be invalid for all future releases.

JUMP_IF_TRUE_OR_POP and JUMP_IF_FALSE_OR_POP have been removed.

[tonybaloney]

Almost working, it seems to be mixing up the CALL opcode so it's trying to call len on self (the module) even though it's a builtin

python ../atheris/example_fuzzers/custom_mutator_example.py
python(31049,0x7ff847b44340) malloc: nano zone abandoned due to inability to reserve vm space.
INFO: Using preloaded libfuzzer
INFO: found LLVMFuzzerCustomMutator (0x10bcbbad0). Disabling -len_control by default.
INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 1565970448
INFO: Loaded 1 modules   (16677 inline 8-bit counters): 16677 [0x10c204b88, 0x10c208cad),
INFO: Loaded 1 PC tables (16677 PCs): 16677 [0x10c208cb0,0x10c249f00),
INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes
INFO: A corpus is not provided, starting from an empty corpus
#2	INITED cov: 45 ft: 45 corp: 1/1b exec/s: 0 rss: 91Mb

 === Uncaught Python exception: ===
AttributeError: module 'atheris' has no attribute 'len'
Traceback (most recent call last):
  File "/Users/anthonyshaw/projects/cpython/../atheris/example_fuzzers/custom_mutator_example.py", line -1, in TestOneInput
AttributeError: module 'atheris' has no attribute 'len'

==31049== ERROR: libFuzzer: fuzz target exited
    #0 0x108cd24a5 in __sanitizer_print_stack_trace+0x35 (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x544a5) (BuildId: 756bb7515781379f84412f22c4274ffd2400000010000000000a0a0000030d00)
    #1 0x10c1d1db8 in fuzzer::PrintStackTrace() FuzzerUtil.cpp:210
    #2 0x10c1b3d0c in fuzzer::Fuzzer::ExitCallback() FuzzerLoop.cpp:250
    #3 0x7ff804342ba7 in __cxa_finalize_ranges+0x19f (libsystem_c.dylib:x86_64+0x2aba7) (BuildId: 0773ddbc707e3b56ad3e97aaa9b2c3ed32000000200000000100000000030d00)
    #4 0x7ff8043429ba in exit+0x22 (libsystem_c.dylib:x86_64+0x2a9ba) (BuildId: 0773ddbc707e3b56ad3e97aaa9b2c3ed32000000200000000100000000030d00)
    #5 0x10792e93f in Py_Exit pylifecycle.c:2988
    #6 0x107948778 in _PyErr_PrintEx pythonrun.c
    #7 0x107945995 in _PyRun_SimpleFileObject pythonrun.c:439
    #8 0x107944771 in _PyRun_AnyFileObject pythonrun.c:78
    #9 0x1079c0096 in Py_RunMain main.c:689
    #10 0x1079c1344 in pymain_main main.c:719
    #11 0x1079c1657 in Py_BytesMain main.c:743
    #12 0x7ff80411741e in start+0x76e (dyld:x86_64+0xfffffffffff6e41e) (BuildId: f22a114397323e23a8b7cbade6bb830132000000200000000100000000030d00)

SUMMARY: libFuzzer: fuzz target exited
MS: 1 Custom-; base unit: adc83b19e793491b1c6ea0fd8b46cd9f32e592fc
0x78,0x9c,0xf3,0xc8,0x4,0x0,0x0,0xfb,0x0,0xb2,
x\234\363\310\004\000\000\373\000\262
artifact_prefix='./'; Test unit written to ./crash-34d8a0eeba0ec73df6e771631fc49f68dedfc122
Base64: eJzzyAQAAPsAsg==

[AidenRHall]

Thanks for writing this up Tony! Our project is definitely sensitive to these kinds of changes and they can be somewhat tricky to debug. Please let us know if we can help in any way :)

[n-bes]

Python 3.13 is coming. Any updates here?

[ranvit]

@tonybaloney @AidenRHall

I think we need to match opcode_caches against the definition of _PyOpcode_Caches in cpython 3.12, seen here

so its not sufficient to delete PRECALL, we need to update a bunch of opcodes' sizes.

And its changing even further in cpython 3.13

I'm not familiar with cpython, so

• idk why the opcode_cache is being redefined in this package
• idk if theres other cpython internals being redefined, that also need to be kept in sync across python versions
• idk if there's a more scalable way to keep this package sync'd -- perhaps a build step that automates the retrieval of opcode cache sizes and any other cpython internals?

Or, you could stop supporting backwards compatibility and update src/native/codetable_gen.cc per cpython version? Or throw a bunch of if/else at it.

Hope this helps someone get started on the enhancement! 🫡