Regression: libxdc decode error when fuzzing with 6.0 Nyx kernel

[Wenzel]

We've spotted a regression starting from the 6.0 Nyx kernel where libxdc would continously throw libxdc_decode errors as soon as fuzzing starts for certain targets:

A complete repro is available for the Zephyr target, in development:

git clone https://github.com/IntelLabs/kAFL.git -b regression/6.0_tnt_zephyr
cd kAFL
make deploy
# let kafl setup, reboot ...
cd kAFL/kafl/examples/zephyr_x86_32
docker build -t kafl_zephyr .
docker run -ti --rm -v $PWD:/workdir -v $PWD/../nyx_api.h:/usr/local/include/nyx_api.h kafl_zephyr
./run.sh build TEST
# CTRL-D (exit container)
./run.sh fuzz

On the previous 5.10.73 kernel, these errors doesn't appear.

ping @schumilo, since you worked on the 6.0 patches, do you have any idea what could be the cause ? (the main changes with 6.0 is that we remove custom FDL code)

[schumilo]

Sorry, I haven't had the time to long into this earlier.
At the moment I can't reproduce the issue due to another issue I encounter while trying to run the the Docker build command:

ERROR: update failed for projects: hal_silabs, littlefs
The command '/bin/bash -c west update' returned a non-zero code: 1

Am I missing something important here?

[Wenzel]

Hey @schumilo

Just tried to build the Docker container my machine, the west update line worked fine:

I don't understand how you got that output ?
A transient network issue maybe ?
Can you give more context ?

[schumilo]

Yeah, it seemed to be a network issue. Modifying the Dockerfile and running west update twice fixed it for me. Anyway, I can reproduce and confirm the decoder issue with kernel 6.0 and will take a look at it tomorrow.

[schumilo]

Quick update: I don't believe it has anything to do with using the dirty ring implementation in kernel 6.0. In fact, there is a hidden option in QEMU-Nyx to debug both FDL and dirty ring by men-comparing every guest page (and restoring ones that we might have missed by using either FDL or dirty ring), and even with this mode enabled and having the certainty that the guest memory is properly restored, the issue is still there.

Something seems to be off with kernel 6.0 and its vmx-pt patches...

[schumilo]

Hey @Wenzel!
Unfortunately, it has took me a bit longer than expected to finally find the time to troubleshoot the issue.
However, the bug should now be fixed with the following commits:

nyx-fuzz/QEMU-Nyx@5a25709
nyx-fuzz/QEMU-Nyx@d5f0efe
nyx-fuzz/QEMU-Nyx@02a6f2a

In addition, I also updated libxdc to add a sanity check to prevent empty filter configurations from being accepted, as this was what caused the bug in first place.

nyx-fuzz/libxdc@78f0fb6...b99a519#diff-bcc7456c8245c331ef06076f30589730d67ffab606bd07bd1a22e79e6db1e49e

[Wenzel]

Hi @schumilo ! 😃

Thank you for the feedback this morning, and investigating the issue !!
I confirm this fix works on the Zephyr and Windows userspace examples, where we observed the bug initially.

I already merged your commits into kAFL's repository, rolled out new released for QEMU and libxdc, and I'm preparing a new kAFL release:
IntelLabs/kAFL#253

🚀