Random Byte Changes vs NOPs

[asdfjkl]

First of all, thanks very much for providing this convenient python implementation!

I would like to point out one thing however: In your blog-post on the topic ( https://blog.ledger.com/whitebox_ecdsa ) you write that by changing bytes randomly you break 84/97 of the cited contest. In https://eprint.iacr.org/2022/448 a number 88/97 is reported by replacing instructions by NOPs. It would be interesting to see if changing bytes randomly breaks challenges that cannot be broken by skipping instructions, or if the first is a proper subset of the latter.

Also: Did you just randomly change byte values in both the data and code segment of the binary, or replace instructions in the code segment by (random) valid opcodes? After all, if there are invalid opcodes in the program, it is very likely to just crash without any output.

[adrianthillard]

Thanks for your feedback!

At the moment, we did not study exactly all the obtained effects on all the challenges. It would be an interesting avenue to do some statistical analysis on these effect, and, indeed, to maybe restrict the possible attack values/location to better understand the underlying effects.

The philosophy of the approach was indeed totally random and with as little human intervention as possible. Sometimes the data themselves are perturbed, and some other time the code is perturbed. The perturbation in the code can hit instructions, but it can also hit their operands. We share your observation that invalid opcodes would likely crash the program -as well as invalid addresses-. The subset of interesting fault locations/values might indeed be way smaller than our searching space. However, it requires a bit more work to be able to automatically understand whether the targeted location corresponds to an operation, an operand, or something else, and we were really aiming for a "brute" approach. This would nonetheless be a really nice improvement.