Patching upstream sources

Ideally we would pull upstream sources directly into liboqs without modification. Doing so helps auditors and ensures that we can easily accept new code from upstream. But sometimes we have to make changes to the upstream code. So the copy_from_upstream.py script includes a patching mechanism.

Patches should be used sparingly. They should not be used in place of shims (https://github.com/open-quantum-safe/liboqs/tree/main/src/common/pqclean_shims) for minor API differences. If possible, patches should be offered back upstream so that liboqs only has to maintain patches temporarily.

How to patch upstream sources through copy_from_upstream.py

If you need to patch some upstream code, run ./copy_from_upstream.py -k verify from the scripts/copy_from_upstream directory. The output should look something like this:

-----
Total schemes: 161 - 161 match upstream up to local patches, 0 differ
-----
Patches applied:
	pqclean-sphincs.patch
	pqcrystals-kyber-yml.patch
	pqcrystals-kyber-ref-shake.patch
	pqcrystals-kyber-avx2-shake.patch
	pqcrystals-dilithium-yml.patch
	pqcrystals-dilithium-ref-shake.patch
	pqcrystals-dilithium-avx2-shake.patch
-----

You will now find a scripts/copy_from_upstream/repos directory with several git repositories inside of it. The commit log of each repo looks like

[upstream commit] -> [patch 1] -> [patch 2] -> ...

The patches come from patch files in scripts/copy_from_upstream/patches. The upstream commit ID and the list of patches is stored in copy_from_upstream.yml. Each time you run copy_from_upstream.py the repos are modified as follows:

git fetch --depth=1 origin upstream_commit
git reset --hard upstream_commit
for each patch in the patches list
  git apply patch
  git add .
  git commit -m 'Applied [PATCH FILE NAME]'
end

To create your own patch, edit the code in scripts/copy_from_upstream/repos, commit your changes, and then export your changes to a patch file in scripts/copy_from_upstream/patches using the command

git diff-tree -p [COMMIT] > [COPY_FROM_UPSTREAM_DIR]/patches/[PATCH_NAME]

Finally, add the name of your patch to the appropriate "patches" list in copy_from_upstream.yml.

Feel free to make multiple commits, edit the commit history, etc. Just make sure that all of your work will be reconstructed by the above sequence of git apply calls.

You can now run ./copy_from_upstream.py -k copy to pull your patched upstream into liboqs.