Skip to content

Commit

Permalink
[RFC PATCH v3 00/11] tcg-plugins: add hooks for discontinuities
Browse files Browse the repository at this point in the history
https://lore.kernel.org/qemu-devel/[email protected]

---

From: Julian Ganz <[email protected]>
To: [email protected]
Cc: Julian Ganz <[email protected]>
Subject: [RFC PATCH v3 00/11] tcg-plugins: add hooks for discontinuities
Date: Mon,  2 Dec 2024 20:26:41 +0100
Message-ID: <[email protected]>
X-Mailer: git-send-email 2.45.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Rspamd-Bar: -
X-Rspamd-Report: MID_CONTAINS_FROM(1) BAYES_HAM(-3) MIME_GOOD(-0.1)
 R_MISSING_CHARSET(0.5)
X-Rspamd-Score: -1.6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=nut.email; s=uberspace;
 h=from:to:cc:subject:date;
 bh=lhp/MzIhcQwQgKNSdC0sX2veCqSwD9bp274LH0tKfMo=;
 b=C+hLkkvZSYegywm++I5e4yctqswLk6sFQPI6z+7ebske4ee+HnD0d5tgbVAjQElL4f8LLFGGwE
 bL0Hn9yc9wYy++k8J4RGbD5YW2ZsWjhu+vAE90s2yd8UlMglpQJPdcl2QqlqCfB9LqS3kii29SPp
 VYrPYb4KYXmp5IYlTuZeBtynLViye83kdVd6OVjIGsiOVmg+hg4nQBd2PJEio0M3AVRUQ0zf0wEQ
 RO8Cl1FMElhPk9FpltLEU+vo8e6de+zx3g4EJiSYcsd0fdtoaZP16NwLud9kjigY2ED8m5ffnPQ4
 B3VTxo/6oHvR1c3X50pMUdRDPr5IvhK+4zuUKFFL9hyV6ddWPOGvqrDGlVkTaCcaM0XOyZ9a/E6B
 ULNQ8ZgH5VaD8vVgtygwZDOn87dbCbX6KKgrvkYFTddGTt+zyWK9keS2e/unZtRhLi3p0sB3HAF9
 X05KYn48624+OXnUeIXbI4Xq1DXUCP0LAB2Wi8Z1yyU6sSpV/X44x8feenYG1WYkfrHJ0nBiVrc9
 l/5Dsfw5qtRi+9IRcd5ySczAVhgCyuhF1jervJiCk571g2CTTtEqGkRFkqFjqAzT0owzRM90qwHF
 CyJ5in4adhTFOtd3XhaBLsuT5YO6bgVkXfrQZIhxiuktV/jHMY3QE+cGiWBQfFGL5yiBw8lscafg
 U=
Received-SPF: pass client-ip=2a00:d0c0:200:0:1c7b:a6ff:fee0:8ea4;
 [email protected]; helo=mailgate02.uberspace.is
X-Spam_score_int: -20
X-Spam_score: -2.1
X-Spam_bar: --
X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001,
 SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-BeenThere: [email protected]
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: <qemu-devel.nongnu.org>
List-Unsubscribe: <https://lists.nongnu.org/mailman/options/qemu-devel>,
 <mailto:[email protected]?subject=unsubscribe>
List-Archive: <https://lists.nongnu.org/archive/html/qemu-devel>
List-Post: <mailto:[email protected]>
List-Help: <mailto:[email protected]?subject=help>
List-Subscribe: <https://lists.nongnu.org/mailman/listinfo/qemu-devel>,
 <mailto:[email protected]?subject=subscribe>
Errors-To: [email protected]
Sender: [email protected]

Some analysis greatly benefits, or depends on, information about
certain types of dicontinuities such as interrupts. For example, we may
need to handle the execution of a new translation block differently if
it is not the result of normal program flow but of an interrupt.

Even with the existing interfaces, it is more or less possible to
discern these situations, e.g. as done by the cflow plugin. However,
this process poses a considerable overhead to the core analysis one may
intend to perform.

These changes introduce a generic and easy-to-use interface for plugin
authors in the form of a callback for discontinuities. Patch 1 defines
an enumeration of some trap-related discontinuities including somewhat
narrow definitions of the discontinuity evetns and a callback type.
Patch 2 defines the callback registration function. Patch 3 adds some
hooks for triggering the callbacks. Patch 4 adds an example plugin
showcasing the new API. Patches 5 through 6 call the hooks for a
selection of architectures, mapping architecture specific events to the
three categories defined in patch 1. Future non-RFC patchsets will call
these hooks for all architectures (that have some concept of trap or
interrupt). Finally, patch 11 supplies a test plugin asserting that the
next PC provided to the plugin points to the next instruction executed.

Sidenote: I'm likely doing something wrong for one architecture or
the other. These patches are untested for most of them.

Since v2 (tcg-plugins: add hooks for interrupts, exceptions and traps):
  - Switched from traps as core concept to more generic discontinuities
  - Switched from semihosting to hostcall as term for emulated traps
  - Added enumeration of events and dedicated callback type
  - Make callback receive event type as well as origin and target PC
    (as requested by Pierrick Bouvier)
  - Combined registration functions for different traps into a single
    one for all types of discontinuities (as requested by Pierrick
    Bouvier)
  - Migrated records in example plugin from fully pre-allocated to a
    scoreboard (as suggested by Pierrick Bouvier)
  - Handle PSCI calls as hostcall (as pointed out by Peter Maydell)
  - Added hooks for ARM Cortex M arches (as pointed out by Peter
    Maydell)
  - Added hooks for Alpha targets
  - Added hooks for MIPS targets
  - Added a plugin for testing some of the interface behaviour

Since v1:
  - Split the one callback into multiple callbacks
  - Added a target-agnostic definition of the relevant event(s)
  - Call hooks from architecture-code rather than accel/tcg/cpu-exec.c
  - Added a plugin showcasing API usage

Julian Ganz (11):
  plugins: add types for callbacks related to certain discontinuities
  plugins: add API for registering discontinuity callbacks
  plugins: add hooks for new discontinuity related callbacks
  contrib/plugins: add plugin showcasing new dicontinuity related API
  target/alpha: call plugin trap callbacks
  target/arm: call plugin trap callbacks
  target/avr: call plugin trap callbacks
  target/mips: call plugin trap callbacks
  target/riscv: call plugin trap callbacks
  target/sparc: call plugin trap callbacks
  tests: add plugin asserting correctness of discon event's to_pc

 contrib/plugins/meson.build         |  3 +-
 contrib/plugins/traps.c             | 96 +++++++++++++++++++++++++++++
 include/qemu/plugin-event.h         |  3 +
 include/qemu/plugin.h               | 13 ++++
 include/qemu/qemu-plugin.h          | 58 +++++++++++++++++
 plugins/core.c                      | 67 ++++++++++++++++++++
 target/alpha/helper.c               | 12 ++++
 target/arm/helper.c                 | 25 ++++++++
 target/arm/tcg/m_helper.c           | 18 ++++++
 target/avr/helper.c                 |  3 +
 target/mips/tcg/sysemu/tlb_helper.c | 11 ++++
 target/riscv/cpu_helper.c           |  9 +++
 target/sparc/int32_helper.c         |  7 +++
 target/sparc/int64_helper.c         | 10 +++
 tests/tcg/plugins/discons.c         | 95 ++++++++++++++++++++++++++++
 tests/tcg/plugins/meson.build       |  2 +-
 16 files changed, 430 insertions(+), 2 deletions(-)
 create mode 100644 contrib/plugins/traps.c
 create mode 100644 tests/tcg/plugins/discons.c

--
2.45.2

Signed-off-by: GitHub Actions Bot <[email protected]>
  • Loading branch information
GitHub Actions Bot committed Dec 2, 2024
1 parent 108b201 commit 8580eee
Show file tree
Hide file tree
Showing 20 changed files with 729 additions and 272 deletions.
307 changes: 307 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,307 @@
on: push

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:
checkapply:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
# to debug container live from GitHub
# - uses: mxschmitt/action-tmate@v3
- run: bash -c '[ ! -f shazam.log ] || { cat shazam.log; exit 1; }'

checkpatch-ignore-signoff:
needs: checkapply
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- run: git fetch -a origin --unshallow || true
- run: git remote add upstream -f https://gitlab.com/qemu-project/qemu
- run: ./scripts/checkpatch.pl --no-signoff $(git merge-base upstream/master HEAD)..HEAD

checkpatch-with-signoff:
needs: checkapply
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- run: git fetch -a origin --unshallow || true
- run: git remote add upstream -f https://gitlab.com/qemu-project/qemu
- run: ./scripts/checkpatch.pl $(git merge-base upstream/master HEAD)..HEAD

# use docker-run to not rebuild images
# images are built daily and pushed on pbolinaro/qemu-ci:*
build-cross:
needs: checkapply
runs-on: ubuntu-24.04
strategy:
fail-fast: false
matrix:
container: [alpine,centos9,debian,debian-all-test-cross,debian-amd64-cross,debian-arm64-cross,debian-armhf-cross,debian-hexagon-cross,debian-i686-cross,debian-legacy-test-cross,debian-loongarch-cross,debian-mips64el-cross,debian-mipsel-cross,debian-ppc64el-cross,debian-riscv64-cross,debian-s390x-cross,debian-tricore-cross,fedora,fedora-rust-nightly,opensuse-leap,ubuntu2204]
steps:
- uses: actions/checkout@v4
- run: pip install meson
- run: make docker-run J=$(nproc) RUNC=podman TEST=test-build IMAGE=docker.io/pbolinaro/qemu-ci:${{matrix.container}}

build:
needs: checkapply
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- run: >
podman run --init --rm -it -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian
bash -cx './configure && ninja -C build install'
- run: >
podman run --init --rm -it -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian
./build/qemu-system-x86_64 -nographic -plugin ./build/contrib/plugins/libstoptrigger,icount=1000000 -plugin ./build/tests/tcg/plugins/libinsn -d plugin
build-cross-mingw64:
needs: checkapply
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- run: >
podman run --init --rm -it -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:fedora-win64-cross
bash -cx './configure $QEMU_CONFIGURE_OPTS && ninja -C build install'
build-windows:
needs: checkapply
runs-on: windows-2022
strategy:
fail-fast: false
matrix:
sys: [UCRT64, CLANG64, MINGW64]
defaults:
run:
shell: msys2 {0}
steps:
- uses: msys2/setup-msys2@v2
with:
update: true
msystem: ${{matrix.sys}}
- run: pacman -S --noconfirm curl git
- uses: actions/checkout@v4
- run: >
pacman -S --noconfirm
base-devel binutils bison diffutils flex git grep make sed
${MINGW_PACKAGE_PREFIX}-toolchain
${MINGW_PACKAGE_PREFIX}-glib2
${MINGW_PACKAGE_PREFIX}-gtk3
${MINGW_PACKAGE_PREFIX}-libnfs
${MINGW_PACKAGE_PREFIX}-libssh
${MINGW_PACKAGE_PREFIX}-ninja
${MINGW_PACKAGE_PREFIX}-pixman
${MINGW_PACKAGE_PREFIX}-pkgconf
${MINGW_PACKAGE_PREFIX}-python
${MINGW_PACKAGE_PREFIX}-SDL2
${MINGW_PACKAGE_PREFIX}-zstd
- run: ./configure && ninja -C build
- run: ./build/qemu-system-x86_64 -nographic -plugin ./build/contrib/plugins/libstoptrigger,icount=1000000 -plugin ./build/tests/tcg/plugins/libinsn -d plugin

build-macos-x86_64:
needs: checkapply
runs-on: macos-13
steps:
- uses: actions/checkout@v4
- run: brew install --quiet $(brew deps --include-build qemu) || true
# on macos, werror is not on by default
- run: ./configure --enable-werror && ninja -C build
- run: ./build/qemu-system-x86_64 -nographic -plugin ./build/contrib/plugins/libstoptrigger,icount=1000000 -plugin ./build/tests/tcg/plugins/libinsn -d plugin

build-macos-aarch64:
needs: checkapply
runs-on: macos-14
steps:
- uses: actions/checkout@v4
- run: brew install --quiet $(brew deps --include-build qemu) || true
# on macos, werror is not on by default
- run: ./configure --enable-werror && ninja -C build
- run: ./build/qemu-system-x86_64 -nographic -plugin ./build/contrib/plugins/libstoptrigger,icount=1000000 -plugin ./build/tests/tcg/plugins/libinsn -d plugin

build-misc:
needs: checkapply
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- run: >
podman run --init --rm -it -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian
bash -cx './configure --disable-user --disable-system --enable-docs --enable-tools && ninja -C build install'
build-32bits:
needs: checkapply
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- run: >
podman run --init --rm -it -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian-i686-cross
bash -cx './configure $QEMU_CONFIGURE_OPTS && ninja -C build install'
build-big-endian:
needs: checkapply
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- run: >
podman run --init --rm -it -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian-s390x-cross
bash -cx './configure $QEMU_CONFIGURE_OPTS && ninja -C build install'
build-debug:
needs: checkapply
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- run: >
podman run --init --rm -it -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian
bash -cx './configure --enable-debug --enable-asan --enable-ubsan && ninja -C build install'
- run: >
podman run --init --rm -it -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian
./build/qemu-system-x86_64 -nographic -plugin ./build/contrib/plugins/libstoptrigger,icount=1000000 -plugin ./build/tests/tcg/plugins/libinsn -d plugin
build-static:
needs: checkapply
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- run: >
podman run --init --rm -it -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian
bash -cx './configure --disable-system --disable-tools --disable-guest-agent --disable-docs --static && ninja -C build install'
build-tsan:
needs: checkapply
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- run: >
podman run --init --rm -it -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian
bash -cx './configure --enable-tsan && ninja -C build install'
build-clang:
needs: checkapply
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- run: >
podman run --init --rm -it -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian
bash -cx './configure --cxx=clang++ --cc=clang --host-cc=clang && ninja -C build install'
build-clang-latest:
needs: checkapply
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- run: >
podman run --init --rm -it -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian
bash -cx 'LLVM_VERSION=19 && apt update && apt install -y lsb-release wget software-properties-common gnupg && wget https://apt.llvm.org/llvm.sh && bash llvm.sh ${LLVM_VERSION} && ./configure --cxx=clang++-${LLVM_VERSION} --cc=clang-${LLVM_VERSION} --host-cc=clang-${LLVM_VERSION} && ninja -C build install'
build-rust:
needs: checkapply
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- run: >
podman run --init --rm -it -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:fedora-rust-nightly
bash -cx './configure $QEMU_CONFIGURE_OPTS --enable-rust && ninja -C build install'
build-disable-tcg:
needs: checkapply
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- run: >
podman run --init --rm -it -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian
bash -cx './configure --disable-tcg && ninja -C build install'
build-disable-kvm:
needs: checkapply
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- run: >
podman run --init --rm -it -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian
bash -cx './configure --disable-kvm && ninja -C build install'
build-disable-tcg-kvm-for-xen:
needs: checkapply
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- run: >
podman run --init --rm -it -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian
bash -cx './configure --disable-tcg --disable-kvm && ninja -C build install'
build-minimal:
needs: checkapply
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- run: >
podman run --init --rm -it -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian
bash -cx './configure --without-default-features --without-default-devices --disable-kvm --disable-tcg && ninja -C build install'
check-tcg:
needs: checkapply
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- run: >
podman run --init --rm -it -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian-all-test-cross
bash -cx './configure $QEMU_CONFIGURE_OPTS --enable-debug-tcg && ninja -C build'
- run: >
podman run --init --privileged --rm -it -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian-all-test-cross
bash -cx "make -k check-tcg"
# run all meson tests, including functional. Run -j1 to avoid sporadic issues.
check:
needs: checkapply
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
# we use image with download cache filled. Solves servers flakiness.
- run: >
podman run --init --rm -it -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian-precache-tests
bash -cx './configure --enable-debug-tcg && ninja -C build'
- run: sudo chown $USER:$USER /dev/kvm
- run: >
podman run --init --privileged --rm -it -v /dev/kvm:/dev/kvm -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian-precache-tests
bash -cx "make -k check SPEED=thorough TIMEOUT_MULTIPLIER=5"
check-avocado:
needs: checkapply
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
# add more time for all tests
- run: sed -i -e 's/timeout = .*/timeout = 3600/' $(find tests/avocado/ -type f)
# we use image with download cache filled. Solves servers flakiness.
- run: >
podman run --init --rm -it -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian-precache-tests
bash -cx './configure --enable-debug-tcg && ninja -C build'
- run: sudo chown $USER:$USER /dev/kvm
- run: >
podman run --init --privileged --rm -it -v /dev/kvm:/dev/kvm -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian-precache-tests
bash -cx "make -k check-avocado"
54 changes: 54 additions & 0 deletions .github/workflows/containers.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
on:
schedule:
- cron: '0 6 * * *'
workflow_dispatch:

permissions: write-all

jobs:
build_container:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
# cd tests/docker/dockerfiles/
# ls *docker | sed -e 's/.docker//' | tr '\n' ','
# remove: debian-bootstrap,debian-toolchain
container: [alpine,centos9,debian-all-test-cross,debian-amd64-cross,debian-arm64-cross,debian-armhf-cross,debian,debian-hexagon-cross,debian-i686-cross,debian-legacy-test-cross,debian-loongarch-cross,debian-mips64el-cross,debian-mipsel-cross,debian-ppc64el-cross,debian-riscv64-cross,debian-s390x-cross,debian-tricore-cross,debian-xtensa-cross,fedora,fedora-rust-nightly,fedora-win64-cross,opensuse-leap,python,ubuntu2204]
steps:
- uses: actions/checkout@v4
- run: make docker-image-${{matrix.container}} RUNC=podman V=1
- run: podman tag qemu/${{matrix.container}} docker.io/pbolinaro/qemu-ci:${{matrix.container}}
- run: podman login -u pbolinaro -p ${{secrets.DOCKERHUB_PASSWORD}}
- run: podman push docker.io/pbolinaro/qemu-ci:${{matrix.container}}

build_container_debian-precache-tests:
runs-on: ubuntu-latest
steps:
# we clean up runner first, to get more disk space
- run: docker system prune -af && sudo rm -rf /opt/*
- uses: actions/checkout@v4
- run: make docker-image-debian RUNC=podman V=1
# fill download cache for functional and check-avocado
# running check-avocado without any qemu binary will only download data
# in /root/avocado
- run: >
podman run -it -v $(pwd):$(pwd) -w $(pwd) qemu/debian
./precache_tests.sh
# commit result as a new image. Cache will be in /root/.cache and /root/avocado
- run: podman commit "$(podman ps -aq)" docker.io/pbolinaro/qemu-ci:debian-precache-tests
- run: podman login -u pbolinaro -p ${{secrets.DOCKERHUB_PASSWORD}}
- run: podman push docker.io/pbolinaro/qemu-ci:debian-precache-tests

keepalive-job:
name: Keepalive Workflow
if: ${{ always() }}
needs: build_container
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- uses: actions/checkout@v4
- uses: gautamkrishnar/keepalive-workflow@v2
with:
use_api: false
Loading

0 comments on commit 8580eee

Please sign in to comment.