Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(linux): Move API verification to separate action #10637

Merged
merged 1 commit into from
May 21, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
120 changes: 120 additions & 0 deletions .github/workflows/api-verification.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
name: "API Verification"
on:
workflow_run:
workflows: [Ubuntu packaging]
types:
- completed

env:
STATUS_CONTEXT: 'API Verification'
PKG_NAME: 'libkeymancore'

jobs:
setup_environment:
runs-on: ubuntu-latest
outputs:
VERSION: ${{ steps.environment_step.outputs.VERSION }}
PRERELEASE_TAG: ${{ steps.environment_step.outputs.PRERELEASE_TAG }}
GIT_SHA: ${{ steps.environment_step.outputs.GIT_SHA }}
GIT_BASE: ${{ steps.environment_step.outputs.GIT_BASE }}
IS_TEST_BUILD: ${{ steps.environment_step.outputs.IS_TEST_BUILD }}
GIT_BRANCH: ${{ steps.environment_step.outputs.GIT_BRANCH }}
GIT_BASE_BRANCH: ${{ steps.environment_step.outputs.GIT_BASE_BRANCH }}
GIT_USER: ${{ steps.environment_step.outputs.GIT_USER }}

steps:
- name: Restore artifacts
uses: actions/cache/restore@13aacd865c20de90d75de3b17ebe84f7a17d57d2 # v4.0.0
with:
path: |
artifacts
key: artifacts-key-${GITHUB_RUN_ID}
restore-keys: artifacts-key-

- name: Read environment
id: environment_step
run: |
cat artifacts/env >> $GITHUB_OUTPUT

api_verification:
name: Verify API for libkeymancore.so
needs: setup_environment
runs-on: ubuntu-latest

steps:
- name: Set pending status on PR builds
id: set_status
if: needs.setup_environment.outputs.IS_TEST_BUILD == 'true'
shell: bash
run: |
gh api \
--method POST \
-H "Accept: application/vnd.github+json" \
/repos/$GITHUB_REPOSITORY/statuses/${{ needs.setup_environment.outputs.GIT_SHA }} \
-f state='pending' \
-f target_url="$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID" \
-f description='API verification started' \
-f context="$STATUS_CONTEXT"

- name: Checkout
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 #v4.1.1
with:
ref: '${{ needs.setup_environment.outputs.GIT_SHA }}'
fetch-depth: 0

- name: Install devscripts
uses: ./.github/actions/apt-install
with:
packages: devscripts equivs

- name: "Verify API for libkeymancore.so (${{ needs.setup_environment.outputs.GIT_BRANCH }}, branch ${{ needs.setup_environment.outputs.GIT_BASE_BRANCH }}, by ${{ needs.setup_environment.outputs.GIT_USER }}) - "
run: |
cd linux
./scripts/deb-packaging.sh \
--gha \
--bin-pkg "${GITHUB_WORKSPACE}/artifacts/${PKG_NAME}_${{ needs.setup_environment.outputs.VERSION }}-1${{ needs.setup_environment.outputs.PRERELEASE_TAG }}+$(lsb_release -c -s)1_amd64.deb" \
--git-sha "${{ needs.setup_environment.outputs.GIT_SHA }}" \
--git-base "${{ needs.setup_environment.outputs.GIT_BASE }}" \
verify 2>> $GITHUB_STEP_SUMMARY

- name: Archive .symbols file
uses: actions/upload-artifact@26f96dfa697d77e81fd5907df203aa23a56210a8 # v4.3.0
with:
name: libkeymancore.symbols
path: linux/debian/tmp/DEBIAN/symbols
if: always()

set_status:
name: Set result status on PR builds
needs: [setup_environment, api_verification]
runs-on: ubuntu-latest
if: ${{ always() && needs.setup_environment.outputs.IS_TEST_BUILD == 'true' }}
steps:
- name: Set success
if: needs.api_verification.result == 'success'
run: |
echo "RESULT=success" >> $GITHUB_ENV
echo "MSG=Package build succeeded" >> $GITHUB_ENV

- name: Set cancelled
if: needs.api_verification.result == 'cancelled'
run: |
echo "RESULT=error" >> $GITHUB_ENV
echo "MSG=Package build cancelled" >> $GITHUB_ENV

- name: Set failure
if: needs.api_verification.result == 'failure'
run: |
echo "RESULT=failure" >> $GITHUB_ENV
echo "MSG=Package build failed" >> $GITHUB_ENV

- name: Set final status
run: |
gh api \
--method POST \
-H "Accept: application/vnd.github+json" \
/repos/$GITHUB_REPOSITORY/statuses/${{ needs.setup_environment.outputs.GIT_SHA }} \
-f state="$RESULT" \
-f target_url="$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID" \
-f description="$MSG" \
-f context="$STATUS_CONTEXT"
53 changes: 20 additions & 33 deletions .github/workflows/deb-packaging.yml
Original file line number Diff line number Diff line change
Expand Up @@ -291,70 +291,57 @@ jobs:
done
echo "::endgroup::"

api_verification:
prepare_api_verification:
name: Verify API for libkeymancore.so
needs: [sourcepackage, binary_packages_released]
runs-on: ubuntu-latest

steps:
- name: Checkout
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 #v4.1.1
with:
ref: '${{ github.event.client_payload.buildSha }}'
fetch-depth: 0

- name: Download Artifacts
uses: actions/download-artifact@6b208ae046db98c579e8a3aa621ab581ff575935 # v4.1.1
with:
path: artifacts
merge-multiple: true

- name: Install devscripts
uses: ./.github/actions/apt-install
with:
packages: devscripts equivs

- name: Verify API
- name: Save environment
run: |
cd linux
# Extract line containing "Package: libkeymancore1" and then strip "Package: "
PKG_NAME=$(grep -E 'Package: libkeymancore([0-9]+|$)' debian/control)
PKG_NAME="${PKG_NAME#Package: }"
./scripts/deb-packaging.sh \
--gha \
--bin-pkg "${GITHUB_WORKSPACE}/artifacts/${PKG_NAME}_${{ needs.sourcepackage.outputs.VERSION }}-1${{ needs.sourcepackage.outputs.PRERELEASE_TAG }}+jammy1_amd64.deb" \
--git-sha "${{ github.event.client_payload.buildSha }}" \
--git-base "${{ github.event.client_payload.baseRef }}" \
verify 2>> $GITHUB_STEP_SUMMARY

- name: Archive .symbols file
uses: actions/upload-artifact@26f96dfa697d77e81fd5907df203aa23a56210a8 # v4.3.0
echo "VERSION=${{ needs.sourcepackage.outputs.VERSION }}" > artifacts/env
echo "PRERELEASE_TAG=${{ needs.sourcepackage.outputs.PRERELEASE_TAG }}" >> artifacts/env
echo "GIT_SHA=${{ github.event.client_payload.buildSha }}" >> artifacts/env
echo "GIT_BASE=${{ github.event.client_payload.baseRef }}" >> artifacts/env
echo "IS_TEST_BUILD=${{ github.event.client_payload.isTestBuild }}" >> artifacts/env
echo "GIT_BRANCH=${{ github.event.client_payload.branch }}" >> artifacts/env
echo "GIT_BASE_BRANCH=${{ github.event.client_payload.baseBranch }}" >> artifacts/env
echo "GIT_USER=${{ github.event.client_payload.user }}" >> artifacts/env

- name: Cache artifacts
uses: actions/cache/save@13aacd865c20de90d75de3b17ebe84f7a17d57d2 # v4.0.0
with:
name: libkeymancore.symbols
path: linux/debian/tmp/DEBIAN/symbols
if: always()
path: |
artifacts
key: artifacts-key-${GITHUB_RUN_ID}

# We intentionally ignore the results of binary_packages_unreleased
set_status:
name: Set result status on PR builds
needs: [sourcepackage, binary_packages_released, api_verification, autopkg_test]
needs: [sourcepackage, binary_packages_released, prepare_api_verification, autopkg_test]
runs-on: ubuntu-latest
if: ${{ always() && github.event.client_payload.isTestBuild == 'true' }}
steps:
- name: Set success
if: needs.sourcepackage.result == 'success' && needs.binary_packages_released.result == 'success' && needs.api_verification.result == 'success' && needs.autopkg_test.result == 'success'
if: needs.sourcepackage.result == 'success' && needs.binary_packages_released.result == 'success' && needs.prepare_api_verification.result == 'success' && needs.autopkg_test.result == 'success'
run: |
echo "RESULT=success" >> $GITHUB_ENV
echo "MSG=Package build succeeded" >> $GITHUB_ENV

- name: Set cancelled
if: needs.sourcepackage.result == 'cancelled' || needs.binary_packages_released.result == 'cancelled' || needs.api_verification.result == 'cancelled' || needs.autopkg_test.result == 'cancelled'
if: needs.sourcepackage.result == 'cancelled' || needs.binary_packages_released.result == 'cancelled' || needs.prepare_api_verification.result == 'cancelled' || needs.autopkg_test.result == 'cancelled'
run: |
echo "RESULT=error" >> $GITHUB_ENV
echo "MSG=Package build cancelled" >> $GITHUB_ENV

- name: Set failure
if: needs.sourcepackage.result == 'failure' || needs.binary_packages_released.result == 'failure' || needs.api_verification.result == 'failure' || needs.autopkg_test.result == 'failure'
if: needs.sourcepackage.result == 'failure' || needs.binary_packages_released.result == 'failure' || needs.prepare_api_verification.result == 'failure' || needs.autopkg_test.result == 'failure'
run: |
echo "RESULT=failure" >> $GITHUB_ENV
echo "MSG=Package build failed" >> $GITHUB_ENV
Expand Down
12 changes: 6 additions & 6 deletions docs/core-api-verification.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Keyman Core API verification

During GHA Debian package build we verify that the API didn't change
After the GHA Debian package build we verify that the API didn't change
without being documented in `linux/debian/libkeymancore.symbols`.

[Debian policy](https://www.debian.org/doc/debian-policy/ch-sharedlibs#run-time-shared-libraries)
Expand All @@ -21,7 +21,7 @@ have to begin with a space character.

Example:

```
```text
km_core_state_actions_get@Base 17.0.197
```

Expand Down Expand Up @@ -51,13 +51,13 @@ instantiations.

To work around this, we list the C++ symbols as `optional` that get flaged:

```
```text
(c++|optional)"typeinfo name for std::codecvt_utf8_utf16<char16_t, 1114111ul, (std::codecvt_mode)0>@Base" 17.0.244
```

`dpkg-gensymbols` will report the mangled C++ name in the diff, e.g.

```
```text
+ _ZTSSt18codecvt_utf8_utf16IDsLm1114111ELSt12codecvt_mode0EE@Base 17.0.244
```

Expand All @@ -70,7 +70,7 @@ echo "_ZTSSt18codecvt_utf8_utf16IDsLm1114111ELSt12codecvt_mode0EE@Base" | c++fil

which will output the demangled name:

```
```text
typeinfo name for std::codecvt_utf8_utf16<char16_t, 1114111ul, (std::codecvt_mode)0>@Base
```

Expand Down Expand Up @@ -101,7 +101,7 @@ match the API verification will fail.

## How this works

`.github/workflows/deb-packaging.yml` contains a `Verify API` step
`.github/workflows/api-verification.yml` contains a `Verify API` step
which will call `linux/scripts/deb-packaging.sh` with the `verify`
parameter. Before calling the script it will download and extract
the binary package artifacts.
Expand Down
9 changes: 8 additions & 1 deletion linux/scripts/deb-packaging.sh
Original file line number Diff line number Diff line change
Expand Up @@ -186,6 +186,14 @@ get_api_version_from_core() {
echo "${api_version}"
}

# Check if the API version got updated
# Returns:
# 0 - if the API version got updated
# 1 - the .symbols file got changed but the API version didn't get updated
# 2 - if we're in the alpha tier and the API version got updated since
# the last stable version
# NOTE: it is up to the caller to check if this is a major version
# change that requires an API version update.
# Check if the API version got updated
# Returns:
# 0 - if the API version got updated
Expand All @@ -209,7 +217,6 @@ is_api_version_updated() {
case ${TIER} in
alpha)
local STABLE_VERSION STABLE_API_VERSION STABLE_BRANCH
STABLE_API_VERSION=0
STABLE_VERSION=$((${VERSION%%.*} - 1))
STABLE_BRANCH="origin/stable-${STABLE_VERSION}.0"
STABLE_API_VERSION=$(get_api_version_in_symbols_file "${STABLE_BRANCH}")
Expand Down
2 changes: 1 addition & 1 deletion linux/scripts/test/deb-packaging.tests.sh
Original file line number Diff line number Diff line change
Expand Up @@ -252,7 +252,7 @@ test_check_updated_version_number__LineRemoved_InAlpha_FileMissingInStable_ApiVe
# simulate a commit that renamed the .symbols file
git mv linux/debian/libkeymancore1.symbols linux/debian/libfoo1.symbols
sed -i 's/libkeymancore/libfoo/' linux/scripts/deb-packaging.sh
git add linux/scripts/deb-packaging.sh
git add linux/scripts/deb-packaging.sh
sed -i 's/libkeymancore/libfoo/' linux/debian/libfoo1.symbols
git add linux/debian/libfoo1.symbols
git commit -m "renamed library"
Expand Down
Loading