first cut at API checking

It turns out there is a consumer of cabal-install-solver, so I have
added it to API generation and checking.

.github/workflows/check-api.skip.yml

@@ -0,0 +1,49 @@
+
+name: Check API Skip
+
+# This Workflow is special and contains a workaround for a known limitation of GitHub CI.
+#
+# The problem: We don't want to run the "check-api" jobs on PRs which contain only changes
+# to the docs, since these jobs take a long time to complete without providing any benefit.
+# We therefore use path-filtering in the workflow triggers for the bootstrap jobs, namely
+# "paths-ignore: doc/**". But the "Check API post job" is a required job, therefore a PR cannot
+# be merged unless the "Check API post job" completes succesfully, which it doesn't do if we
+# filter it out.
+#
+# The solution: We use a second job with the same name which always returns the exit code 0.
+# The logic implemented for "required" workflows accepts if 1) at least one job with that name
+# runs through, AND 2) If multiple jobs of that name exist, then all jobs of that name have to
+# finish successfully.
+on:
+  push:
+    paths:
+      - 'doc/**'
+      - '**/README.md'
+      - 'CONTRIBUTING.md'
+      - "changelog.d/**"
+      # only top level for these, because various test packages have them too
+      - "*/ChangeLog.md"
+      - "*/changelog.md"
+      - "release-notes/**"
+    branches:
+      - master
+  pull_request:
+    paths:
+      - 'doc/**'
+      - '**/README.md'
+      - 'CONTRIBUTING.md'
+      - "changelog.d/**"
+      - "*/ChangeLog.md"
+      - "*/changelog.md"
+      - "release-notes/**"
+  release:
+    types:
+      - created
+
+jobs:
+  check-api-post-job:
+    if: always()
+    name: Check API post job
+    runs-on: ubuntu-latest
+    steps:
+      - run: exit 0

.github/workflows/check-api.yml

@@ -0,0 +1,136 @@
+name: Check API
+
+on:
+  push:
+    paths-ignore:
+      - "doc/**"
+      - "**/README.md"
+      - "CONTRIBUTING.md"
+      - "changelog.d/**"
+      # only top level for these, because various test packages have them too
+      - "*/ChangeLog.md"
+      - "*/changelog.md"
+      - "release-notes/**"
+    branches:
+      - master
+  pull_request:
+    paths-ignore:
+      - "doc/**"
+      - "**/README.md"
+      - "CONTRIBUTING.md"
+      - "changelog.d/**"
+      - "*/ChangeLog.md"
+      - "*/changelog.md"
+      - "release-notes/**"
+  release:
+    types:
+      - created
+  workflow_call:
+
+jobs:
+  check-api:
+    name: Check API using ${{ matrix.sys.os }} ghc-${{ matrix.ghc }}
+    runs-on: ${{ matrix.sys.os }}
+    strategy:
+      matrix:
+        # we check API only on one platform and ghc release, since it shouldn't
+        # vary elsewhere and the API tracer is sensitive to both
+        sys:
+          - { os: ubuntu-latest }
+        ghc:
+          [
+            # print-api only supports a small subset of ghc versions
+            "9.10.1",
+          ]
+
+    steps:
+
+      - uses: actions/checkout@v4
+
+      - uses: haskell-actions/setup@v2
+        id: setup-haskell
+        with:
+          ghc-version: ${{ matrix.ghc }}
+          cabal-version: 3.12.1.0 # see https://github.com/haskell/cabal/pull/10251
+          ghcup-release-channel: https://raw.githubusercontent.com/haskell/ghcup-metadata/master/ghcup-prereleases-0.0.8.yaml
+
+      # I was going to use the canned action, but it only supports a single package and reinstalls the same binary each time
+      - name: Install print-api
+        run: |
+          wget -q https://github.com/Kleidukos/print-api/releases/download/v0.1.0.1/print-api-0.1.0.1-Linux-static-${{ matrix.ghc }}-x86_64.tar.gz
+          tar -xzf print-api-0.1.0.1-Linux-static-${{ matrix.ghc }}-x86_64.tar.gz
+          mkdir -p "$HOME/.local/bin"
+          mv print-api "$HOME/.local/bin/print-api"
+          chmod +x "$HOME/.local/bin/print-api"
+          echo "$HOME/.local/bin" >> $GITHUB_PATH
+
+      # print-api needs environment files. It also doesn't make a lot of sense to use the cached builds, sadly,
+      # since they're special in different ways (bootstrap and validate) and we want a vanilla build. And there
+      # isn't enough cache space to make a third cache, even though this is a very limited build.
+      - name: Build Cabal with environment files
+        run: cabal build Cabal-syntax Cabal cabal-install-solver --write-ghc-environment-files=always
+
+      - name: Generate Cabal-syntax, Cabal, and cabal-install-solver APIs
+        run: make generate-api
+
+      # upload the new API records as artifacts, since there's no guarantee that a contributor could produce
+      # them (wrong platform or ghc version). This must happen _before_ we check the API, because the
+      # point is to have them available on API mismatch so they can be updated.
+      - uses: actions/upload-artifact@v4
+        with:
+          name: Cabal-api
+          path: '*.api'
+
+      - name: Check Cabal-syntax, Cabal, and cabal-install-solver APIs
+        run: |
+          rc=0
+          if diff -c Cabal-syntax/Cabal-syntax-${{ matrix.ghc }}.api Cabal-syntax-${{ matrix.ghc }}.api >api.tmp; then
+            :
+          else
+            echo "Cabal-syntax API changed"
+            if [ $(wc -l < api.tmp) -lt 50 ]; then
+              cat api.tmp
+            else
+              echo Diff too large for GitHub viewer
+            fi
+            rc=1
+          fi
+          if diff -c Cabal/Cabal-${{ matrix.ghc }}.api Cabal-${{ matrix.ghc }}.api >api.tmp; then
+            :
+          else
+            echo "Cabal API changed"
+            if [ $(wc -l < api.tmp) -lt 50 ]; then
+              cat api.tmp
+            else
+              echo Diff too large for GitHub viewer
+            fi
+            rc=1
+          fi
+          if diff -c cabal-install-solver/cabal-install-solver-${{ matrix.ghc }}.api cabal-install-solver-${{ matrix.ghc }}.api >api.tmp; then
+            :
+          else
+            echo "cabal-install-solver API changed"
+            if [ $(wc -l < api.tmp) -lt 50 ]; then
+              cat api.tmp
+            else
+              echo Diff too large for GitHub viewer
+            fi
+            rc=1
+          fi
+          if [ $rc -ne 0 ]; then
+            echo "The new APIs are in the artifact uploaded in the previous step."
+            exit $rc
+          fi
+
+  # See check-api.skip.yml for why we need this
+  - check-api-post-job:
+    if: always()
+    name: Check API post job
+    runs-on: ubuntu-latest
+    needs: check-api
+
+    steps:
+      - run: |
+          echo "jobs info: ${{ toJSON(needs) }}"
+      - if: contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled')
+        run: exit 1

.gitignore

@@ -107,3 +107,6 @@ bench.html
 
 # ignore the downloaded binary files
 scripts/release/binary-downloads/
+
+# ignore generated API files
+/*.api

CONTRIBUTING.md

@@ -193,6 +193,44 @@ We use automated whitespace convention checking. Violations can be fixed by
 running [fix-whitespace](https://hackage.haskell.org/package/fix-whitespace). If
 you push a fix of a whitespace violation, please do so in a _separate commit_.
 
+API Changes and Check API job
+-----------------------------
+
+The `Check API` job tests the `Cabal`, `Cabal-syntax`, and `cabal-install-solver`
+packages for API changes. It's useful to indicate when a changelog is needed and
+which PRs aren't appropriate for backports.
+
+If the `Check API` job fails, you will find in its build artifacts (at the bottom
+of the "upload artifacts" step, immediately before the actual API check) a ZIP file
+containing the new API records. You can download this and replace the existing API
+descriptions, which can be found in the package top level directories, with `.api`
+suffixes. Generating them locally is possible with the [check-api tool](https://github.com/Kleidukos/print-api), but
+is not guaranteed to produce the same result as the CI job does.
+
+If you do wish to generate a local API record, install [`print-api`](https://github.com/Kleidukos/print-api/releases/tag/v0.1.0.1) and
+run it on the `Cabal`, `Cabal-syntax`, and `cabal-install-solver` packages, from
+the top level directory of the Cabal repo:
+
+    make generate-api
+
+You will need `ghc-9.10.1` to be on `$PATH`; `ghcup` is the easiest way to do this.
+
+The resulting `Cabal-syntax.api`, `Cabal.api`, and `cabal-install-solver` files
+can then be compared to the ones in the `Cabal-syntax`, `Cabal`, and
+`cabal-install-solver` package directories.
+
+    make check-api
+
+If necessary, you can then install the API records:
+
+    make update-api
+
+It is also possible to do this individually; see the `Makefile`.
+
+Note that different compiler versions and different architectures will alter the
+output. It is not expected that different Linux distributions will, but you may
+need to use the static build if you aren't using Ubuntu 22.04.
+
 Other Conventions
 -----------------