Merge latest changes into release-1.31

.github/ISSUE_TEMPLATE/create_release_branch.md

@@ -13,16 +13,16 @@ Make sure to follow the steps below and ensure all actions are completed and sig
 - **K8s version**: 1.xx
 
 <!-- Set this to the name of the person responsible for running the release tasks, e.g. @neoaggelos -->
-- **Owner**:
+- **Owner**: `who plans to do the work`
 
 <!-- Set this to the name of the team-member that will sign-off the tasks -->
-- **Reviewer**:
+- **Reviewer**:  `who plans to review the work`
 
 <!-- Link to PR to initialize the release branch (see below) -->
-- **PR**:
--
+- **PR**: https://github.com/canonical/k8s-snap/pull/`<int>`
+
 <!-- Link to PR to initialize auto-update job for the release branch (see below) -->
-- **PR**:
+- **PR**: https://github.com/canonical/k8s-snap/pull/`<int>`
 
 #### Actions
 
@@ -53,7 +53,7 @@ The steps are to be followed in-order, each task must be completed by the person
 - [ ] **Owner**: Create `release-1.xx` branch from latest `master` in k8s-dqlite
   - `git clone [email protected]:canonical/k8s-dqlite.git ~/tmp/release-1.xx`
   - `pushd ~/tmp/release-1.xx`
-  - `git switch main`
+  - `git switch master`
   - `git pull`
   - `git checkout -b release-1.xx`
   - `git push origin release-1.xx`
@@ -89,7 +89,7 @@ The steps are to be followed in-order, each task must be completed by the person
 - [ ] **Owner**: Create `release-1.xx` branch from latest `main` in rawfile-localpv
   - `git clone [email protected]:canonical/rawfile-localpv.git ~/tmp/release-1.xx`
   - `pushd ~/tmp/release-1.xx`
-  - `git switch main`
+  - `git switch rockcraft`
   - `git pull`
   - `git checkout -b release-1.xx`
   - `git push origin release-1.xx`
@@ -98,7 +98,6 @@ The steps are to be followed in-order, each task must be completed by the person
 - [ ] **Reviewer**: Ensure `release-1.xx` branch is based on latest changes on `main` at the time of the release cut.
 - [ ] **Owner**: Create PR to initialize `release-1.xx` branch:
   - [ ] Update `KUBERNETES_RELEASE_MARKER` to `stable-1.xx` in [/build-scripts/hack/update-component-versions.py][]
-  - [ ] Update `master` to `release-1.xx` in [/build-scripts/components/k8s-dqlite/version][]
   - [ ] Update `"main"` to `"release-1.xx"` in [/build-scripts/hack/generate-sbom.py][]
   - [ ] `git commit -m 'Release 1.xx'`
   - [ ] Create PR against `release-1.xx` with the changes and request review from **Reviewer**. Make sure to update the issue `Information` section with a link to the PR.
@@ -107,43 +106,22 @@ The steps are to be followed in-order, each task must be completed by the person
   - [ ] Add `release-1.xx` in [.github/workflows/update-components.yaml][]
   - [ ] Remove unsupported releases from the list (if applicable, consult with **Reviewer**)
   - [ ] Create PR against `main` with the changes and request review from **Reviewer**. Make sure to update the issue information with a link to the PR.
-- [ ] **Reviewer**: On merge, confirm [Auto-update strict branch] action runs to completion and that the `autoupdate/release-1.xx-strict` branch is created.
-- [ ] **Owner**: Create launchpad builders for `release-1.xx`
-  - [ ] Go to [lp:k8s][] and do **Import now** to pick up all latest changes.
-  - [ ] Under **Branches**, select `release-1.xx`, then **Create snap package**
-  - [ ] Set **Snap recipe name** to `k8s-snap-1.xx`
-  - [ ] Set **Owner** to `Canonical Kubernetes (containers)`
-  - [ ] Set **The project that this Snap is associated with** to `k8s`
-  - [ ] Set **Series** to Infer from snapcraft.yaml
-  - [ ] Set **Processors** to `AMD x86-64 (amd64)` and `ARM ARMv8 (arm64)`
-  - [ ] Enable **Automatically build when branch changes**
-  - [ ] Enable **Automatically upload to store**
-  - [ ] Set **Registered store name** to `k8s`
-  - [ ] In **Store Channels**, set **Track** to `1.xx-classic` and **Risk** to `edge`. Leave **Branch** empty
-  - [ ] Click **Create snap package** at the bottom of the page.
-- [ ] **Owner**: Create launchpad builders for `release-1.xx-strict`
-  - [ ] Return to [lp:k8s][].
-  - [ ] Under **Branches**, select `autoupdate/release-1.xx-strict`, then **Create snap package**
-  - [ ] Set **Snap recipe name** to `k8s-snap-1.xx-strict`
-  - [ ] Set **Owner** to `Canonical Kubernetes (containers)`
-  - [ ] Set **The project that this Snap is associated with** to `k8s`
-  - [ ] Set **Series** to Infer from snapcraft.yaml
-  - [ ] Set **Processors** to `AMD x86-64 (amd64)` and `ARM ARMv8 (arm64)`
-  - [ ] Enable **Automatically build when branch changes**
-  - [ ] Enable **Automatically upload to store**
-  - [ ] Set **Registered store name** to `k8s`
-  - [ ] In **Store Channels**, set **Track** to `1.xx` and **Risk** to `edge`. Leave **Branch** empty
-  - [ ] Click **Create snap package** at the bottom of the page.
+- [ ] **Reviewer**: On merge, confirm [Auto-update strict branch] action runs to completion and that the `autoupdate/release-1.xx-*` flavor branches are created.
+   - [ ] autoupdate/release-1.xx-strict
+   - [ ] autoupdate/release-1.xx-moonray
+- [ ] **Owner**: Create launchpad builders for `release-1.xx` and flavors
+  - [ ] Run the [Confirm Snap Builds][] Action
 - [ ] **Reviewer**: Ensure snap recipes are created in [lp:k8s/+snaps][]
-  - look for `k8s-snap-1.xx`
-  - look for `k8s-snap-1.xx-strict`
+  - [ ] look for `k8s-snap-1.xx-classic`
+  - [ ] look for `k8s-snap-1.xx-strict`
+  - [ ] look for `k8s-snap-1.xx-moonray`
+  - [ ] make sure each is "Authorized for Store Upload"
 
 #### After release
 
 - [ ] **Owner** follows up with the **Reviewer** and team about things to improve around the process.
 - [ ] **Owner**: After a few weeks of stable CI, update default track to `1.xx/stable` via
   - On the snap [releases page][], select `Track` > `1.xx`
-- [ ] **Reviewer**: Ensure snap recipes are created in [lp:k8s/+snaps][]
 
 
 <!-- LINKS -->
@@ -161,6 +139,7 @@ The steps are to be followed in-order, each task must be completed by the person
 [.github/workflows/update-components.yaml]: ../workflows/update-components.yaml
 [/build-scripts/components/hack/update-component-versions.py]: ../../build-scripts/components/hack/update-component-versions.py
 [/build-scripts/components/k8s-dqlite/version]: ../../build-scripts/components/k8s-dqlite/version
-[/build-scripts/hack/generate-sbom.py]: ../..//build-scripts/hack/generate-sbom.py
+[/build-scripts/hack/generate-sbom.py]: ../../build-scripts/hack/generate-sbom.py
 [lp:k8s]: https://code.launchpad.net/~cdk8s/k8s/+git/k8s-snap
 [lp:k8s/+snaps]: https://launchpad.net/k8s/+snaps
+[Confirm Snap Builds]: https://github.com/canonical/canonical-kubernetes-release-ci/actions/workflows/create-release-branch.yaml

.github/workflows/auto-merge-successful-prs.yaml

@@ -0,0 +1,29 @@
+name: Auto-merge Successful PRs
+
+on:
+  workflow_dispatch:
+  schedule:
+    - cron: "0 */4 * * *"  # Every 4 hours
+
+permissions:
+  contents: read
+
+jobs:
+  merge-successful-prs:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Harden Runner
+        uses: step-security/harden-runner@v2
+        with:
+          egress-policy: audit
+      - name: Checking out repo
+        uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: '3.12'
+      - name: Auto-merge pull requests if all status checks pass
+        env:
+          GH_TOKEN: ${{ secrets.BOT_TOKEN }}
+        run: |
+            build-scripts/hack/auto-merge-successful-pr.py

.github/workflows/automatic-doc-checks.yml

@@ -3,13 +3,15 @@ name: Core Documentation Checks
 on:
   - workflow_dispatch
 
+permissions:
+  contents: read
+
 concurrency:
   group: ${{ github.workflow }}-${{ github.ref }}
   cancel-in-progress: true
 
 jobs:
   documentation-checks:
-    uses: canonical/documentation-workflows/.github/workflows/documentation-checks.yaml@main
-    with:
-      working-directory: 'docs/moonray'
-
+    - uses: canonical/documentation-workflows/.github/workflows/documentation-checks.yaml@main
+      with:
+        working-directory: 'docs/moonray'

.github/workflows/cron-jobs.yaml

@@ -6,7 +6,7 @@
 
 permissions:
   contents: read
- 
+
 jobs:
   TICS:
     permissions:
@@ -27,6 +27,9 @@
         uses: actions/checkout@v4
         with:
           ref: ${{matrix.branch}}
+      - uses: actions/setup-python@v5
+        with:
+          python-version: '3.12'
       - name: Install Go
         uses: actions/setup-go@v5
         with:
@@ -47,22 +50,22 @@
 
           # TICS requires us to have the test results in cobertura xml format under the
           # directory use below
-          make go.unit
+          sudo make go.unit
           go install github.com/boumenot/gocover-cobertura@latest
           gocover-cobertura < coverage.txt > coverage.xml
           mkdir .coverage
           mv ./coverage.xml ./.coverage/
 
           # Install the TICS and staticcheck
-          go install honnef.co/go/tools/cmd/staticcheck@v0.4.7
+          go install honnef.co/go/tools/cmd/staticcheck@v0.5.1
           . <(curl --silent --show-error 'https://canonical.tiobe.com/tiobeweb/TICS/api/public/v1/fapi/installtics/Script?cfg=default&platform=linux&url=https://canonical.tiobe.com/tiobeweb/TICS/')
 
           # We need to have our project built
           # We load the dqlite libs here instead of doing through make because TICS
           # will try to build parts of the project itself
           sudo add-apt-repository -y ppa:dqlite/dev
           sudo apt install dqlite-tools libdqlite-dev -y
-          make clean
+          sudo make clean
           go build -a ./...
 
           TICSQServer -project k8s-snap -tmpdir /tmp/tics -branchdir $HOME/work/k8s-snap/k8s-snap/
@@ -79,6 +82,8 @@
           - { branch: main, channel: latest/edge }
           # Stable branches
           # Add branches to test here
+          - { branch: release-1.30, channel: 1.30-classic/edge }
+          - { branch: release-1.31, channel: 1.31-classic/edge }
 
     steps:
       - name: Harden Runner
@@ -103,6 +108,8 @@
           format: "sarif"
           output: "trivy-k8s-repo-scan--results.sarif"
           severity: "MEDIUM,HIGH,CRITICAL"
+        env:
+          TRIVY_DB_REPOSITORY: "public.ecr.aws/aquasecurity/trivy-db"
       - name: Gather Trivy repo scan results
         run: |
           cp trivy-k8s-repo-scan--results.sarif ./sarifs/
@@ -111,7 +118,10 @@
           snap download k8s --channel ${{ matrix.channel }}
           mv ./k8s*.snap ./k8s.snap
           unsquashfs k8s.snap
-          ./trivy rootfs ./squashfs-root/ --format sarif > sarifs/snap.sarif
+          for var in $(env | grep -o '^TRIVY_[^=]*'); do
+            unset "$var"
+          done
+          ./trivy --db-repository public.ecr.aws/aquasecurity/trivy-db rootfs ./squashfs-root/ --format sarif > sarifs/snap.sarif
       - name: Get HEAD sha
         run: |
           SHA="$(git rev-parse HEAD)"

.github/workflows/docs-spelling-checks.yml

@@ -0,0 +1,32 @@
+name: Documentation Spelling Check
+
+on:
+  workflow_dispatch:
+  # pull_request:
+  #   paths:
+  #     - 'docs/**'
+permissions:
+  contents: read
+
+jobs:
+  spell-check:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - name: Install aspell
+        run: sudo apt-get install aspell aspell-en
+      - id: spell-check
+        name: Spell Check
+        run: make spelling
+        working-directory: docs/canonicalk8s
+        continue-on-error: true
+      # - if: ${{ github.event_name == 'pull_request' && steps.spell-check.outcome == 'failure' }}
+      #   uses: actions/github-script@v6
+      #   with:
+      #     script: |
+      #       github.rest.issues.createComment({
+      #         issue_number: context.issue.number,
+      #         owner: context.repo.owner,
+      #         repo: context.repo.repo,
+      #         body: 'Hi, looks like pyspelling job found some issues, you can check it [here](${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }})'
+      #       })

.github/workflows/go.yaml

@@ -2,6 +2,8 @@
 
 on:
   push:
+    paths-ignore:
+      - 'docs/**'
     branches:
       - main
       - autoupdate/strict
@@ -10,6 +12,8 @@
       - 'autoupdate/release-[0-9]+.[0-9]+-strict'
       - 'autoupdate/sync/**'
   pull_request:
+    paths-ignore:
+      - 'docs/**'
 
 permissions:
   contents: read
@@ -19,6 +23,7 @@
     permissions:
       contents: read  # for actions/checkout to fetch code
       pull-requests: write  # for marocchino/sticky-pull-request-comment to create or update PR comment
+      checks: write # for golangci/golangci-lint-action to checks to allow the action to annotate code in the PR.
     name: Unit Tests & Code Quality
     runs-on: ubuntu-latest
 
@@ -67,6 +72,19 @@
         # root ownership so the tests must be run as root:
         run: sudo make go.unit
 
+      - name: dqlite-for-golangci-lint
+        working-directory: src/k8s
+        run: |
+          sudo add-apt-repository ppa:dqlite/dev
+          sudo apt update
+          sudo apt install dqlite-tools libdqlite-dev
+
+      - name: golangci-lint
+        uses: golangci/golangci-lint-action@v6
+        with:
+          version: v1.61
+          working-directory: src/k8s
+
   test-binary:
     name: Binaries
     runs-on: ubuntu-latest

.github/workflows/integration-informing.yaml

@@ -2,11 +2,15 @@ name: Informing Integration Tests
 
 on:
   push:
+    paths-ignore:
+      - 'docs/**'
     branches:
       - main
       - 'release-[0-9]+.[0-9]+'
       - 'autoupdate/sync/**'
   pull_request:
+    paths-ignore:
+      - 'docs/**'
 
 permissions:
   contents: read
@@ -17,7 +21,7 @@ jobs:
     runs-on: ubuntu-20.04
     strategy:
       matrix:
-        patch: ["strict", "moonray"]
+        patch: ["moonray"]
       fail-fast: false
     steps:
       - name: Harden Runner
@@ -54,16 +58,16 @@ jobs:
     strategy:
       matrix:
         os: ["ubuntu:20.04"]
-        patch: ["strict", "moonray"]
+        patch: ["moonray"]
       fail-fast: false
-    runs-on: ubuntu-20.04
+    runs-on: ["self-hosted", "Linux", "AMD64", "jammy", "large"]
     steps:
       - name: Check out code
         uses: actions/checkout@v4
       - name: Setup Python
         uses: actions/setup-python@v5
         with:
-          python-version: '3.8'
+          python-version: '3.10'
       - name: Install tox
         run: pip install tox
       - name: Install lxd
@@ -72,29 +76,33 @@ jobs:
           sudo lxd init --auto
           sudo usermod --append --groups lxd $USER
           sg lxd -c 'lxc version'
+          sudo iptables -I DOCKER-USER -i lxdbr0 -j ACCEPT
+          sudo iptables -I DOCKER-USER -o lxdbr0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
       - name: Download snap
         uses: actions/download-artifact@v4
         with:
           name: k8s-${{ matrix.patch }}.snap
-          path: build
+          path: ${{ github.workspace }}/build
       - name: Apply ${{ matrix.patch }} patch
         run: |
           ./build-scripts/patches/${{ matrix.patch }}/apply
       - name: Run end to end tests
+        env:
+          TEST_SNAP: ${{ github.workspace }}/build/k8s-${{ matrix.patch }}.snap
+          TEST_SUBSTRATE: lxd
+          TEST_LXD_IMAGE: ${{ matrix.os }}
+          TEST_FLAVOR: ${{ matrix.patch }}
+          TEST_INSPECTION_REPORTS_DIR: ${{ github.workspace }}/inspection-reports
         run: |
-          export TEST_SNAP="$PWD/build/k8s-${{ matrix.patch }}.snap"
-          export TEST_SUBSTRATE=lxd
-          export TEST_LXD_IMAGE=${{ matrix.os }}
-          export TEST_INSPECTION_REPORTS_DIR="$HOME/inspection-reports"
           cd tests/integration && sg lxd -c 'tox -e integration'
       - name: Prepare inspection reports
         if: failure()
         run: |
-          tar -czvf inspection-reports.tar.gz -C $HOME inspection-reports
+          tar -czvf inspection-reports.tar.gz -C ${{ github.workspace }} inspection-reports
           echo "artifact_name=inspection-reports-${{ matrix.os }}-${{ matrix.patch }}" | sed 's/:/-/g' >> $GITHUB_ENV
       - name: Upload inspection report artifact
         if: failure()
         uses: actions/upload-artifact@v4
         with:
           name: ${{ env.artifact_name }}
-          path: inspection-reports.tar.gz
+          path: ${{ github.workspace }}/inspection-reports.tar.gz