Missing SLSA provenance and cosigns, scripts to detect

[scop]

What

I wrote two ugly and naive scripts to check for missing SLSA provenance and cosign configs.

cosigns.sh

#!/bin/sh

# gh api '/search/code?q="cmd:+cosign"+filename:.goreleaser.yml+filename:.goreleaser.yaml+path:/' -H Accept:application/vnd.github+json -H X-GitHub-Api-Version:2022-11-28 --paginate >cosigns.json

# ^this is naturally just goreleaser things, I'm sure some other useful queries for this could be constructed.
# But it's a start, and already finds a bunch of omissions.

for repo in $(jq -r ".items.[].repository.full_name" <cosigns.json | sort -u); do
    test -f "pkgs/$repo/registry.yaml" || continue
    grep -LF cosign: "pkgs/$repo/registry.yaml"
done

slsas.sh

#!/bin/sh

# gh api /search/code?q=slsa-framework/slsa-github-generator/.github/workflows+path:.github/workflows' -H Accept:application/vnd.github+json -H X-GitHub-Api-Version:2022-11-28 --paginate >slsas.json

for repo in $(jq -r ".items.[].repository.full_name" <slsas.json | sort -u); do
    test -f "pkgs/$repo/registry.yaml" || continue
    grep -LF slsa_provenance: "pkgs/$repo/registry.yaml"
done

Output from gh could naturally be piped directly in instead of asking to be stored in separate files per the comments, but I did this for iterating over the script implementations a bit.

It should also be noted that running the gh commands will drain one's GH REST API rate limit due to --paginate and many results.

Perhaps these would be useful enough to be somehow included in aqua-registry or serve as basis for better implementations. See note below for current output on my system.

Why

Better SLSA and cosign coverage.

Note

$ ./slsas.sh
jq: error (at <stdin>:0): Cannot iterate over null (null) # this is because I exceeded my GH REST API rate limit and the JSON ended up having that error in it
pkgs/argoproj/argo-rollouts/registry.yaml
pkgs/fission/fission/registry.yaml
pkgs/google/go-containerregistry/registry.yaml
pkgs/google/mtail/registry.yaml
pkgs/jreleaser/jreleaser/registry.yaml
pkgs/kptdev/kpt/registry.yaml
pkgs/kyverno/kyverno/registry.yaml
pkgs/restic/restic/registry.yaml
pkgs/slsa-framework/slsa-verifier/registry.yaml
pkgs/xeol-io/xeol/registry.yaml

$ ./cosigns.sh
pkgs/abhimanyu003/sttr/registry.yaml
pkgs/a-h/templ/registry.yaml
pkgs/bitnami-labs/sealed-secrets/registry.yaml
pkgs/carvel-dev/imgpkg/registry.yaml
pkgs/carvel-dev/kapp/registry.yaml
pkgs/carvel-dev/kbld/registry.yaml
pkgs/carvel-dev/vendir/registry.yaml
pkgs/carvel-dev/ytt/registry.yaml
pkgs/cert-manager/cmctl/registry.yaml
pkgs/FairwindsOps/gonogo/registry.yaml
pkgs/FairwindsOps/pluto/registry.yaml
pkgs/FairwindsOps/polaris/registry.yaml
pkgs/FairwindsOps/rbac-lookup/registry.yaml
pkgs/fission/fission/registry.yaml
pkgs/fluxcd/flux2/registry.yaml
pkgs/google/yamlfmt/registry.yaml
pkgs/helm/chart-releaser/registry.yaml
pkgs/helm/chart-testing/registry.yaml
pkgs/kubepug/kubepug/registry.yaml
pkgs/kyverno/kyverno/registry.yaml
pkgs/loft-sh/vcluster/registry.yaml
pkgs/orlangure/gocovsh/registry.yaml
pkgs/purpleclay/dns53/registry.yaml
pkgs/securego/gosec/registry.yaml
pkgs/sigstore/gitsign/registry.yaml
pkgs/sigstore/rekor/registry.yaml
pkgs/smallstep/certificates/registry.yaml
pkgs/smallstep/cli/registry.yaml
pkgs/stacklok/frizbee/registry.yaml
pkgs/stackrox/kube-linter/registry.yaml
pkgs/suzuki-shunsuke/ghalint/registry.yaml
pkgs/suzuki-shunsuke/pinact/registry.yaml
pkgs/suzuki-shunsuke/sort-issue-template/registry.yaml
pkgs/suzuki-shunsuke/tfaction-go/registry.yaml
pkgs/terramate-io/terramate/registry.yaml
pkgs/tofuutils/tenv/registry.yaml
pkgs/Trendyol/kink/registry.yaml
pkgs/trufflesecurity/trufflehog/registry.yaml
pkgs/twpayne/chezmoi/registry.yaml

SLSA

• argoproj/argo-rollouts feat(argoproj/argo-rollouts): Re-scaffold #32224
• fission/fission
• google/go-containerregistry fix(google/go-containerregistry): Re-scaffold #32235
• google/mtail fix(google/mtail): Re-scaffold #32236
• jreleaser/jreleaser
  • re-scaffold blocked by Crash on cmdx s jreleaser/jreleaser aqua#3596
  • verify blocked by SLSA provenance verify tag problem with jreleaser/jreleaser aqua#3613
• kptdev/kpt fix(kptdev/kpt): Re-scaffold #32238
• kyverno/kyverno
• restic/restic provenance for containers only
• slsa-framework/slsa-verifier
• xeol-io/xeol fix(xeol-io/xeol): Re-scaffold #32357

Cosign

• abhimanyu003/sttr feat(abhimanyu003/sttr): cosign config #32687
• a-h/templ
• bitnami-labs/sealed-secrets
• carvel-dev/imgpkg
• carvel-dev/kapp
• carvel-dev/kbld
• carvel-dev/vendir
• carvel-dev/ytt
• cert-manager/cmctl
• FairwindsOps/gonogo
• FairwindsOps/pluto
• FairwindsOps/polaris
• FairwindsOps/rbac-lookup
• fission/fission
• fluxcd/flux2
• google/yamlfmt re-scaffold blocked by Crash on gr google/yamlfmt aqua#3597 feat(google/yamlfmt): cosign config #32610
• helm/chart-releaser
• helm/chart-testing
• kubepug/kubepug
• kyverno/kyverno
• loft-sh/vcluster
• orlangure/gocovsh
• purpleclay/dns53
• securego/gosec
• sigstore/gitsign
• sigstore/rekor
• smallstep/certificates
• smallstep/cli
• stacklok/frizbee
• stackrox/kube-linter
• suzuki-shunsuke/ghalint
• suzuki-shunsuke/pinact
• suzuki-shunsuke/sort-issue-template
• suzuki-shunsuke/tfaction-go
• terramate-io/terramate
• tofuutils/tenv feat(tofuutils/tenv): cosign #32284
• Trendyol/kink
• trufflesecurity/trufflehog
• twpayne/chezmoi

[suzuki-shunsuke]

Thank you for your great work!

I'm working on improving the code generation.

• feat(generate-registry): generate slsa_provenance aqua#3560
• Support generate-registry configuration file aqua#3561

The file extension of SLSA Provenance (.intoto.jsonl) is unique, but the extension of Cosign (.sig, .pem) isn't, so I think it's hard to generate cosign setting automatically.
Furthermore, Cosign needs several parameters.

e.g.

aqua-registry/pkgs/suzuki-shunsuke/tfcmt/registry.yaml

Lines 16 to 25 in 43d29a4

	cosign:
	opts:
	- --certificate-identity-regexp
	- "https://github\\.com/suzuki-shunsuke/go-release-workflow/\\.github/workflows/release\\.yaml@.*"
	- --certificate-oidc-issuer
	- "https://token.actions.githubusercontent.com"
	- --signature
	- https://github.com/suzuki-shunsuke/tfcmt/releases/download/{{.Version}}/tfcmt_{{trimV .Version}}_checksums.txt.sig
	- --certificate
	- https://github.com/suzuki-shunsuke/tfcmt/releases/download/{{.Version}}/tfcmt_{{trimV .Version}}_checksums.txt.pem

[suzuki-shunsuke]

• chore: support aqua gr config #32221 (comment)

This is still in beta, but I've added new features to cmdx s:

• cmdx s generates slsa_provenance
• You can filter versions and assets by configuration file

You can scaffold configuration file by aqua gr -init <package name>.

aqua >= v2.45.0-0 v2.45.0-2 is required: aqua upa v2.45.0-2 aquaproj/aqua#3562 (comment)

e.g.

aqua gr -init fission/fission

aqua-generate-registry.yaml is generated.
Please edit version and asset fields.

e.g.

name: fission/fission
version_filter: Version matches "^v?\\d"
all_assets_filter: not ((Asset matches "\\.json$") or (Asset matches "\\.yaml$")) # Ignore JSON and YAML

About the configuration file, please see aquaproj/aqua#3562 (comment) .

Then you can pass the configuration to cmdx s by -c option:

cmdx s -c aqua-generate-registry.yaml <package name>

Note that configuration file is optional.
Basically you don't need to use configuration files.
Configuration files are useful if you want to exclude some versions and assets.

[scop]

Nice! I'll give it a try later.

Regarding cosign, sometimes files related to it could be relatively reliably identified from *keyless.sig and/or *keyless.pem on the checksums. Even though it requires more parameters, those are mostly predictable as well. Maybe also making use of that there are both .sig and .pem (with or without "keyless"), and they are on the checksums file could yield few enough false positives for it to be useful, too.