Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace please with Makefile #53

Merged
merged 84 commits into from
Mar 11, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
84 commits
Select commit Hold shift + click to select a range
682d6e8
Add DEP 001 'Replace please with Makefile' (Part of #62)
ptzianos Feb 21, 2024
a32dce9
Tidy up go.mod and vendor dependencies (Part of #62)
ptzianos Jan 26, 2024
6fb03e3
Delete third_party/{binary|docker|go|k8s|plugins|proto} (Part of #62)
ptzianos Jan 26, 2024
ee4d344
Remove plzconfig files (Part of #62)
ptzianos Jan 26, 2024
c87bd3e
Remove pleasew script (Part of #62)
ptzianos Jan 26, 2024
13e5395
Cleanup policies from please BUILD files (Part of #62)
ptzianos Jan 28, 2024
32069d0
Remove leftover please build defs (Part of #62)
ptzianos Jan 28, 2024
c46fcea
Remove all please BUILD files from pkg directory (Part of #62)
ptzianos Jan 26, 2024
6482659
Remove all please BUILD files from components directory (Part of #62)
ptzianos Jan 26, 2024
84da96c
Remove all please BUILD files from examples directory (Part of #62)
ptzianos Feb 2, 2024
012fbde
Add Helm Charts for all example pipelines (Part of #62)
ptzianos Feb 2, 2024
9bf3225
Fix kustomizations for all example pipelines (Part of #62)
ptzianos Feb 2, 2024
21a42f5
Fix parameter names of git source Task (Fixes #70)
ptzianos Jan 31, 2024
03004a9
Fix parameter names of producer aggregator task (Fixes #69)
ptzianos Jan 31, 2024
51ebe26
Fix parameter names of enricher aggregator Task (Fixes #68)
ptzianos Jan 31, 2024
428dc12
Cleanup kustomize-component-generator (Part of #62)
ptzianos Jan 28, 2024
0ccab18
Improve error messages generated by kustomize-component-generator (Pa…
ptzianos Jan 31, 2024
d6dd43d
Set custom container registry and tag for component images (Fixes #66…
ptzianos Feb 2, 2024
84e41e9
Cleanup third_party directory shell scripts and Tekton manifests (Par…
ptzianos Jan 28, 2024
ab0c3a5
Add Bash util script (Part of #62)
ptzianos Feb 2, 2024
c5f19f0
Add scripts for generating TektonCD Helm charts (Part of #62)
ptzianos Feb 2, 2024
1cbaa4e
Add make targets for building component kustomizations (Part of #62)
ptzianos Jan 28, 2024
5615d7a
Add make targets for deploying dev environment (Part of #62)
ptzianos Jan 30, 2024
49a2784
Add make targets to fetch and generate Open API schema (Part of #62)
ptzianos Jan 28, 2024
43ddd43
Add make targets for Go protos (Part of #62)
ptzianos Feb 2, 2024
303bd76
Add make targets for cleaning up repo (Part of #62)
ptzianos Jan 29, 2024
0032570
Add make target for generating example Helm templates (Part of #62)
ptzianos Feb 2, 2024
1fca7d2
Allow user to define custom base images for components (Fixes #66)
ptzianos Jan 31, 2024
48dac25
Add K8s dev deployment manifests and cleanup old build/k8s (Part of #62)
ptzianos Feb 2, 2024
64d6918
Refresh all component kustomization.yamls (Part of #62)
ptzianos Jan 28, 2024
2381f4e
Generate all example pipeline runs (Part of #62)
ptzianos Feb 2, 2024
ce839d4
Refactor lint Github action (Fixes #63)
ptzianos Jan 28, 2024
bcfe95f
Refactor test Github action (Fixes #73)
ptzianos Jan 28, 2024
6971cd9
Refactor format Github action (Fixes #65)
ptzianos Jan 28, 2024
52e118f
Refactor publish Github action (Part of #74)
ptzianos Jan 29, 2024
f3b34bc
Improve code quality for pkg/... (Part of #62)
ptzianos Jan 28, 2024
1c12d7e
Improve code quality for components/... (Part of #62)
ptzianos Jan 28, 2024
df4797c
Fix lint errors of ensure_hashes command (Part of #62)
ptzianos Jan 28, 2024
5add5b9
Add script for setting up KiND cluster (Part of #62)
ptzianos Jan 29, 2024
4fbd854
Update docs to reflect new workflows (Part of #62)
ptzianos Jan 28, 2024
8411dba
delete tag release script (Part of #62)
ptzianos Jan 31, 2024
ce74454
Remove ./build/releases/BUILD (Part of #62)
ptzianos Feb 5, 2024
8d46225
Homogenise Task command paths (Part of #62)
ptzianos Feb 5, 2024
a2864e5
Refactor kustomize component generator to not use SMP (Part of #91)
ptzianos Feb 6, 2024
98c7c5c
Add DEP 002 'Move away from kustomize-component-generator' (Part of #91)
ptzianos Feb 21, 2024
64e9134
Replace kustomize component generator with draconctl (Part of #91)
ptzianos Feb 10, 2024
cbfec97
Disregard protobuf generated files from lint tests (Fixes #63)
ptzianos Feb 15, 2024
1c1fea4
Cleanup 'kustomize-component-generator' related tests (Part of #91)
ptzianos Feb 16, 2024
034e152
Add DEP 003 for migrations (Part of #90)
Mar 11, 2024
86e7bfd
Add draconctl migration subcommands (Part of #90)
ptzianos Feb 21, 2024
7d076cb
Add draconctl migration e2e tests (Part of #90)
ptzianos Feb 21, 2024
41f5b3a
Fix failing down migrations for base enricher (Fixes #96)
ptzianos Feb 21, 2024
238bf4c
Add test coverage to tests (Part of #97)
ptzianos Feb 21, 2024
d959936
Ignore folder where test coverage report will be placed (Part of #97)
ptzianos Feb 21, 2024
db5ca4c
Fix script that sets up kind with a local container registry and upda…
ptzianos Feb 28, 2024
6a2b751
Add container for draconctl and document how to run migrations (Part …
ptzianos Feb 28, 2024
0560fb5
Build static go binaries (Fixes #95)
ptzianos Feb 29, 2024
d29d377
Refactor reviewdog GH action configuration (Fixes #63)
ptzianos Mar 1, 2024
efddf01
Add Chart file to cdxgen example pipeline (Fixes #101)
Mar 2, 2024
dadef42
Temporarily turn off go/staticcheck (Part of #63)
Mar 2, 2024
b4e5c3e
Fix workspace of git clone task (Fixes #102)
Mar 5, 2024
a9053d2
Fix parameters of golang project pipelinerun (Fixes #103)
Mar 5, 2024
b3a7241
Remove producer-aggregator-anchors parameter from task (Fixes #104)
Mar 6, 2024
eb5755b
Update git clone task to version 0.9 (Fixes #106)
Mar 6, 2024
c34ef26
Change getting started doc formatting so that it's easier to copy pas…
Mar 6, 2024
0dd5218
Add Makefile target to print out variable values (Part of #62)
Mar 6, 2024
abe18b4
Improve getting started docs (Part of #62)
Mar 6, 2024
2a5fc57
Fix executable path of deduplication enricher (Part of #62)
Mar 6, 2024
a5d9994
Set predictable values for MongoDB root user/password (Part of #62)
Mar 6, 2024
b61c860
Fix default ES host (Part of #62)
Mar 6, 2024
9f89b83
Fix mongodb service host in consumer (Part of #62)
Mar 6, 2024
7c36f8b
Change workspace from source-code-ws to output (Fixes #109)
Mar 6, 2024
82e3e79
Remove anchor from producer aggregator task (Part of #62)
Mar 8, 2024
58dc748
Make the Nancy task image configurable (Fixes #107)
Mar 8, 2024
145451f
producer/nancy: Mark source code repo as safe (Fixes #108)
Mar 8, 2024
3dfd21e
Regenerate all example pipeline Helm manifests (Part #91)
ptzianos Feb 16, 2024
48db2ff
Reconfigure all pipelineruns to match the updated pipeline manifests …
Mar 6, 2024
93b691c
consumer/stdout-json: Print scan tags (Fixes #112)
Mar 8, 2024
8ee0e85
Clarify getting started docs (Part of #98)
Mar 11, 2024
7e54907
Simplify loading of Tekton manifests (Part of #91)
Mar 11, 2024
25b9e4e
Output migrations inspect as JSON (Part of #90)
Mar 11, 2024
1bf24e4
Don't fail migrations because issues index already exists (Fixes #118)
Mar 11, 2024
04b02fc
Split migrations tests from Go tests for more clarity (Part of #)
Mar 11, 2024
338e01b
Fix flaky Test_FromDraconEnrichedIssuesRun (Fixes #105)
Mar 11, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
The diff you're trying to view is too large. We only load the first 3000 changed files.
16 changes: 12 additions & 4 deletions .github/workflows/format.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,17 @@ jobs:
build:
name: Format
runs-on: ubuntu-latest
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
steps:
- name: Check out code
uses: actions/checkout@v2
- name: Check out code
uses: actions/checkout@v4

- name: Format
run: ./pleasew fmt-all
- name: Setup Go 1.21.6
uses: actions/setup-go@v4
with:
go-version: 1.21.6

- name: Format
run: make install-lint-tools fmt && test $(git diff -p | wc -l) -eq 0
36 changes: 19 additions & 17 deletions .github/workflows/lint.yml
Original file line number Diff line number Diff line change
@@ -1,41 +1,43 @@
---
name: Lint
on:
pull_request:
pull_request_target:
types:
- opened
- synchronize
- reopened
branches:
- main

env:
PLZ_CONFIG_PROFILE: ci
SEGMENT_DOWNLOAD_TIMEOUT_MINS: "2"

permissions:
# Grant the ability to checkout the repository
contents: read

# Allow reviewdog to comment on PRs
# Grant the ability to write comments on a pull request
pull-requests: write

jobs:
build:
lint:
name: Lint
runs-on: ubuntu-latest
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
steps:
- name: Check out code
uses: actions/checkout@v3
uses: actions/checkout@v4

- name: Cache plz
id: cache-plz
uses: actions/cache@v3
- name: Setup Go 1.21.6
uses: actions/setup-go@v4
with:
path: .plz-cache
key: ${{ runner.os }}-plz
go-version: 1.21.6

- name: Setup reviewdog
run: go install github.com/reviewdog/reviewdog/cmd/reviewdog@latest

- name: Install linters
run: make install-lint-tools

- name: Lint
env:
REVIEWDOG_GITHUB_API_TOKEN: "${{ github.token }}"
GOLANGCI_LINT_CACHE: "${{ github.workspace }}/.golangci-lint-cache"
run: "./pleasew -p -v=2 lint"
REVIEWDOG_GITHUB_API_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
run: make lint
28 changes: 2 additions & 26 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,39 +5,15 @@ on:
- 'v*'
branches:
- main
# Trigger, but do nothing in PRs so we can check that the workflow is valid.
pull_request:
branches:
- main

env:
PLZ_CONFIG_PROFILE: ci

permissions:
# Grant the ability to checkout the repository
contents: read

jobs:
pre-release:
name: Publish
if: github.event_name != 'pull_request' && github.ref_type == 'tag'
runs-on: ubuntu-latest
permissions:
# Grant the ability to create GitHub Releases.
contents: write
steps:
- name: Check out code
uses: actions/checkout@v2

- name: Create pre-release
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: ./pleasew run //build/releases:pre-release

push-docker-images:
publish-docker-images:
name: Push Docker images
runs-on: ubuntu-latest
if: github.event_name != 'pull_request'
permissions:
# Grant the ability to write to GitHub Packages (push Docker images to
# GitHub Container Registry).
Expand All @@ -56,4 +32,4 @@ jobs:
password: ${{ secrets.GITHUB_TOKEN }}

- name: Publish Docker images
run: ./pleasew run //build/releases:publish-images
run: make publish-component-containers
52 changes: 18 additions & 34 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,42 +8,11 @@ on:
branches:
- main

env:
PLZ_CONFIG_PROFILE: ci
SEGMENT_DOWNLOAD_TIMEOUT_MINS: "2"

permissions:
# Grant the ability to checkout the repository
contents: read

jobs:
build:
name: Build
runs-on: ubuntu-latest
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}-build
cancel-in-progress: true
steps:
- name: Check out code
uses: actions/checkout@v3

- name: Cache plz
id: cache-plz
uses: actions/cache@v3
with:
path: .plz-cache
key: ${{ runner.os }}-plz

- name: Build
env:
# This uses a public read-only PAT tied to VJftw, please don't abuse! :pray:
# See: https://github.com/actions/setup-node/issues/49 for reasons.
GITHUB_TOKEN: ${{ secrets.VJFTW_GITHUB_TOKEN }}
CONSUMER_JIRA_API_TOKEN: ${{secrets.CONSUMER_JIRA_API_TOKEN}}
CONSUMER_JIRA_USER: ${{secrets.CONSUMER_JIRA_USER}}
CONSUMER_JIRA_URL: ${{secrets.CONSUMER_JIRA_URL}}
run: ./pleasew build -p -v 2 //...

test:
name: Test
runs-on: ubuntu-latest
Expand All @@ -52,7 +21,22 @@ jobs:
cancel-in-progress: true
steps:
- name: Check out code
uses: actions/checkout@v3
uses: actions/checkout@v4

- name: Test
run: ./pleasew test //...
- name: Setup Go 1.21.6
uses: actions/setup-go@v4
with:
go-version: 1.21.6

- name: Run Go tests
run: make go-tests

- name: Run migration tests
run: make migration-tests

- name: Coveralls
uses: coverallsapp/github-action@v2
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
format: golang
file: tests/output/cover.out
9 changes: 3 additions & 6 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,10 +1,7 @@
/dist/
/vendor/
__pycache__
*.pyc
**/node_modules/
.plz_clean*

# Entries below this point are managed by Please (DO NOT EDIT)
plz-out
.plzconfig.local
bin
.custom_image
tests/output
65 changes: 0 additions & 65 deletions .plzconfig

This file was deleted.

3 changes: 0 additions & 3 deletions .plzconfig.ci

This file was deleted.

57 changes: 30 additions & 27 deletions .reviewdog.yml
Original file line number Diff line number Diff line change
@@ -1,42 +1,45 @@
---
runner:
go/vet:
cmd: go vet $(go list ./... | grep -v /vendor/ | grep -v github.com/ocurity/dracon/api/proto/)
format: govet

go/staticcheck:
cmd: |
./pleasew run //third_party/binary/dominikh/go-tools:staticcheck -- $(./pleasew query alltargets --include go | grep -v "//third_party/" | cut -f3- -d/ | cut -f1 -d: | awk '{ print "./" $0 }' | xargs)
errorformat:
- '%E%f:%l:%c: %m'
cmd: staticcheck $(go list ./... | grep -v /vendor/ | grep -v github.com/ocurity/dracon/api/proto)
format: staticcheck

go/revive:
cmd: |
./pleasew run //third_party/binary/mgechev/revive:revive -- $(./pleasew query alltargets --include go | grep -v "//third_party/" | cut -f3- -d/ | cut -f1 -d: | awk '{ print "./" $0 }' | xargs)
cmd: revive -exclude ./vendor/... -exclude ./api/proto/... -config revive.toml ./...
errorformat:
- '%E%f:%l:%c: %m'

go/vet:
cmd: |
./pleasew run "//third_party/go:toolchain|go" -- vet $(./pleasew query alltargets --include go | grep -v "//third_party/" | cut -f3- -d/ | cut -f1 -d: | awk '{ print "./" $0 }' | xargs)
- '%f:%l:%c: %m'

go/containedctx:
cmd: go vet -vettool=$(which containedctx) $(go list ./... | grep -v /vendor/ | grep -v github.com/ocurity/dracon/api/proto)
errorformat:
- '%E%f:%l:%c: %m'
- '%f:%l:%c: %m'

go/gosec:
cmd: |
./pleasew run //third_party/binary/securego/gosec:gosec -- -fmt golint $(./pleasew query alltargets --include go | grep -v "//third_party/" | cut -f3- -d/ | cut -f1 -d: | awk '{ print "./" $0 }' | xargs)
go/ineffassign:
cmd: ineffassign $(go list ./... | grep -v /vendor/ | grep -v github.com/ocurity/dracon/api/proto)
errorformat:
- '%E%f:%l:%c: %m'
- '%f:%l:%c: %m'

go/fmt:
cmd: |
./pleasew run //third_party/go/mvdan.cc/gofumpt:gofumpt -- -d $(./pleasew query alltargets --include go | grep -v "//third_party/" | cut -f3- -d/ | cut -f1 -d: | awk '{ print "./" $0 }' | xargs)
format: diff
level: error
go/errorlint:
cmd: go-errorlint -errorf-multi -errorf -test $(go list ./... | grep -v /vendor/ | grep -v github.com/ocurity/dracon/api/proto)
errorformat:
- '%f:%l:%c: %m'

buf_lint:
cmd: ./pleasew run "//third_party/binary/bufbuild/buf:buf|buf" -- lint --path api/
level: error
go/errcheck:
cmd: errcheck -asserts -blank $(go list ./... | grep -v /vendor/ | grep -v github.com/ocurity/dracon/api/proto)
errorformat:
- '%f:%l:%c:%m'

buf_format:
cmd: ./pleasew run "//third_party/binary/bufbuild/buf:buf|buf" -- format --diff --path api/
format: diff
# TODO: figure out how to set errorformat to parse the error output
# actionlint:
# cmd: actionlint
# errorformat:
# -

buf_lint:
cmd: buf lint . --path ./api/
level: error
format: buf
Loading
Loading