Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disable MongoDB locally and Improve local golang pipeline example #463

Merged
merged 3 commits into from
Nov 4, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions deploy/smithy/chart/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,11 @@ kibana:
tlsEnabled: false

mongodb:
# if set, a MongoDB instance will be deployed
enabled: true
# if set, a MongoDB instance will be deployed.
# Disabled as there's currently an issue with M1-M3 processors and workarounds are necessary
# https://github.com/bitnami/containers/issues/53407
# It's better to stop leveraging MongoDB locally until the issue is resolved.
enabled: false
# auth:
# enabled: true
# usernames: ["consumer-mongodb"]
Expand Down
5 changes: 4 additions & 1 deletion deploy/smithy/values/dev.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,10 @@ kibana:
tlsEnabled: false

mongodb:
enabled: true
# Disabled as there's currently an issue with M1-M3 processors and workarounds are necessary
# https://github.com/bitnami/containers/issues/53407
# It's better to stop leveraging MongoDB locally until the issue is resolved.
enabled: false
auth:
enabled: true
usernames: ["consumer-mongodb"]
Expand Down
48 changes: 24 additions & 24 deletions deploy/tektoncd/pipeline/templates/customresourcedefinitions.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,25 +11,37 @@ metadata:
helm.sh/chart: '{{ include "tekton.chart" . }}'
pipeline.tekton.dev/release: v0.44.0
version: v0.44.0
name: verificationpolicies.tekton.dev
name: clustertasks.tekton.dev
spec:
conversion:
strategy: Webhook
webhook:
clientConfig:
service:
name: tekton-pipelines-webhook
namespace: tekton-pipelines
conversionReviewVersions:
- v1beta1
group: tekton.dev
names:
categories:
- tekton
- tekton-pipelines
kind: VerificationPolicy
plural: verificationpolicies
singular: verificationpolicy
scope: Namespaced
kind: ClusterTask
plural: clustertasks
singular: clustertask
preserveUnknownFields: false
scope: Cluster
versions:
- name: v1alpha1
- name: v1beta1
schema:
openAPIV3Schema:
type: object
x-kubernetes-preserve-unknown-fields: true
served: true
storage: true
subresources:
status: {}
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
Expand All @@ -43,37 +55,25 @@ metadata:
helm.sh/chart: '{{ include "tekton.chart" . }}'
pipeline.tekton.dev/release: v0.44.0
version: v0.44.0
name: clustertasks.tekton.dev
name: verificationpolicies.tekton.dev
spec:
conversion:
strategy: Webhook
webhook:
clientConfig:
service:
name: tekton-pipelines-webhook
namespace: tekton-pipelines
conversionReviewVersions:
- v1beta1
group: tekton.dev
names:
categories:
- tekton
- tekton-pipelines
kind: ClusterTask
plural: clustertasks
singular: clustertask
preserveUnknownFields: false
scope: Cluster
kind: VerificationPolicy
plural: verificationpolicies
singular: verificationpolicy
scope: Namespaced
versions:
- name: v1beta1
- name: v1alpha1
schema:
openAPIV3Schema:
type: object
x-kubernetes-preserve-unknown-fields: true
served: true
storage: true
subresources:
status: {}
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
Expand Down
17 changes: 8 additions & 9 deletions docs/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,8 +35,7 @@ pipeline which will:
* Scan the repository with [gosec](https://github.com/securego/gosec)
and [nancy](https://github.com/sonatype-nexus-community/nancy)
* Enrich the findings with CODEOWNERS annotation
* Report the enriched results on [MongoDB](https://github.com/mongodb/mongo)
and [ElasticSearch](https://github.com/elastic/elasticsearch)
* Log the enriched results

### Set up Smithy and its dependencies

Expand All @@ -48,10 +47,8 @@ This command will:
TBD - if not KIND
* deploy Smithy dependencies and Custom Resource Definitions (CRDs).
Most of these dependencies are required by the example pipelines:
* MongoDB
* Elasticsearch
* Kibana
* MongoDB
* Postgres

All the dependencies are built using smithy's current [latest release](https://github.com/smithy-security/smithy/tags).
Expand Down Expand Up @@ -106,8 +103,7 @@ And of its tasks by executing:
kubectl get taskruns -w -n smithy
NAME SUCCEEDED REASON STARTTIME COMPLETIONTIME
smithy-golang-project-7hqmc-base True Succeeded 27m 26m
smithy-golang-project-7hqmc-consumer-elasticsearch True Succeeded 23m 23m
smithy-golang-project-7hqmc-consumer-mongodb True Succeeded 23m 17m
smithy-golang-project-7hqmc-consumer-stdout-json-pod True Succeeded 23m 23m
smithy-golang-project-7hqmc-enricher-aggregator True Succeeded 24m 23m
smithy-golang-project-7hqmc-enricher-codeowners True Succeeded 24m 24m
smithy-golang-project-7hqmc-git-clone True Succeeded 27m 25m
Expand All @@ -123,8 +119,7 @@ kubectl get pods -w -n smithy
NAME READY STATUS RESTARTS AGE
smithy-es-default-0 1/1 Running 0 24m
smithy-golang-project-7hqmc-base-pod 0/1 Completed 0 22m
smithy-golang-project-7hqmc-consumer-elasticsearch-pod 0/1 Running 0 19m
smithy-golang-project-7hqmc-consumer-mongodb-pod 0/1 Running 0 19m
smithy-golang-project-7hqmc-consumer-stdout-json-pod 0/1 Completed 0 19m
smithy-golang-project-7hqmc-enricher-aggregator-pod 0/2 Completed 0 19m
smithy-golang-project-7hqmc-enricher-codeowners-pod 0/2 Completed 0 19m
smithy-golang-project-7hqmc-git-clone-pod 0/2 Completed 0 22m
Expand All @@ -135,7 +130,11 @@ smithy-kb-5df6fcb8c7-tsbg6 1/1 Running 0
smithy-postgresql-0 1/1 Running 0 25m
```

You can then check the enriched results stored in MongoDB and Elasticsearch.
You can then check the enriched results on the logs of the json consumer:

```shell
kubectl logs smithy-golang-project-7hqmc-consumer-stdout-json-pod -n smithy
```

### Debugging

Expand Down
1 change: 0 additions & 1 deletion examples/pipelines/dast-project/kustomization.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,5 +8,4 @@ components:
- pkg:helm/smithy-security-oss-components/producer-aggregator
- pkg:helm/smithy-security-oss-components/enricher-deduplication
- pkg:helm/smithy-security-oss-components/enricher-aggregator
- pkg:helm/smithy-security-oss-components/consumer-mongodb
- pkg:helm/smithy-security-oss-components/consumer-elasticsearch
3 changes: 1 addition & 2 deletions examples/pipelines/golang-project/kustomization.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,5 +10,4 @@ components:
- pkg:helm/smithy-security-oss-components/producer-aggregator
- pkg:helm/smithy-security-oss-components/enricher-codeowners
- pkg:helm/smithy-security-oss-components/enricher-aggregator
- pkg:helm/smithy-security-oss-components/consumer-mongodb
- pkg:helm/smithy-security-oss-components/consumer-elasticsearch
- pkg:helm/smithy-security-oss-components/consumer-stdout-json
1 change: 0 additions & 1 deletion examples/pipelines/iac-project/kustomization.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,5 +11,4 @@ components:
- pkg:helm/smithy-security-oss-components/producer-aggregator
- pkg:helm/smithy-security-oss-components/enricher-deduplication
- pkg:helm/smithy-security-oss-components/enricher-aggregator
- pkg:helm/smithy-security-oss-components/consumer-mongodb
- pkg:helm/smithy-security-oss-components/consumer-elasticsearch
1 change: 0 additions & 1 deletion examples/pipelines/java-project/kustomization.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,5 +10,4 @@ components:
- pkg:helm/smithy-security-oss-components/producer-aggregator
- pkg:helm/smithy-security-oss-components/enricher-deduplication
- pkg:helm/smithy-security-oss-components/enricher-aggregator
- pkg:helm/smithy-security-oss-components/consumer-mongodb
- pkg:helm/smithy-security-oss-components/consumer-elasticsearch
1 change: 0 additions & 1 deletion examples/pipelines/misc-project/kustomization.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,5 +9,4 @@ components:
- pkg:helm/smithy-security-oss-components/producer-aggregator
- pkg:helm/smithy-security-oss-components/enricher-deduplication
- pkg:helm/smithy-security-oss-components/enricher-aggregator
- pkg:helm/smithy-security-oss-components/consumer-mongodb
- pkg:helm/smithy-security-oss-components/consumer-elasticsearch
1 change: 0 additions & 1 deletion examples/pipelines/nancy-purl-project/kustomization.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,3 @@ components:
- pkg:helm/smithy-security-oss-components/producer-aggregator
- pkg:helm/smithy-security-oss-components/enricher-deduplication
- pkg:helm/smithy-security-oss-components/enricher-aggregator
- pkg:helm/smithy-security-oss-components/consumer-mongodb
1 change: 0 additions & 1 deletion examples/pipelines/python-project/kustomization.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,5 +10,4 @@ components:
- pkg:helm/smithy-security-oss-components/producer-aggregator
- pkg:helm/smithy-security-oss-components/enricher-codeowners
- pkg:helm/smithy-security-oss-components/enricher-aggregator
- pkg:helm/smithy-security-oss-components/consumer-mongodb
- pkg:helm/smithy-security-oss-components/consumer-elasticsearch
1 change: 0 additions & 1 deletion examples/pipelines/rails-project/kustomization.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,5 +9,4 @@ components:
- pkg:helm/smithy-security-oss-components/producer-aggregator
- pkg:helm/smithy-security-oss-components/enricher-codeowners
- pkg:helm/smithy-security-oss-components/enricher-aggregator
- pkg:helm/smithy-security-oss-components/consumer-mongodb
- pkg:helm/smithy-security-oss-components/consumer-elasticsearch
1 change: 0 additions & 1 deletion examples/pipelines/sca-project/kustomization.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,5 +9,4 @@ components:
- pkg:helm/smithy-security-oss-components/producer-aggregator
- pkg:helm/smithy-security-oss-components/enricher-codeowners
- pkg:helm/smithy-security-oss-components/enricher-aggregator
- pkg:helm/smithy-security-oss-components/consumer-mongodb
- pkg:helm/smithy-security-oss-components/consumer-elasticsearch
1 change: 0 additions & 1 deletion examples/pipelines/typescript-project/kustomization.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,5 +10,4 @@ components:
- pkg:helm/smithy-security-oss-components/producer-aggregator
- pkg:helm/smithy-security-oss-components/enricher-codeowners
- pkg:helm/smithy-security-oss-components/enricher-aggregator
- pkg:helm/smithy-security-oss-components/consumer-mongodb
- pkg:helm/smithy-security-oss-components/consumer-elasticsearch
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,3 @@ components:
- pkg:helm/smithy-security-oss-components/producer-aggregator
- pkg:helm/smithy-security-oss-components/enricher-deduplication
- pkg:helm/smithy-security-oss-components/enricher-aggregator
- pkg:helm/smithy-security-oss-components/consumer-mongodb