Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add environment variables and auto-generate secrets in k8s deployment… #1927

Merged
merged 2 commits into from
Sep 17, 2023
Merged

Add environment variables and auto-generate secrets in k8s deployment… #1927

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
53f2b7e
Add environment variables and auto-generate secrets in k8s deployment…
dominikriemer Sep 13, 2023
f57d4ac
Merge branch 'dev' into 1926-improve-handling-of-secrets-in-k8s
dominikriemer Sep 17, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 9 additions & 3 deletions installer/k8s/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,12 @@ rm -rf ${HOME}/streampipes-k8s
|-------------------------------------------------|---------------------------------------------------------|------------------------------------------|
| streampipes.version | StreamPipes version | "0.93.0-SNAPSHOT" |
| streampipes.registry | StreamPipes registry URL | "apachestreampipes" |
| streampipes.auth.secretName | The secret name for storing secrets | "sp-secrets" |
| streampipes.auth.users.admin.user | The initial admin user | "[email protected]" |
| streampipes.auth.users.admin.password | The initial admin password (leave empty for autogen) | "admin" |
| streampipes.auth.users.service.user | The initial service account user | "sp-service-client" |
| streampipes.auth.users.service.secret | The initial service account secret | empty (auto-generated) |
| streampipes.auth.encryption.passcode | Passcode for value encryption | empty (auto-generated) |
| streampipes.core.appName | StreamPipes backend application name | "backend" |
| streampipes.core.port | StreamPipes backend port | 8030 |
| streampipes.core.persistence.storageClassName | Storage class name for backend PVs | "hostpath" |
Expand Down Expand Up @@ -162,7 +168,7 @@ rm -rf ${HOME}/streampipes-k8s
| external.couchdb.appName | CouchDB application name | "couchdb" |
| external.couchdb.version | CouchDB version | 3.3.1 |
| external.couchdb.user | CouchDB admin username | "admin" |
| external.couchdb.password | CouchDB admin password | "admin" |
| external.couchdb.password | CouchDB admin password | empty (auto-generated) |
| external.couchdb.port | Port for the CouchDB service | 5984 |
| external.couchdb.service.name | Name of the CouchDB service | "couchdb" |
| external.couchdb.service.port | TargetPort of the CouchDB service | 5984 |
Expand All @@ -177,8 +183,8 @@ rm -rf ${HOME}/streampipes-k8s
| external.influxdb.appName | InfluxDB application name | "influxdb" |
| external.influxdb.version | InfluxDB version | 2.6 |
| external.influxdb.username | InfluxDB admin username | "admin" |
| external.influxdb.password | InfluxDB admin password | "sp-admin" |
| external.influxdb.adminToken | InfluxDB admin token | "sp-admin" |
| external.influxdb.password | InfluxDB admin password | empty (auto-generated) |
| external.influxdb.adminToken | InfluxDB admin token | empty (auto-generated) |
| external.influxdb.initOrg | InfluxDB initial organization | "sp" |
| external.influxdb.initBucket | InfluxDB initial bucket | "sp" |
| external.influxdb.initMode | InfluxDB initialization mode | "setup" |
Expand Down
31 changes: 31 additions & 0 deletions installer/k8s/templates/core/backend-deployment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,37 @@ spec:
- name: SP_NATS_PORT
value: "{{ .Values.external.nats.service.port }}"
{{- end }}
- name: SP_INITIAL_ADMIN_EMAIL
value: {{ .Values.streampipes.auth.users.admin.user}}
- name: SP_INITIAL_ADMIN_PASSWORD
valueFrom:
secretKeyRef:
name: {{ .Values.streampipes.auth.secretName }}
key: sp-initial-admin-password
- name: SP_INITIAL_SERVICE_USER
value: "{{ .Values.streampipes.auth.users.service.user }}"
- name: SP_INITIAL_SERVICE_USER_SECRET
valueFrom:
secretKeyRef:
name: {{ .Values.streampipes.auth.secretName }}
key: sp-initial-client-secret
- name: SP_ENCRYPTION_PASSCODE
valueFrom:
secretKeyRef:
name: {{ .Values.streampipes.auth.secretName}}
key: sp-encryption-passcode
- name: SP_COUCHDB_USER
value: "{{ .Values.external.couchdb.user }}"
- name: SP_COUCHDB_PASSWORD
valueFrom:
secretKeyRef:
name: {{ .Values.streampipes.auth.secretName }}
key: sp-couchdb-password
- name: SP_TS_STORAGE_TOKEN
valueFrom:
secretKeyRef:
name: {{ .Values.streampipes.auth.secretName }}
key: sp-ts-storage-token
ports:
- containerPort: {{ .Values.streampipes.core.port }}
volumeMounts:
Expand Down
18 changes: 18 additions & 0 deletions ...ments/extensions-all-iiot-deployment.yaml → .../core/extensions-all-iiot-deployment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,24 @@ spec:
imagePullPolicy: {{ .Values.pullPolicy }}
ports:
- containerPort: {{ .Values.extensions.iiot.port }}
env:
- name: SP_CLIENT_USER
value: "{{ .Values.streampipes.auth.users.service.user }}"
- name: SP_CLIENT_SECRET
valueFrom:
secretKeyRef:
name: "{{ .Values.streampipes.auth.secretName }}"
key: sp-initial-client-secret
- name: SP_COUCHDB_PASSWORD
valueFrom:
secretKeyRef:
name: {{ .Values.streampipes.auth.secretName }}
key: sp-couchdb-password
- name: SP_TS_STORAGE_TOKEN
valueFrom:
secretKeyRef:
name: "{{ .Values.streampipes.auth.secretName }}"
key: sp-ts-storage-token
livenessProbe:
tcpSocket:
port: {{ .Values.extensions.iiot.port }}
Expand Down
0 ...elements/extensions-all-iiot-service.yaml → ...tes/core/extensions-all-iiot-service.yaml
View file
Open in desktop
File renamed without changes.
28 changes: 28 additions & 0 deletions installer/k8s/templates/core/secrets.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

apiVersion: v1
kind: Secret
metadata:
name: sp-secrets
namespace: {{ .Release.Namespace | quote }}
type: Opaque
data:
sp-initial-admin-password: {{ ternary (randAlphaNum 10) .Values.streampipes.auth.users.admin.password (empty .Values.streampipes.auth.users.admin.password) | b64enc | quote }}
sp-initial-client-secret: {{ ternary (randAlphaNum 35) .Values.streampipes.auth.users.service.secret (empty .Values.streampipes.auth.users.service.secret) | b64enc | quote }}
sp-encryption-passcode: {{ ternary (randAlphaNum 20) .Values.streampipes.auth.encryption.passcode (empty .Values.streampipes.auth.encryption.passcode) | b64enc | quote }}
sp-couchdb-password: {{ ternary (randAlphaNum 20) .Values.external.couchdb.password (empty .Values.external.couchdb.password) | b64enc | quote }}
sp-ts-storage-password: {{ ternary (randAlphaNum 20) .Values.external.influxdb.password (empty .Values.external.influxdb.password) | b64enc | quote }}
sp-ts-storage-token: {{ ternary (randAlphaNum 20) .Values.external.influxdb.adminToken (empty .Values.external.influxdb.adminToken) | b64enc | quote }}
7 changes: 5 additions & 2 deletions installer/k8s/templates/external/couchdb/couchdb-deployment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,10 @@ spec:
- name: COUCHDB_USER
value: {{ .Values.external.couchdb.user }}
- name: COUCHDB_PASSWORD
value: {{ .Values.external.couchdb.password }}
valueFrom:
secretKeyRef:
name: {{ .Values.streampipes.auth.secretName }}
key: sp-couchdb-password
ports:
- containerPort: {{ .Values.external.couchdb.port }}
volumeMounts:
Expand All @@ -66,4 +69,4 @@ spec:
port: {{ .Values.external.couchdb.port }}
initialDelaySeconds: {{ .Values.initialDelaySeconds }}
periodSeconds: {{ .Values.periodSeconds }}
failureThreshold: {{ .Values.failureThreshold }}
failureThreshold: {{ .Values.failureThreshold }}
12 changes: 9 additions & 3 deletions installer/k8s/templates/external/influxdb/influxdb-deployment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,9 +53,15 @@ spec:
- name: DOCKER_INFLUXDB_INIT_USERNAME
value: {{ .Values.external.influxdb.username }}
- name: DOCKER_INFLUXDB_INIT_PASSWORD
value: {{ .Values.external.influxdb.password }}
valueFrom:
secretKeyRef:
name: {{ .Values.streampipes.auth.secretName }}
key: sp-ts-storage-password
- name: DOCKER_INFLUXDB_INIT_ADMIN_TOKEN
value: {{ .Values.external.influxdb.adminToken }}
valueFrom:
secretKeyRef:
name: {{ .Values.streampipes.auth.secretName }}
key: sp-ts-storage-token
- name: DOCKER_INFLUXDB_INIT_ORG
value: {{ .Values.external.influxdb.initOrg }}
- name: DOCKER_INFLUXDB_INIT_BUCKET
Expand Down Expand Up @@ -87,4 +93,4 @@ spec:
port: {{ .Values.external.influxdb.httpPort }}
initialDelaySeconds: {{ .Values.initialDelaySeconds }}
periodSeconds: {{ .Values.periodSeconds }}
failureThreshold: {{ .Values.failureThreshold }}
failureThreshold: {{ .Values.failureThreshold }}
15 changes: 13 additions & 2 deletions installer/k8s/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,17 @@ hostPath: ""
streampipes:
version: "0.93.0-SNAPSHOT"
registry: "apachestreampipes"
auth:
secretName: "sp-secrets"
users:
admin:
user: "[email protected]"
password: "admin"
service:
user: "sp-service-client"
secret:
encryption:
passcode:
core:
appName: "backend"
port: 8030
Expand Down Expand Up @@ -93,7 +104,7 @@ external:
appName: "couchdb"
version: 3.3.1
user: "admin"
password: "admin"
password:
port: 5984
service:
name: "couchdb"
Expand All @@ -108,7 +119,7 @@ external:
version: 2.6
username: "admin"
password: "sp-admin"
adminToken: "sp-admin"
adminToken:
initOrg: "sp"
initBucket: "sp"
# For database migration in v0.91.0 - set init mode to 'upgrade' to migrate an existing installation
Expand Down