-
Notifications
You must be signed in to change notification settings - Fork 19
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #572 from RH-csaggin/ait_solr
add solr for AI-telemetry
- Loading branch information
Showing
15 changed files
with
364 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| apiVersion: kustomize.config.k8s.io/v1beta1 | ||
| kind: Kustomization | ||
|
|
||
| resources: | ||
| - solr-secret.yaml |
15 changes: 15 additions & 0 deletions
15
ai-telemetry/base/externalsecrets/solr-externalsecret.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| apiVersion: external-secrets.io/v1beta1 | ||
| kind: ExternalSecret | ||
| metadata: | ||
| name: solr-secret | ||
| namespace: solr | ||
| spec: | ||
| refreshInterval: 15s | ||
| secretStoreRef: | ||
| name: nerc-cluster-secrets | ||
| kind: ClusterSecretStore | ||
| target: | ||
| name: solr-secret | ||
| dataFrom: | ||
| - extract: | ||
| key: $ENV/$CLUSTER/ai-telemetry/solr |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -2,3 +2,4 @@ apiVersion: kustomize.config.k8s.io/v1beta1 | |
| kind: Kustomization | ||
| resources: | ||
| - zookeeper | ||
| - solr | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,41 @@ | ||
| apiVersion: v1 | ||
| kind: ConfigMap | ||
| metadata: | ||
| name: solr-scripts | ||
| labels: | ||
| app.kubernetes.io/instance: solr | ||
| app.kubernetes.io/name: solr | ||
| app.kubernetes.io/component: solr | ||
| data: | ||
| init-certs.sh: |- | ||
| #!/bin/bash | ||
| . /opt/bitnami/scripts/liblog.sh | ||
| if [[ -f "/certs/keystore.p12" ]] && [[ -f "/certs/truststore.p12" ]]; then | ||
| # the user provided keystore.p12 and truststore.p12 files (prefered) | ||
| cp "/certs/keystore.p12" "/opt/bitnami/solr/certs/keystore.p12" | ||
| cp "/certs/truststore.p12" "/opt/bitnami/solr/certs/truststore.p12" | ||
| elif [[ -f "/certs/ca.crt" ]] && [[ -f "/certs/tls.key" ]] && [[ -f "/certs/tls.crt" ]]; then | ||
| # the user provided ca.crt & tls.key & tls.crt so we "calculate" keystore.p12 and truststore.p12 | ||
| openssl pkcs12 -export -in "/certs/tls.crt" \ | ||
| -inkey "/certs/tls.key" -out "/tmp/keystore.p12" \ | ||
| -passin pass:"/certs/tls.key" -passout pass:"${SOLR_SSL_KEY_STORE_PASSWORD}" | ||
| keytool -importkeystore -srckeystore "/tmp/keystore.p12" \ | ||
| -srcstoretype PKCS12 \ | ||
| -srcstorepass "${SOLR_SSL_KEY_STORE_PASSWORD}" \ | ||
| -deststorepass "${SOLR_SSL_KEY_STORE_PASSWORD}" \ | ||
| -destkeystore "/opt/bitnami/solr/certs/keystore.p12" \ | ||
| -noprompt | ||
| rm "/tmp/keystore.p12" | ||
| keytool -import -file "/certs/ca.crt" -keystore "/opt/bitnami/solr/certs/truststore.p12" -storepass "${SOLR_SSL_TRUST_STORE_PASSWORD}" -noprompt | ||
| else | ||
| info "No certificate files provided ... nothing to do ..." | ||
| fi | ||
| setup.sh: |- | ||
| #!/bin/bash | ||
| NODE_ID="${MY_POD_NAME#"solr-"}" | ||
| if [[ "$NODE_ID" -eq "0" ]]; then | ||
| export SOLR_CLOUD_BOOTSTRAP=yes | ||
| fi | ||
| # Use hostname instead of IP to register in ZooKeeper | ||
| export SOLR_HOST="${MY_POD_NAME}.solr-headless.solr.svc.cluster.local" | ||
| /opt/bitnami/scripts/solr/entrypoint.sh /opt/bitnami/scripts/solr/run.sh |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| apiVersion: kustomize.config.k8s.io/v1beta1 | ||
| kind: Kustomization | ||
| namespace: solr | ||
| resources: | ||
| - networkpolicy.yaml | ||
| - configmap.yaml | ||
| - service-headless.yaml | ||
| - service.yaml | ||
| - serviceaccount.yaml | ||
| - statefulset.yaml |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| kind: NetworkPolicy | ||
| apiVersion: networking.k8s.io/v1 | ||
| metadata: | ||
| name: solr | ||
| labels: | ||
| app.kubernetes.io/instance: solr | ||
| app.kubernetes.io/name: solr | ||
| spec: | ||
| podSelector: | ||
| matchLabels: | ||
| app.kubernetes.io/instance: solr | ||
| app.kubernetes.io/name: solr | ||
| policyTypes: | ||
| - Ingress | ||
| - Egress | ||
| egress: | ||
| - {} | ||
| ingress: | ||
| - ports: | ||
| - port: 8983 | ||
| - port: 8983 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| apiVersion: v1 | ||
| kind: Service | ||
| metadata: | ||
| name: solr-headless | ||
| labels: | ||
| app.kubernetes.io/instance: solr | ||
| app.kubernetes.io/name: solr | ||
| app.kubernetes.io/component: solr | ||
| spec: | ||
| type: ClusterIP | ||
| clusterIP: None | ||
| publishNotReadyAddresses: true | ||
| ports: | ||
| - name: tcp-client | ||
| port: 8983 | ||
| protocol: TCP | ||
| targetPort: http | ||
| selector: | ||
| app.kubernetes.io/instance: solr | ||
| app.kubernetes.io/name: solr | ||
| app.kubernetes.io/component: solr |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| apiVersion: v1 | ||
| kind: Service | ||
| metadata: | ||
| name: solr | ||
| labels: | ||
| app.kubernetes.io/instance: solr | ||
| app.kubernetes.io/name: solr | ||
| app.kubernetes.io/component: solr | ||
| spec: | ||
| type: ClusterIP | ||
| sessionAffinity: None | ||
| ports: | ||
| - name: tcp-client | ||
| port: 8983 | ||
| protocol: TCP | ||
| targetPort: http | ||
| selector: | ||
| app.kubernetes.io/instance: solr | ||
| app.kubernetes.io/name: solr | ||
| app.kubernetes.io/component: solr |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| apiVersion: v1 | ||
| kind: ServiceAccount | ||
| metadata: | ||
| name: solr | ||
| labels: | ||
| app.kubernetes.io/instance: solr | ||
| app.kubernetes.io/name: solr | ||
| app.kubernetes.io/component: solr | ||
| automountServiceAccountToken: false |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,196 @@ | ||
| apiVersion: apps/v1 | ||
| kind: StatefulSet | ||
| metadata: | ||
| name: solr | ||
| labels: | ||
| app.kubernetes.io/instance: solr | ||
| app.kubernetes.io/name: solr | ||
| app.kubernetes.io/component: solr | ||
| spec: | ||
| podManagementPolicy: Parallel | ||
| replicas: 3 | ||
| revisionHistoryLimit: 10 | ||
| selector: | ||
| matchLabels: | ||
| app.kubernetes.io/instance: solr | ||
| app.kubernetes.io/name: solr | ||
| app.kubernetes.io/component: solr | ||
| serviceName: solr-headless | ||
| updateStrategy: | ||
| rollingUpdate: {} | ||
| type: RollingUpdate | ||
| template: | ||
| metadata: | ||
| labels: | ||
| app.kubernetes.io/instance: solr | ||
| app.kubernetes.io/name: solr | ||
| app.kubernetes.io/component: solr | ||
| spec: | ||
| serviceAccountName: solr | ||
| automountServiceAccountToken: false | ||
| affinity: | ||
| podAffinity: | ||
| podAntiAffinity: | ||
| preferredDuringSchedulingIgnoredDuringExecution: | ||
| - podAffinityTerm: | ||
| labelSelector: | ||
| matchLabels: | ||
| app.kubernetes.io/instance: solr | ||
| app.kubernetes.io/name: solr | ||
| app.kubernetes.io/component: solr | ||
| topologyKey: kubernetes.io/hostname | ||
| weight: 1 | ||
| nodeAffinity: | ||
| enableServiceLinks: true | ||
| initContainers: | ||
| - name: prepare-server-dir | ||
| image: docker.io/bitnami/solr:9.7.0-debian-12-r0 | ||
| imagePullPolicy: "IfNotPresent" | ||
| resources: | ||
| limits: | ||
| cpu: "1" | ||
| ephemeral-storage: 2Gi | ||
| memory: 2Gi | ||
| requests: | ||
| cpu: 100m | ||
| ephemeral-storage: 50Mi | ||
| memory: 100Mi | ||
| securityContext: | ||
| allowPrivilegeEscalation: false | ||
| capabilities: | ||
| drop: | ||
| - ALL | ||
| privileged: false | ||
| readOnlyRootFilesystem: true | ||
| command: | ||
| - /bin/bash | ||
| args: | ||
| - -ec | ||
| - | | ||
| #!/bin/bash | ||
| . /opt/bitnami/scripts/liblog.sh | ||
| info "Copying server dir to empty dir" | ||
| # In order to not break the application functionality (such as upgrades or plugins) we need | ||
| # to make the base directory writable, so we need to copy it to an empty dir volume | ||
| cp -r --preserve=mode /opt/bitnami/solr/server /emptydir/app-server-dir | ||
| info "Copy operation completed" | ||
| volumeMounts: | ||
| - name: empty-dir | ||
| mountPath: /emptydir | ||
| containers: | ||
| - name: solr | ||
| image: docker.io/bitnami/solr:9.7.0-debian-12-r0 | ||
| imagePullPolicy: "IfNotPresent" | ||
| securityContext: | ||
| allowPrivilegeEscalation: false | ||
| capabilities: | ||
| drop: | ||
| - ALL | ||
| privileged: false | ||
| readOnlyRootFilesystem: true | ||
| command: | ||
| - /scripts/setup.sh | ||
| env: | ||
| - name: BITNAMI_DEBUG | ||
| value: "false" | ||
| - name: MY_POD_NAME | ||
| valueFrom: | ||
| fieldRef: | ||
| fieldPath: metadata.name | ||
| - name: SOLR_ENABLE_CLOUD_MODE | ||
| value: "yes" | ||
| - name: SOLR_NUMBER_OF_NODES | ||
| value: "3" | ||
| - name: SOLR_PORT_NUMBER | ||
| value: "8983" | ||
| - name: SOLR_SERVER_DIRECTORY | ||
| value: "server" | ||
| - name: SOLR_COLLECTION | ||
| value: "my-collection" | ||
| - name: SOLR_COLLECTION_SHARDS | ||
| value: "3" | ||
| - name: SOLR_COLLECTION_REPLICAS | ||
| value: "1" | ||
| - name: SOLR_ENABLE_AUTHENTICATION | ||
| value: "yes" | ||
| - name: SOLR_ADMIN_USERNAME | ||
| value: "admin" | ||
| - name: SOLR_ADMIN_PASSWORD | ||
| valueFrom: | ||
| secretKeyRef: | ||
| name: solr-secret | ||
| key: solr-password | ||
| - name: SOLR_ZK_HOSTS | ||
| value: "zookeeper.zookeeper.svc:2181" | ||
| ports: | ||
| - name: http | ||
| containerPort: 8983 | ||
| resources: | ||
| limits: | ||
| cpu: "1" | ||
| ephemeral-storage: 2Gi | ||
| memory: 3Gi | ||
| requests: | ||
| cpu: 850m | ||
| ephemeral-storage: 50Mi | ||
| memory: 1Gi | ||
| livenessProbe: | ||
| failureThreshold: 6 | ||
| initialDelaySeconds: 40 | ||
| periodSeconds: 10 | ||
| successThreshold: 1 | ||
| timeoutSeconds: 15 | ||
| exec: | ||
| command: | ||
| - /bin/bash | ||
| - -ec | ||
| - | | ||
| curl --silent --connect-timeout 15000 --user ${SOLR_ADMIN_USERNAME}:${SOLR_ADMIN_PASSWORD} http://localhost:${SOLR_PORT_NUMBER}/solr/admin/info/system | grep --quiet '\"status\":0' | ||
| readinessProbe: | ||
| failureThreshold: 6 | ||
| initialDelaySeconds: 60 | ||
| periodSeconds: 10 | ||
| successThreshold: 1 | ||
| timeoutSeconds: 15 | ||
| exec: | ||
| command: | ||
| - /bin/bash | ||
| - -ec | ||
| - | | ||
| curl --silent --connect-timeout 15000 --user ${SOLR_ADMIN_USERNAME}:${SOLR_ADMIN_PASSWORD} http://localhost:${SOLR_PORT_NUMBER}/api/node/health | grep --quiet '\"status\":\"OK\"' | ||
| volumeMounts: | ||
| - name: empty-dir | ||
| mountPath: /tmp | ||
| subPath: tmp-dir | ||
| - name: empty-dir | ||
| mountPath: /opt/bitnami/solr/server | ||
| subPath: app-server-dir | ||
| - name: empty-dir | ||
| mountPath: /opt/bitnami/solr/logs | ||
| subPath: app-logs-dir | ||
| - name: empty-dir | ||
| mountPath: /opt/bitnami/solr/tmp | ||
| subPath: app-tmp-dir | ||
| - name: scripts | ||
| mountPath: /scripts/setup.sh | ||
| subPath: setup.sh | ||
| - name: data | ||
| mountPath: /bitnami/solr | ||
| volumes: | ||
| - name: empty-dir | ||
| emptyDir: {} | ||
| - name: scripts | ||
| configMap: | ||
| name: solr-scripts | ||
| defaultMode: 0755 | ||
| volumeClaimTemplates: | ||
| - apiVersion: v1 | ||
| kind: PersistentVolumeClaim | ||
| metadata: | ||
| name: data | ||
| spec: | ||
| accessModes: | ||
| - "ReadWriteOnce" | ||
| resources: | ||
| requests: | ||
| storage: "50Gi" |
9 changes: 9 additions & 0 deletions
9
ai-telemetry/overlays/externalsecrets/solr-externalsecret.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| apiVersion: external-secrets.io/v1beta1 | ||
| kind: ExternalSecret | ||
| metadata: | ||
| name: solr-secret | ||
| namespace: solr | ||
| spec: | ||
| dataFrom: | ||
| - extract: | ||
| key: nerc/nerc-ocp-obs/ai-telemetry/solr |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| apiVersion: kustomize.config.k8s.io/v1beta1 | ||
| kind: Kustomization | ||
| resources: | ||
| - namespace.yaml |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| apiVersion: v1 | ||
| kind: Namespace | ||
| metadata: | ||
| name: solr | ||
| spec: {} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| apiVersion: kustomize.config.k8s.io/v1beta1 | ||
| kind: Kustomization | ||
| commonLabels: | ||
| nerc.mghpcc.org/bundle: solr | ||
| resources: | ||
| - ../../base/core/namespaces/solr |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters