Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[worker] update chart to reference self-managed rather than self-hosted for Self Managed Cloud #434

Merged
merged 11 commits into from
Jan 30, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions .mise.toml
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
[tools]
actionlint = '1.7.1'
github-cli = '2.52.0'
helm = '3.15'
helm-ct = '3.11.0'
helm-docs = '1.13.1'
pre-commit = '3.8.0'
actionlint = "1.7.7"
github-cli = "2.65.0"
helm = "3.17"
helm-ct = "3.12.0"
helm-docs = '1.14.2'
pre-commit = "4.1.0"
shellcheck = '0.10.0'
yamllint = '1.35.1'
36 changes: 18 additions & 18 deletions charts/prefect-worker/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,9 +76,9 @@ Workers each have a type corresponding to the execution environment to which the
These settings will ensure that the worker connects to the proper account, workspace, and work pool.
View your Account ID and Workspace ID in your browser URL when logged into Prefect Cloud. For example: `https://app.prefect.cloud/account/abc-my-account-id-is-here/workspaces/123-my-workspace-id-is-here`

### Configuring a Worker for Self Hosted Cloud (not to be confused with [Prefect Server](https://github.com/PrefectHQ/prefect-helm/tree/main/charts/prefect-worker#configuring-a-worker-for-prefect-server))
### Configuring a Worker for Self-managed Cloud (not to be confused with [Self-hosted Prefect Server](https://github.com/PrefectHQ/prefect-helm/tree/main/charts/prefect-worker#configuring-a-worker-for-prefect-server))

1. Create a Kubernetes secret for a Prefect Self Hosted Cloud API key
1. Create a Kubernetes secret for a Prefect Self-managed Cloud API key

First create a file named `api-key.yaml` with the following contents:

Expand All @@ -93,12 +93,12 @@ Workers each have a type corresponding to the execution environment to which the
key: <base64-encoded-api-key>
```

Replace `<base64-encoded-api-key>` with your Prefect Self Hosted Cloud API key encoded in base64. The helm chart looks for a secret of this name and schema, this can be overridden in the `values.yaml`.
Replace `<base64-encoded-api-key>` with your Prefect Self-managed Cloud API key encoded in base64. The helm chart looks for a secret of this name and schema, this can be overridden in the `values.yaml`.

You can use the following command to generate the base64-encoded value:

```bash
echo -n "your-prefect-self-hosted-cloud-api-key" | base64
echo -n "your-prefect-self-managed-cloud-api-key" | base64
```

Then apply the `api-key.yaml` file to create the Kubernetes secret:
Expand All @@ -119,20 +119,20 @@ Workers each have a type corresponding to the execution environment to which the

```yaml
worker:
apiConfig: selfHosted
apiConfig: selfManaged
config:
workPool: <target work pool name>
selfHostedCloudApiConfig:
selfManagedCloudApiConfig:
# If the prefect server is located external to this cluster, set a fully qualified domain name as the apiUrl
# If the prefect server pod is deployed to this cluster, use the cluster DNS endpoint: http://<prefect-server-service-name>.<namespace>.svc.cluster.local:<prefect-server-port>/api
apiUrl: "https://<DNS of Self Hosted Cloud API>"
apiUrl: "https://<DNS of Self-managed Cloud API>"
accountId: <target account ID>
workspaceId: <target workspace ID>
uiUrl: "https://<DNS of Self Hosted Cloud UI>"
uiUrl: "https://<DNS of Self-managed Cloud UI>"
```

These settings will ensure that the worker connects to the proper account, workspace, and work pool.
View your Account ID and Workspace ID in your browser URL when logged into Prefect Cloud. For example: `https://self-hosted-prefect.company/account/abc-my-account-id-is-here/workspaces/123-my-workspace-id-is-here`
View your Account ID and Workspace ID in your browser URL when logged into Prefect Cloud. For example: `https://self-managed-prefect.company/account/abc-my-account-id-is-here/workspaces/123-my-workspace-id-is-here`

### Configuring a Worker for Prefect Server

Expand Down Expand Up @@ -179,7 +179,7 @@ Workers each have a type corresponding to the execution environment to which the

Prefect documentation on [basic auth](https://docs.prefect.io/v3/develop/settings-and-profiles#security-settings)

Self-hosted Prefect servers can be equipped with a Basic Authentication string for an administrator/password combination. Assuming you are running a self-hosted server with basic auth enabled, you can authenticate your worker with the same credentials.
Self-hosted Prefect servers can be equipped with a Basic Authentication string for an administrator/password combination. Assuming you are running a Self-hosted server with basic auth enabled, you can authenticate your worker with the same credentials.

The format of the auth string is `admin:<my-password>` (no brackets).

Expand Down Expand Up @@ -305,7 +305,7 @@ worker:
| serviceAccount.create | bool | `true` | specifies whether a ServiceAccount should be created |
| serviceAccount.name | string | `""` | the name of the ServiceAccount to use. if not set and create is true, a name is generated using the common.names.fullname template |
| worker.affinity | object | `{}` | affinity for worker pods assignment |
| worker.apiConfig | string | `"cloud"` | one of 'cloud', 'selfHosted', or 'server' |
| worker.apiConfig | string | `"cloud"` | one of 'cloud', 'selfManaged', or 'server' |
| worker.autoscaling.enabled | bool | `false` | enable autoscaling for the worker |
| worker.autoscaling.maxReplicas | int | `1` | maximum number of replicas to scale up to |
| worker.autoscaling.minReplicas | int | `1` | minimum number of replicas to scale down to |
Expand Down Expand Up @@ -376,13 +376,13 @@ worker:
| worker.resources.limits | object | `{"cpu":"1000m","memory":"1Gi"}` | the requested limits for the worker container |
| worker.resources.requests | object | `{"cpu":"100m","memory":"256Mi"}` | the requested resources for the worker container |
| worker.revisionHistoryLimit | int | `10` | the number of old ReplicaSets to retain to allow rollback |
| worker.selfHostedCloudApiConfig.accountId | string | `""` | prefect account ID |
| worker.selfHostedCloudApiConfig.apiKeySecret.key | string | `"key"` | prefect API secret key |
| worker.selfHostedCloudApiConfig.apiKeySecret.name | string | `"prefect-api-key"` | prefect API secret name |
| worker.selfHostedCloudApiConfig.apiUrl | string | `""` | prefect API url (PREFECT_API_URL) |
| worker.selfHostedCloudApiConfig.cloudApiUrl | string | `""` | This is used in self managed cloud instances to congfigure events and logs over websockets |
| worker.selfHostedCloudApiConfig.uiUrl | string | `""` | self hosted UI url |
| worker.selfHostedCloudApiConfig.workspaceId | string | `""` | prefect workspace ID |
| worker.selfManagedCloudApiConfig.accountId | string | `""` | prefect account ID |
| worker.selfManagedCloudApiConfig.apiKeySecret.key | string | `"key"` | prefect API secret key |
| worker.selfManagedCloudApiConfig.apiKeySecret.name | string | `"prefect-api-key"` | prefect API secret name |
| worker.selfManagedCloudApiConfig.apiUrl | string | `""` | prefect API url (PREFECT_API_URL) |
| worker.selfManagedCloudApiConfig.cloudApiUrl | string | `""` | This is used in self managed cloud instances to congfigure events and logs over websockets |
| worker.selfManagedCloudApiConfig.uiUrl | string | `""` | self hosted UI url |
| worker.selfManagedCloudApiConfig.workspaceId | string | `""` | prefect workspace ID |
| worker.serverApiConfig.apiUrl | string | `""` | prefect API url (PREFECT_API_URL) |
| worker.serverApiConfig.uiUrl | string | `"http://localhost:4200"` | prefect UI url |
| worker.tolerations | list | `[]` | tolerations for worker pods assignment |
Expand Down
20 changes: 10 additions & 10 deletions charts/prefect-worker/README.md.gotmpl
Original file line number Diff line number Diff line change
Expand Up @@ -76,9 +76,9 @@ Workers each have a type corresponding to the execution environment to which the
These settings will ensure that the worker connects to the proper account, workspace, and work pool.
View your Account ID and Workspace ID in your browser URL when logged into Prefect Cloud. For example: `https://app.prefect.cloud/account/abc-my-account-id-is-here/workspaces/123-my-workspace-id-is-here`

### Configuring a Worker for Self Hosted Cloud (not to be confused with [Prefect Server](https://github.com/PrefectHQ/prefect-helm/tree/main/charts/prefect-worker#configuring-a-worker-for-prefect-server))
### Configuring a Worker for Self-managed Cloud (not to be confused with [Self-hosted Prefect Server](https://github.com/PrefectHQ/prefect-helm/tree/main/charts/prefect-worker#configuring-a-worker-for-prefect-server))

1. Create a Kubernetes secret for a Prefect Self Hosted Cloud API key
1. Create a Kubernetes secret for a Prefect Self-managed Cloud API key

First create a file named `api-key.yaml` with the following contents:

Expand All @@ -93,12 +93,12 @@ Workers each have a type corresponding to the execution environment to which the
key: <base64-encoded-api-key>
```

Replace `<base64-encoded-api-key>` with your Prefect Self Hosted Cloud API key encoded in base64. The helm chart looks for a secret of this name and schema, this can be overridden in the `values.yaml`.
Replace `<base64-encoded-api-key>` with your Prefect Self-managed Cloud API key encoded in base64. The helm chart looks for a secret of this name and schema, this can be overridden in the `values.yaml`.

You can use the following command to generate the base64-encoded value:

```bash
echo -n "your-prefect-self-hosted-cloud-api-key" | base64
echo -n "your-prefect-self-managed-cloud-api-key" | base64
```

Then apply the `api-key.yaml` file to create the Kubernetes secret:
Expand All @@ -119,20 +119,20 @@ Workers each have a type corresponding to the execution environment to which the

```yaml
worker:
apiConfig: selfHosted
apiConfig: selfManaged
config:
workPool: <target work pool name>
selfHostedCloudApiConfig:
selfManagedCloudApiConfig:
# If the prefect server is located external to this cluster, set a fully qualified domain name as the apiUrl
# If the prefect server pod is deployed to this cluster, use the cluster DNS endpoint: http://<prefect-server-service-name>.<namespace>.svc.cluster.local:<prefect-server-port>/api
apiUrl: "https://<DNS of Self Hosted Cloud API>"
apiUrl: "https://<DNS of Self-managed Cloud API>"
accountId: <target account ID>
workspaceId: <target workspace ID>
uiUrl: "https://<DNS of Self Hosted Cloud UI>"
uiUrl: "https://<DNS of Self-managed Cloud UI>"
```

These settings will ensure that the worker connects to the proper account, workspace, and work pool.
View your Account ID and Workspace ID in your browser URL when logged into Prefect Cloud. For example: `https://self-hosted-prefect.company/account/abc-my-account-id-is-here/workspaces/123-my-workspace-id-is-here`
View your Account ID and Workspace ID in your browser URL when logged into Prefect Cloud. For example: `https://self-managed-prefect.company/account/abc-my-account-id-is-here/workspaces/123-my-workspace-id-is-here`

### Configuring a Worker for Prefect Server

Expand Down Expand Up @@ -179,7 +179,7 @@ Workers each have a type corresponding to the execution environment to which the

Prefect documentation on [basic auth](https://docs.prefect.io/v3/develop/settings-and-profiles#security-settings)

Self-hosted Prefect servers can be equipped with a Basic Authentication string for an administrator/password combination. Assuming you are running a self-hosted server with basic auth enabled, you can authenticate your worker with the same credentials.
Self-hosted Prefect servers can be equipped with a Basic Authentication string for an administrator/password combination. Assuming you are running a Self-hosted server with basic auth enabled, you can authenticate your worker with the same credentials.

The format of the auth string is `admin:<my-password>` (no brackets).

Expand Down
35 changes: 35 additions & 0 deletions charts/prefect-worker/UPGRADING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Upgrade guidelines

## > 2025.1.28020410

After version `2025.1.28020410`, the `prefect-worker` chart renamed the `selfHostedCloudApiConfig` key to `selfManagedCloudApiConfig`.

**Before**

```yaml
worker:
selfHostedCloudApiConfig:
apiUrl: ""
accountId: ""
workspaceId: ""
apiKeySecret:
name: prefect-api-key
key: key
cloudApiUrl: ""
uiUrl: ""
```

**After**

```yaml
worker:
selfManagedCloudApiConfig:
apiUrl: ""
accountId: ""
workspaceId: ""
apiKeySecret:
name: prefect-api-key
key: key
cloudApiUrl: ""
uiUrl: ""
```
32 changes: 16 additions & 16 deletions charts/prefect-worker/templates/_helpers.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -28,29 +28,29 @@ Require Prefect Cloud Workspace ID
{{- end -}}

{{/*
Require Self Hosted Cloud Account ID
Require Self-managed Cloud Account ID
*/}}
{{- define "selfHosted.requiredConfig.accountId" -}}
{{- if eq .Values.worker.apiConfig "selfHosted" }}
{{- required "A Prefect Cloud Account ID is required (worker.selfHostedCloudApiConfig.accountId)" .Values.worker.selfHostedCloudApiConfig.accountId -}}
{{- define "selfManaged.requiredConfig.accountId" -}}
{{- if eq .Values.worker.apiConfig "selfManaged" }}
{{- required "A Prefect Cloud Account ID is required (worker.selfManagedCloudApiConfig.accountId)" .Values.worker.selfManagedCloudApiConfig.accountId -}}
{{- end -}}
{{- end -}}

{{/*
Require Self Hosted Cloud Workspace ID
Require Self-managed Cloud Workspace ID
*/}}
{{- define "selfHosted.requiredConfig.workspaceId" -}}
{{- if eq .Values.worker.apiConfig "selfHosted" }}
{{- required "A Prefect Cloud Workspace ID is required (worker.selfHostedCloudApiConfig.workspaceId)" .Values.worker.selfHostedCloudApiConfig.workspaceId -}}
{{- define "selfManaged.requiredConfig.workspaceId" -}}
{{- if eq .Values.worker.apiConfig "selfManaged" }}
{{- required "A Prefect Cloud Workspace ID is required (worker.selfManagedCloudApiConfig.workspaceId)" .Values.worker.selfManagedCloudApiConfig.workspaceId -}}
{{- end -}}
{{- end -}}

{{/*
Require Self Hosted Cloud API URL
Require Self-managed Cloud API URL
*/}}
{{- define "selfHosted.requiredConfig.apiUrl" -}}
{{- if eq .Values.worker.apiConfig "selfHosted" }}
{{- required "The Self Hosted Cloud API URL is required (worker.selfHostedCloudApiConfig.apiUrl)" .Values.worker.selfHostedCloudApiConfig.apiUrl -}}
{{- define "selfManaged.requiredConfig.apiUrl" -}}
{{- if eq .Values.worker.apiConfig "selfManaged" }}
{{- required "The Self-managed Cloud API URL is required (worker.selfManagedCloudApiConfig.apiUrl)" .Values.worker.selfManagedCloudApiConfig.apiUrl -}}
{{- end -}}
{{- end -}}

Expand All @@ -70,8 +70,8 @@ Require Prefect Server API URL
{{- define "worker.apiUrl" -}}
{{- if eq .Values.worker.apiConfig "cloud" }}
{{- printf "%s/accounts/%s/workspaces/%s" .Values.worker.cloudApiConfig.cloudUrl (include "cloud.requiredConfig.accountId" .) (include "cloud.requiredConfig.workspaceId" .) | quote }}
{{- else if eq .Values.worker.apiConfig "selfHosted" }}
{{- printf "%s/accounts/%s/workspaces/%s" (include "selfHosted.requiredConfig.apiUrl" .) (include "selfHosted.requiredConfig.accountId" .) (include "selfHosted.requiredConfig.workspaceId" .) | quote }}
{{- else if eq .Values.worker.apiConfig "selfManaged" }}
{{- printf "%s/accounts/%s/workspaces/%s" (include "selfManaged.requiredConfig.apiUrl" .) (include "selfManaged.requiredConfig.accountId" .) (include "selfManaged.requiredConfig.workspaceId" .) | quote }}
{{- else }}
{{- include "server.requiredConfig.apiUrl" . | quote }}
{{- end }}
Expand All @@ -84,8 +84,8 @@ Require Prefect Server API URL
{{- define "worker.appUrl" -}}
{{- if eq .Values.worker.apiConfig "cloud" }}
{{- printf "https://app.prefect.cloud/account/%s/workspace/%s" (include "cloud.requiredConfig.accountId" .) (include "cloud.requiredConfig.workspaceId" .) | quote }}
{{- else if eq .Values.worker.apiConfig "selfHosted" }}
{{- printf "%s/account/%s/workspace/%s" .Values.worker.selfHostedCloudApiConfig.uiUrl (include "selfHosted.requiredConfig.accountId" .) (include "selfHosted.requiredConfig.workspaceId" .) | quote }}
{{- else if eq .Values.worker.apiConfig "selfManaged" }}
{{- printf "%s/account/%s/workspace/%s" .Values.worker.selfManagedCloudApiConfig.uiUrl (include "selfManaged.requiredConfig.accountId" .) (include "selfManaged.requiredConfig.workspaceId" .) | quote }}
{{- else }}
{{- .Values.worker.serverApiConfig.uiUrl | quote }}
{{- end }}
Expand Down
14 changes: 7 additions & 7 deletions charts/prefect-worker/templates/deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -113,12 +113,12 @@ spec:
secretKeyRef:
name: {{ .Values.worker.cloudApiConfig.apiKeySecret.name }}
key: {{ .Values.worker.cloudApiConfig.apiKeySecret.key }}
{{- else if eq .Values.worker.apiConfig "selfHosted" }}
{{- else if eq .Values.worker.apiConfig "selfManaged" }}
- name: PREFECT_API_KEY
valueFrom:
secretKeyRef:
name: {{ .Values.worker.selfHostedCloudApiConfig.apiKeySecret.name }}
key: {{ .Values.worker.selfHostedCloudApiConfig.apiKeySecret.key }}
name: {{ .Values.worker.selfManagedCloudApiConfig.apiKeySecret.name }}
key: {{ .Values.worker.selfManagedCloudApiConfig.apiKeySecret.key }}
{{- end }}
{{- with .Values.worker }}
resources: {{ coalesce .initContainer.resources .resources | toYaml | nindent 12 }}
Expand Down Expand Up @@ -192,14 +192,14 @@ spec:
secretKeyRef:
name: {{ .Values.worker.cloudApiConfig.apiKeySecret.name }}
key: {{ .Values.worker.cloudApiConfig.apiKeySecret.key }}
{{- else if eq .Values.worker.apiConfig "selfHosted" }}
{{- else if eq .Values.worker.apiConfig "selfManaged" }}
- name: PREFECT_API_KEY
valueFrom:
secretKeyRef:
name: {{ .Values.worker.selfHostedCloudApiConfig.apiKeySecret.name }}
key: {{ .Values.worker.selfHostedCloudApiConfig.apiKeySecret.key }}
name: {{ .Values.worker.selfManagedCloudApiConfig.apiKeySecret.name }}
key: {{ .Values.worker.selfManagedCloudApiConfig.apiKeySecret.key }}
- name: PREFECT_CLOUD_API_URL
value: {{ .Values.worker.selfHostedCloudApiConfig.cloudApiUrl}}
value: {{ .Values.worker.selfManagedCloudApiConfig.cloudApiUrl}}
{{- end }}
- name: PREFECT_DEBUG_MODE
value: {{ .Values.worker.image.debug | quote }}
Expand Down
6 changes: 3 additions & 3 deletions charts/prefect-worker/tests/worker_test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -151,11 +151,11 @@ tests:
path: .spec.template.spec.containers[0].env[?(@.name == "PREFECT_API_URL")].value
value: "https://api.prefect.cloud/api/accounts/my-account-id/workspaces/my-workspace-id"

- it: Should set the correct API configuration for self-hosted cloud
- it: Should set the correct API configuration for self-managed cloud
set:
worker:
apiConfig: "selfHosted"
selfHostedCloudApiConfig:
apiConfig: "selfManaged"
selfManagedCloudApiConfig:
apiUrl: "https://my-prefect-api.com/api"
accountId: "my-account-id"
workspaceId: "my-workspace-id"
Expand Down
14 changes: 7 additions & 7 deletions charts/prefect-worker/values.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -365,10 +365,10 @@
}
}
},
"selfHostedCloudApiConfig": {
"selfManagedCloudApiConfig": {
"type": "object",
"title": "Self Hosted API Config",
"description": "self hosted api configuration",
"title": "Self-managed API Configuration",
"description": "Self-managed API Configuration",
"additionalProperties": false,
"properties": {
"accountId": {
Expand Down Expand Up @@ -405,13 +405,13 @@
},
"uiUrl": {
"type": "string",
"title": "Self Hosted Cloud URL",
"description": "self hosted cloud url"
"title": "Self-managed Cloud URL",
"description": "Self-managed Cloud URL"
},
"apiUrl": {
"type": "string",
"title": "Self Hosted API URL",
"description": "self hosted API url (PREFECT_API_URL)"
"title": "Self-managed API URL",
"description": "Self-managed API URL (PREFECT_API_URL)"
}
}
},
Expand Down
Loading