generated from hashicorp/terraform-provider-scaffolding-framework
-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add original astro runtime version to deployment resource creation #58
Merged
vandyliu
merged 16 commits into
main
from
add-astro-runtime-version-to-deployment-resource-v2
May 15, 2024
Merged
Changes from all commits
Commits
Show all changes
16 commits
Select commit
Hold shift + click to select a range
5826923
Fix bug where you cannot remove cluster workspace restrictions
vandyliu 428e9d9
remove timeout
vandyliu d6b62de
Add astro_runtime_verison to deployment resource, which only works on…
vandyliu b45fd6f
Merge branch 'main' into add-astro-runtime-version-to-deployment-reso…
vandyliu 2bc0577
fix tests
vandyliu 183a6ad
Add original_astro_runtime_version to deployment resource
vandyliu d2625f7
fix update
vandyliu dc3121f
update description for clusters
vandyliu 35bb33a
Merge branch 'main' into add-astro-runtime-version-to-deployment-reso…
vandyliu af83eb4
update
vandyliu 910491b
Merge branch 'add-astro-runtime-version-to-deployment-resource-v2' of…
vandyliu 2d8361d
update makefile
vandyliu c60e76b
change other parallel test to use gcp
vandyliu ebfa06f
change wording: actual -> current
vandyliu 72f3616
Merge branch 'main' into add-astro-runtime-version-to-deployment-reso…
vandyliu da1a0fd
rearrange code but no actual implementation detail changes
vandyliu File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -14,22 +14,23 @@ Deployment resource | |
|
||
```terraform | ||
resource "astro_deployment" "dedicated" { | ||
name = "my dedicated deployment" | ||
description = "an example deployment" | ||
type = "DEDICATED" | ||
cluster_id = "clv17vgft000801kkydsws63x" | ||
contact_emails = ["[email protected]"] | ||
default_task_pod_cpu = "0.25" | ||
default_task_pod_memory = "0.5Gi" | ||
executor = "KUBERNETES" | ||
is_cicd_enforced = true | ||
is_dag_deploy_enabled = true | ||
is_development_mode = false | ||
is_high_availability = true | ||
resource_quota_cpu = "10" | ||
resource_quota_memory = "20Gi" | ||
scheduler_size = "SMALL" | ||
workspace_id = "clnp86ly5000401ndaga21g81" | ||
original_astro_runtime_version = "11.3.0" | ||
name = "my dedicated deployment" | ||
description = "an example deployment" | ||
type = "DEDICATED" | ||
cluster_id = "clv17vgft000801kkydsws63x" | ||
contact_emails = ["[email protected]"] | ||
default_task_pod_cpu = "0.25" | ||
default_task_pod_memory = "0.5Gi" | ||
executor = "KUBERNETES" | ||
is_cicd_enforced = true | ||
is_dag_deploy_enabled = true | ||
is_development_mode = false | ||
is_high_availability = true | ||
resource_quota_cpu = "10" | ||
resource_quota_memory = "20Gi" | ||
scheduler_size = "SMALL" | ||
workspace_id = "clnp86ly5000401ndaga21g81" | ||
environment_variables = [{ | ||
key = "key1" | ||
value = "value1" | ||
|
@@ -38,24 +39,25 @@ resource "astro_deployment" "dedicated" { | |
} | ||
|
||
resource "astro_deployment" "standard" { | ||
name = "my standard deployment" | ||
description = "an example deployment" | ||
type = "STANDARD" | ||
cloud_provider = "AWS" | ||
region = "us-east-1" | ||
contact_emails = [] | ||
default_task_pod_cpu = "0.25" | ||
default_task_pod_memory = "0.5Gi" | ||
executor = "CELERY" | ||
is_cicd_enforced = true | ||
is_dag_deploy_enabled = true | ||
is_development_mode = false | ||
is_high_availability = false | ||
resource_quota_cpu = "10" | ||
resource_quota_memory = "20Gi" | ||
scheduler_size = "SMALL" | ||
workspace_id = "clnp86ly500a401ndaga20g81" | ||
environment_variables = [] | ||
original_astro_runtime_version = "11.3.0" | ||
name = "my standard deployment" | ||
description = "an example deployment" | ||
type = "STANDARD" | ||
cloud_provider = "AWS" | ||
region = "us-east-1" | ||
contact_emails = [] | ||
default_task_pod_cpu = "0.25" | ||
default_task_pod_memory = "0.5Gi" | ||
executor = "CELERY" | ||
is_cicd_enforced = true | ||
is_dag_deploy_enabled = true | ||
is_development_mode = false | ||
is_high_availability = false | ||
resource_quota_cpu = "10" | ||
resource_quota_memory = "20Gi" | ||
scheduler_size = "SMALL" | ||
workspace_id = "clnp86ly500a401ndaga20g81" | ||
environment_variables = [] | ||
worker_queues = [{ | ||
name = "default" | ||
is_default = true | ||
|
@@ -67,18 +69,19 @@ resource "astro_deployment" "standard" { | |
} | ||
|
||
resource "astro_deployment" "hybrid" { | ||
name = "my hybrid deployment" | ||
description = "an example deployment" | ||
type = "HYBRID" | ||
cluster_id = "clnp86ly5000401ndagu20g81" | ||
task_pod_node_pool_id = "clnp86ly5000301ndzfxz895w" | ||
contact_emails = ["[email protected]"] | ||
executor = "KUBERNETES" | ||
is_cicd_enforced = true | ||
is_dag_deploy_enabled = true | ||
scheduler_replicas = 1 | ||
scheduler_au = 5 | ||
workspace_id = "clnp86ly5000401ndaga20g81" | ||
original_astro_runtime_version = "11.3.0" | ||
name = "my hybrid deployment" | ||
description = "an example deployment" | ||
type = "HYBRID" | ||
cluster_id = "clnp86ly5000401ndagu20g81" | ||
task_pod_node_pool_id = "clnp86ly5000301ndzfxz895w" | ||
contact_emails = ["[email protected]"] | ||
executor = "KUBERNETES" | ||
is_cicd_enforced = true | ||
is_dag_deploy_enabled = true | ||
scheduler_replicas = 1 | ||
scheduler_au = 5 | ||
workspace_id = "clnp86ly5000401ndaga20g81" | ||
environment_variables = [{ | ||
key = "key1" | ||
value = "value1" | ||
|
@@ -110,6 +113,7 @@ resource "astro_deployment" "hybrid" { | |
- `default_task_pod_memory` (String) Deployment default task pod memory - required for 'STANDARD' and 'DEDICATED' deployments | ||
- `is_development_mode` (Boolean) Deployment development mode - required for 'STANDARD' and 'DEDICATED' deployments. If changing from 'False' to 'True', the deployment will be recreated | ||
- `is_high_availability` (Boolean) Deployment high availability - required for 'STANDARD' and 'DEDICATED' deployments | ||
- `original_astro_runtime_version` (String) Deployment's original Astro Runtime version. The Terraform provider will use this provided Astro runtime version to create the Deployment. The Astro runtime version can be updated with your Astro project Dockerfile, but if this value is changed, the Deployment will be recreated with this new Astro runtime version. | ||
- `region` (String) Deployment region - required for 'STANDARD' deployments. If changing this value, the deployment will be recreated in the new region | ||
- `resource_quota_cpu` (String) Deployment resource quota CPU - required for 'STANDARD' and 'DEDICATED' deployments | ||
- `resource_quota_memory` (String) Deployment resource quota memory - required for 'STANDARD' and 'DEDICATED' deployments | ||
|
@@ -123,7 +127,7 @@ resource "astro_deployment" "hybrid" { | |
### Read-Only | ||
|
||
- `airflow_version` (String) Deployment Airflow version | ||
- `astro_runtime_version` (String) Deployment Astro Runtime version. The terraform provider will use the latest Astro runtime version for the Deployment. The Astro runtime version can be updated with your Astro project Dockerfile | ||
- `astro_runtime_version` (String) Deployment's current Astro Runtime version | ||
- `created_at` (String) Deployment creation timestamp | ||
- `created_by` (Attributes) Deployment creator (see [below for nested schema](#nestedatt--created_by)) | ||
- `dag_tarball_version` (String) Deployment DAG tarball version | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,20 +1,21 @@ | ||
resource "astro_deployment" "dedicated" { | ||
name = "my dedicated deployment" | ||
description = "an example deployment" | ||
type = "DEDICATED" | ||
cluster_id = "clv17vgft000801kkydsws63x" | ||
contact_emails = ["[email protected]"] | ||
default_task_pod_cpu = "0.25" | ||
default_task_pod_memory = "0.5Gi" | ||
executor = "KUBERNETES" | ||
is_cicd_enforced = true | ||
is_dag_deploy_enabled = true | ||
is_development_mode = false | ||
is_high_availability = true | ||
resource_quota_cpu = "10" | ||
resource_quota_memory = "20Gi" | ||
scheduler_size = "SMALL" | ||
workspace_id = "clnp86ly5000401ndaga21g81" | ||
original_astro_runtime_version = "11.3.0" | ||
name = "my dedicated deployment" | ||
description = "an example deployment" | ||
type = "DEDICATED" | ||
cluster_id = "clv17vgft000801kkydsws63x" | ||
contact_emails = ["[email protected]"] | ||
default_task_pod_cpu = "0.25" | ||
default_task_pod_memory = "0.5Gi" | ||
executor = "KUBERNETES" | ||
is_cicd_enforced = true | ||
is_dag_deploy_enabled = true | ||
is_development_mode = false | ||
is_high_availability = true | ||
resource_quota_cpu = "10" | ||
resource_quota_memory = "20Gi" | ||
scheduler_size = "SMALL" | ||
workspace_id = "clnp86ly5000401ndaga21g81" | ||
environment_variables = [{ | ||
key = "key1" | ||
value = "value1" | ||
|
@@ -23,24 +24,25 @@ resource "astro_deployment" "dedicated" { | |
} | ||
|
||
resource "astro_deployment" "standard" { | ||
name = "my standard deployment" | ||
description = "an example deployment" | ||
type = "STANDARD" | ||
cloud_provider = "AWS" | ||
region = "us-east-1" | ||
contact_emails = [] | ||
default_task_pod_cpu = "0.25" | ||
default_task_pod_memory = "0.5Gi" | ||
executor = "CELERY" | ||
is_cicd_enforced = true | ||
is_dag_deploy_enabled = true | ||
is_development_mode = false | ||
is_high_availability = false | ||
resource_quota_cpu = "10" | ||
resource_quota_memory = "20Gi" | ||
scheduler_size = "SMALL" | ||
workspace_id = "clnp86ly500a401ndaga20g81" | ||
environment_variables = [] | ||
original_astro_runtime_version = "11.3.0" | ||
name = "my standard deployment" | ||
description = "an example deployment" | ||
type = "STANDARD" | ||
cloud_provider = "AWS" | ||
region = "us-east-1" | ||
contact_emails = [] | ||
default_task_pod_cpu = "0.25" | ||
default_task_pod_memory = "0.5Gi" | ||
executor = "CELERY" | ||
is_cicd_enforced = true | ||
is_dag_deploy_enabled = true | ||
is_development_mode = false | ||
is_high_availability = false | ||
resource_quota_cpu = "10" | ||
resource_quota_memory = "20Gi" | ||
scheduler_size = "SMALL" | ||
workspace_id = "clnp86ly500a401ndaga20g81" | ||
environment_variables = [] | ||
worker_queues = [{ | ||
name = "default" | ||
is_default = true | ||
|
@@ -52,18 +54,19 @@ resource "astro_deployment" "standard" { | |
} | ||
|
||
resource "astro_deployment" "hybrid" { | ||
name = "my hybrid deployment" | ||
description = "an example deployment" | ||
type = "HYBRID" | ||
cluster_id = "clnp86ly5000401ndagu20g81" | ||
task_pod_node_pool_id = "clnp86ly5000301ndzfxz895w" | ||
contact_emails = ["[email protected]"] | ||
executor = "KUBERNETES" | ||
is_cicd_enforced = true | ||
is_dag_deploy_enabled = true | ||
scheduler_replicas = 1 | ||
scheduler_au = 5 | ||
workspace_id = "clnp86ly5000401ndaga20g81" | ||
original_astro_runtime_version = "11.3.0" | ||
name = "my hybrid deployment" | ||
description = "an example deployment" | ||
type = "HYBRID" | ||
cluster_id = "clnp86ly5000401ndagu20g81" | ||
task_pod_node_pool_id = "clnp86ly5000301ndzfxz895w" | ||
contact_emails = ["[email protected]"] | ||
executor = "KUBERNETES" | ||
is_cicd_enforced = true | ||
is_dag_deploy_enabled = true | ||
scheduler_replicas = 1 | ||
scheduler_au = 5 | ||
workspace_id = "clnp86ly5000401ndaga20g81" | ||
environment_variables = [{ | ||
key = "key1" | ||
value = "value1" | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just curious about the increase here, is this due to the resource tests being added?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seems like the tests take 2.5 hours to run (and pass) so I upped it