diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 0301e566..e0e92586 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -7,10 +7,9 @@ Welcome to the Terraform Provider Astro project! We're excited that you're inter 1. [Development Environment Setup](#development-environment-setup) 2. [Making Changes](#making-changes) 3. [Testing](#testing) -4. [Submitting Pull Requests](#submitting-pull-requests) -5. [Reporting Issues](#reporting-issues) -6. [Best Practices](#best-practices) -7. [Additional Resources](#additional-resources) +4. [Reporting Issues](#reporting-issues) +5. [Best Practices](#best-practices) +6. [Additional Resources](#additional-resources) ## Development Environment Setup @@ -85,46 +84,30 @@ Ensure you have the following installed: 4. Update documentation if your changes affect the provider's behavior or add new features. ## Testing +1. Run unit tests with `make test`. -1. Run unit tests: - ``` - make test - ``` +2. Run acceptance tests (these will create real resources in your Astro account) with `make testacc`. Acceptance integration tests use a Terraform CLI binary to run real Terraform commands against the Astro API. The goal is to approximate using the provider with Terraform in production as closely as possible. -2. Run acceptance tests (these will create real resources in your Astro account): - ``` - make testacc - ``` - Note: Ensure all required environment variables are set as described in `internal/provider/provider_test_utils.go`. + Using the terraform-plugin-testing framework, each `resource.Test` runs an acceptance test on a resource. + - `ProtoV6ProviderFactories`: map of the provider factories that the test suite will use to create the provider - just has the `astronomer` provider + - `PreCheck`: a function that runs before the test suite starts to check that all the required environment variables are set + - `Steps`: a list of `terraform apply` sequences that the test suite will run. Each step is a `resource.TestStep` that contains a `Config` and `Check` function. + - `Config`: the Terraform configuration that the test will run (ie. the `.tf` file) + - `Check`: function that will verify the state of the resources after the `terraform apply` command has run. -3. Test your changes manually using the `main.tf` file you created earlier: - ``` - terraform init - terraform plan - terraform apply - ``` + In order to run the full suite of Acceptance tests, run `make testacc`. + You will also need to set all the environment variables described in `internal/provider/provider_test_utils.go`. -## Submitting Pull Requests + The acceptance tests will run against the Astronomer API and create/read/update/delete real resources. -1. Commit your changes: - ``` - git add . - git commit -m "Description of your changes" - ``` +3. Test your changes manually using the main.tf file you created earlier: -2. Push your branch to GitHub: ``` - git push origin feature/your-feature-name + terraform init + terraform plan + terraform apply ``` -3. Open a pull request on GitHub. - -4. In your pull request description, include: - - A clear description of the changes - - Any related issue numbers - - Steps to test the changes - - Screenshots or code snippets if applicable - ## Reporting Issues If you encounter a bug or have a suggestion for improvement, please create an issue on the GitHub repository. This helps us track and address problems efficiently. @@ -176,4 +159,4 @@ If you discover a security vulnerability, please do NOT open an issue. Email sec - [Terraform Plugin Framework Documentation](https://developer.hashicorp.com/terraform/plugin/framework) - [Go Documentation](https://golang.org/doc/) -Thank you for contributing to the Terraform Provider Astro project! Your efforts help improve the experience for all Astro users. \ No newline at end of file +Thank you for contributing to the Terraform Provider Astro project! Your efforts help improve the experience for all Astro users. diff --git a/README.md b/README.md index 9ca029fc..afd1e293 100644 --- a/README.md +++ b/README.md @@ -1,39 +1,14 @@ -# Terraform Provider Astro - -## Requirements - -- [Terraform](https://developer.hashicorp.com/terraform/downloads) >= 1.7 -- [Go](https://golang.org/doc/install) >= 1.21 - -## Building The Provider - -1. Clone the repository -2. Enter the repository directory -3. Build the provider using the following `Makefile` command: - -```shell -make dep -make build -``` - -4. The provider binary will be available in the `bin` directory - -## Adding Dependencies - -This provider uses [Go modules](https://github.com/golang/go/wiki/Modules). -Please see the Go documentation for the most up-to-date information about using Go modules. - -To add a new dependency `github.com/author/dependency` to your Terraform provider: - -```shell -go get github.com/author/dependency -go mod tidy -``` - -Then commit the changes to `go.mod` and `go.sum`. +
+ + + +

Official Astro Terraform Provider to configure and manage your Astro infrastructure through an API.

+ + +
## Using the provider -1. Create an [API Token](https://docs.astronomer.io/astro/automation-authentication#step-1-create-an-api-token) to use in the provider. We recommend creating an organization API token since it is the most flexible but the type of your API token will depend on your use case. +1. Create an [API Token](https://docs.astronomer.io/astro/automation-authentication#step-1-create-an-api-token) to use in the provider. Astronomer recommends creating an Organization API token since it is the most flexible but the type of your API token will depend on your use case. 2. Create a `main.tf` file with the following content: ```terraform terraform { @@ -50,6 +25,8 @@ provider "astro" { # your terraform commands here ``` + See [Astro Provider docs](https://registry.terraform.io/providers/astronomer/astro/latest/docs) for supported resources and data sources. + 3. Run the following commands to apply the provider: ```shell export ASTRO_API_TOKEN= @@ -58,154 +35,10 @@ terraform plan # creates a plan consisting of a set of changes that will make yo terraform apply # performs a plan just like terraform plan does, but then actually carries out the planned changes to each resource using the relevant infrastructure provider's API ``` -## Developing the Provider - -If you wish to work on the provider, you'll first need [Go](http://www.golang.org) installed on your machine (see [Requirements](#requirements) above). - -To compile the provider, see [Building The Provider](#building-the-provider). - -To add example docs, add the correspond `.tf` files to the `examples` directory. These should be added for every new data source and resource. - -To run terraform with the provider, create a `.terraformrc` file in your home directory (`~`) with the following content to override the provider installation with the local build: - -```hcl -provider_installation { - dev_overrides { - "registry.terraform.io/astronomer/astro" = "~/terraform-provider-astro/bin" # Your path to the provider binary - } - direct {} -} -``` - -## Example `main.tf` file for development and testing data sources and resources -```terraform -terraform { - required_providers { - astro = { - source = "astronomer/astro" - } - } -} - -# provider configuration -provider "astro" { - organization_id = "" -} - -# get information on an existing workspace -data "astro_workspace" "example" { - id = "" -} - -# output the workspace data to the terminal -output "data_workspace_example" { - value = data.astro_workspace.example -} - -# create a new workspace -resource "astro_workspace" "tf_workspace" { - name = "my workspace" - description = "my first workspace" - cicd_enforced_default = false -} - -# output the newly created workspace resource to the terminal -output "terraform_workspace" { - value = astro_workspace.tf_workspace -} - -# create a new cluster resource -resource "astro_cluster" "tf_cluster" { - type = "DEDICATED" - name = "my first cluster" - region = "us-east-1" - cloud_provider = "AWS" - vpc_subnet_range = "172.20.0.0/20" - workspace_ids = [astro_workspace.tf_workspace.id, data.astro_workspace.example.id] - timeouts = { - create = "3h" - update = "2h" - delete = "20m" - } -} - -# create a new dedicated deployment resource in that cluster -resource "astro_deployment" "tf_dedicated_deployment" { - name = "my first dedicated deployment" - description = "" - cluster_id = astro_cluster.tf_cluster.id - type = "DEDICATED" - contact_emails = ["example@astronomer.io"] - default_task_pod_cpu = "0.25" - default_task_pod_memory = "0.5Gi" - executor = "KUBERNETES" - is_cicd_enforced = true - is_dag_deploy_enabled = true - is_development_mode = false - is_high_availability = true - resource_quota_cpu = "10" - resource_quota_memory = "20Gi" - scheduler_size = "SMALL" - workspace_id = astro_workspace.tf_workspace.id - environment_variables = [{ - key = "key1" - value = "value1" - is_secret = false - }] -} - -# create a new standard deployment resource -resource "astro_deployment" "tf_standard_deployment" { - name = "my first standard deployment" - description = "" - type = "STANDARD" - cloud_provider = "AWS" - region = "us-east-1" - contact_emails = [] - default_task_pod_cpu = "0.25" - default_task_pod_memory = "0.5Gi" - executor = "CELERY" - is_cicd_enforced = true - is_dag_deploy_enabled = true - is_development_mode = false - is_high_availability = false - resource_quota_cpu = "10" - resource_quota_memory = "20Gi" - scheduler_size = "SMALL" - workspace_id = astro_workspace.tf_workspace.id - environment_variables = [] - worker_queues = [{ - name = "default" - is_default = true - astro_machine = "A5" - max_worker_count = 10 - min_worker_count = 0 - worker_concurrency = 1 - }] -} -``` - -## Testing -Unit tests can be run with `make test`. - -### Acceptance tests -Acceptance integration tests use a Terraform CLI binary to run real Terraform commands against the Astro API. The goal is to approximate using the provider with Terraform in production as closely as possible. - -Using the terraform-plugin-testing framework, each `resource.Test` runs an acceptance test on a resource. -- `ProtoV6ProviderFactories`: map of the provider factories that the test suite will use to create the provider - just has the `astronomer` provider -- `PreCheck`: a function that runs before the test suite starts to check that all the required environment variables are set -- `Steps`: a list of `terraform apply` sequences that the test suite will run. Each step is a `resource.TestStep` that contains a `Config` and `Check` function. - - `Config`: the Terraform configuration that the test will run (ie. the `.tf` file) - - `Check`: function that will verify the state of the resources after the `terraform apply` command has run. - -In order to run the full suite of Acceptance tests, run `make testacc`. -You will also need to set all the environment variables described in `internal/provider/provider_test_utils.go`. - -The acceptance tests will run against the Astronomer API and create/read/update/delete real resources. - ## Importing Existing Resources The Astro Terraform Import Script is a tool designed to help you import existing Astro resources into your Terraform configuration. -Currently, this script automates the process of generating Terraform import blocks and resource configurations for the following resources: workspaces, deployments, clusters, hybrid cluster workspace authorizations, API tokens, teams, team roles, and user roles. +This script automates the process of generating Terraform import blocks and resource configurations for the following resources: workspaces, deployments, clusters, hybrid cluster workspace authorizations, API tokens, teams, team roles, and user roles. +See Astro's [import script guide](https://registry.terraform.io/providers/astronomer/astro/latest/docs/guides/import-script) for more information. To use the import script, download the `terraform-provider-astro-import-script` executable file from [releases](https://github.com/astronomer/terraform-provider-astro/releases) based on your OS and architecture and run it with the following command: @@ -226,26 +59,31 @@ On Windows: ### Options - `-resources`: Comma-separated list of resources to import. Accepted values are workspace, deployment, cluster, api_token, team, team_roles, user_roles. -- `-token`: API token to authenticate with the Astro platform. If not provided, the script will attempt to use the `ASTRO_API_TOKEN` environment variable. -- `-organizationId`: Organization ID to import resources from. +- `-token`: API token to authenticate with the Astro platform. This requires the Organization Owner role. If not provided, the script will attempt to use the `ASTRO_API_TOKEN` environment variable. +- `-organizationId`: (Required) Organization ID to import resources from. - `-runTerraformInit`: Run `terraform init` after generating the import configuration. Used for initializing the Terraform state in our GitHub Actions. - `-help`: Display help information. ### Examples -1. Import workspaces and deployments: +1. Import all resources: + ``` + ./terraform-provider-astro-import-script___ -organizationId -token + ``` + +2. Import only workspaces and deployments: ``` - ./terraform-provider-astro-import-script___ -resources workspace,deployment -token your_api_token -organizationId your_org_id + ./terraform-provider-astro-import-script___ -resources workspace,deployment -token -organizationId ``` -2. Import all supported resources and run Terraform init: +3. Import all supported resources and run Terraform init: ``` - ./terraform-provider-astro-import-script___ -resources workspace,deployment,cluster,api_token,team,team_roles,user_roles -token your_api_token -organizationId your_org_id -runTerraformInit + ./terraform-provider-astro-import-script___ -resources workspace,deployment,cluster,api_token,team,team_roles,user_roles -token -organizationId -runTerraformInit ``` -3. Use a different API host (e.g., dev environment): +4. Use a different API host (for example, dev environment): ``` - ./terraform-provider-astro-import-script___ -resources workspace -token your_api_token -organizationId your_org_id + ./terraform-provider-astro-import-script___ -resources workspace -token -organizationId ``` ### Output @@ -277,8 +115,6 @@ The script will generate two main files: 4. **What Terraform versions are required?** - Terraform >= 1.7. -5. **How can I contribute to the provider's development?** - - Submit pull requests, report issues, or suggest improvements on the GitHub repository. ### Troubleshooting @@ -300,4 +136,4 @@ The script will generate two main files: Solution: Ensure your `.terraformrc` file is correctly set up, especially if you're using a local build of the provider for development. -If you encounter any issues not listed here, please check the [GitHub Issues](https://github.com/astronomer/terraform-provider-astro/issues) page or open a new issue with details about your problem. \ No newline at end of file +If you encounter any issues not listed here, please check the [GitHub Issues](https://github.com/astronomer/terraform-provider-astro/issues) page or open a new issue with details about your problem.