Add terraform import script

[ichung08]

Description

Add an import script that will enable users to migrate all of their resources to be managed by terraform. The script will read from existing resources (by listing all orgs, deployments, clusters, etc) and then correctly import them into terraform.

How the script works:

1. Parses all the resources to be imported (which are passed in as args)
2. For each type of resource (eg. Workspace, Cluster, Deployment etc), it gets all the entity ids
3. For each entity id, it generates a Terraform import block in the form of:

import {
	id = "%v"
	to = astro_resource.resource_%v
}

4. After generating all the import blocks, it concatenates all the outputs and writes it to a file, import.tf
5. Once the import blocks are written, it then generates the Terraform HCL configuration using terraform plan -generate-config-out=generated.tf, writing all Terraform resource configs to a file, generated.tf
6. Lastly, the script auto approves and runs terraform apply on the new resources, successfully importing all resources!

Example import.tf file:

terraform {
	required_providers {
		astro = {
			source = "registry.terraform.io/astronomer/astro"
		}
	}
}

provider "astro" {
	organization_id = "clx42kkcm01fo01o06agtmshg"
	host = "https://api.astronomer-dev.io"
}

import {
	id = "clzlybvvw00kv01j6o0fq77ep"
	to = astro_workspace.workspace_clzlybvvw00kv01j6o0fq77ep
}

import {
	id = "clx42trhg01go01o07e0dozht"
	to = astro_cluster.cluster_clx42trhg01go01o07e0dozht
}

import {
	id = "clx42ugkb01gq01od03k7c49a"
	to = astro_cluster.cluster_clx42ugkb01gq01od03k7c49a
}

import {
	id = "clzawipbm00bm01qw98vzzoca"
	to = astro_user_roles.user_clzawipbm00bm01qw98vzzoca
}

Example generated.tf file:

resource "astro_workspace" "workspace_clzlm6cev00ki01lyvqyyoaal" {
  cicd_enforced_default = false
  description           = "Created by Terraform Acceptance Test - should self-cleanup but can delete manually if needed after 2 hours."
  name                  = "TFAcceptanceTest_QPROZQXNVZ_workspace"
}

# __generated__ by Terraform from "clx42trhg01go01o07e0dozht"
resource "astro_cluster" "cluster_clx42trhg01go01o07e0dozht" {
  cloud_provider        = "AWS"
  name                  = "Terraform_AWS_Cluster_DND"
  pod_subnet_range      = null
  region                = "us-east-1"
  service_peering_range = null
  service_subnet_range  = null
  timeouts              = null
  type                  = "DEDICATED"
  vpc_subnet_range      = "172.20.0.0/20"
  workspace_ids         = []
}

# __generated__ by Terraform from "cl7qqe4tf264442d28fttoe7g8"
resource "astro_user_roles" "user_cl7qqe4tf264442d28fttoe7g8" {
  deployment_roles  = null
  organization_role = "ORGANIZATION_OWNER"
  user_id           = "cl7qqe4tf264442d28fttoe7g8"
  workspace_roles   = null
}

🎟 Issue(s)

#115

Migration Doc

🧪 Functional Testing

Workspace

Cluster

Hybrid Cluster Workspace Authorization
No hybrid clusters to test with

Deployment

Team

Team Roles

API Token

User Roles

Importing Multiple Resources

📸 Screenshots

📋 Checklist

• Added/updated applicable tests
• Added/updated examples in the examples/ directory
• Updated any related documentation

[ichung08]

Any go logging library recommendations? The logging works, but want to improve the look of the logs

[vandyliu]

dont forget to test this after merge

[vandyliu]

curious why we need a goroutine here

[ichung08]

the block above parallelizes the import block generation step for each resource in individual go routines

to ensure that all the go routines are finished, we need a separate goroutine to wait for all the goroutines to finish writing to the channel before closing the channel

[vandyliu]

would it work the same without the goroutine?
to me, it seems like without go routine makes more sense so just

wg.Wait()
close(results)

but maybe im missing something

[vandyliu]

whats HCL mean here

[ichung08]

HashiCorp Configuration Language

[vandyliu]

can all these functions just go in the test file instead?

[ichung08]

yes, i wanted to separate it so its cleaner but i can move them all in there too