Create EC2 resource using terraform

[guy9050]

This PR will contain a terraform plan for creating an EC2 instace for hosting CRC.

Signed-off-by: Guy Inger [email protected]

[guy9050]

I was able to successfully create and destroy an EC2 instance with this terraform file in this PR.

Diving into the scrip though, it does more things rather than just create an EC2 instance.
For example, the cluster installation step, do we want to it with terraform? Or would it make sense to integrate Ansible and use it for installing the cluster, swapping the ssh_keys, i.e all the things that are not related to the EC2 resource itself?

[lmilbaum]

I was able to successfully create and destroy an EC2 instance with this terraform file in this PR.

Diving into the scrip though, it does more things rather than just create an EC2 instance. For example, the cluster installation step, do we want to it with terraform? Or would it make sense to integrate Ansible and use it for installing the cluster, swapping the ssh_keys, i.e all the things that are not related to the EC2 resource itself?

I don't think the cluster installation should be handled by Terraform. I think it is part of the provisioning phase. Ansible is better suited for such a task.
Regarding the key swapping. I don't understand why it is needed. Terraform can create a key pair and place the pem file in the developers ~/.ssh folder. Feel free to poke into what Yael has done in coffr/poc.

[adrianriobo]

There are several question around this:

• About the setup of the cluster, currently this is done based on the bash scripts, but there is a spike on going to add the logic to the bundle itself so basically it would be only required to pass certain param when booting the machine and the bundle will be self configured, so not sure how much to we want to deal with it...actually this kind of "conflicts" with some points on [Architecture] Infrastructure Deployment API & Plugin development #23 probably this should be mention during next meeting.

• About the swapping, there are certain requirements that providers would ask on the imported images. at the moment I can not remember if the key setup on AWS is done with an agent or based on cloud-init, anyway images from crc does not include such solutions , so even when you create a aws keypair it is not setup on the machine as the user key... this is done with the key swapping script mentioned by @guy9050

[adrianriobo]

Also about what @guy9050 asked on where does those scripts should be done, in the past when I worked with IaC I added those type of setup steps as resources handled by the framework....this ensures you track the state of it as any other resource...with terraform this typically will be done with remote-exec at least under that point of view. I am working on the pulumi PoC and this is how I am doing it there

[gbraad]

I am working on the pulumi PoC and this is how I am doing it there

@tsebastiani QE has been using a similar tool for their own testing purposes, and this is what Adrian has been working on for some time. It does the same as crc-cloud.sh. Let's also have a look at his code/PoC and see how we can adopt ideas or patterns, as I agree, maintainability is key.

One of the preparations on our end for the 'key swapping' and other tasks it to make the images more self-container/self-sufficient: crc-org/snc#638. Ideally we make modifications that work for both CRC and CRC-Cloud. I am not sure if Ansible is a necessity, but could be considered for more advanced tasks is needed after the main provisioning, though should not be a hard requirement as this might impact CRC.

[tsebastiani]

About the swapping, there are certain requirements that providers would ask on the imported images. at the moment I can not remember if the key setup on AWS is done with an agent or based on cloud-init, anyway images from crc does not include such solutions , so even when you create a aws keypair it is not setup on the machine as the user key... this is done with the key swapping script mentioned by @guy9050

AWS keypair creation is done only because it's mandatory, but it's absolutely useless. In order to make it work effectively (as you said) the image should include some tweaks that are not actually implemented in the crc image (like cloud-init). To overcome this issue, and make the newly deployed image safe, a keypair is generated by the script and is swapped with the default one

crc-cloud/crc-cloud.sh

Line 117 in d846f4c

swap_ssh_key() {

[lmilbaum]

Several points were mentioned in the discussion. I want to focus on one of them. This PR is about bootstrapping only. It deals with only the AWS resources. Key Pair is one of those resources. AFAIK, there are two spikes worked in parallel. One is done one and the second one is with Pulumi. Eventually, we would pick one of them and drop the other. This was my understanding.

[guy9050]

Thanks for the discussion. This PR focuses on merely the EC2 resources needed for the cluster. The CRC cluster installation will have its own PR.

[guy9050]

Suggested change

# Specify the cloud provider

[guy9050]

Suggested change

# Create a resource of type "aws_security_group" to enable SSH

[guy9050]

Not relevant anymore.