Contributing Guidelines

Cycloid Team is glad to see you contributing to this project ! In this document, we will provide you some guidelines in order to help get your contribution accepted.

Reporting an issue

Issues

When you find a bug in Terracognita, it should be reported using GitHub issues. Please define key information like your Operating System (OS), Terracognita origin (docker or from source) and finally the version you are using.

Issue Types

There are 6 types of labels, they can be used for issues or PRs:

enhancement: These track specific feature requests and ideas until they are completed. They can evolve from a specification or they can be submitted individually depending on the size.
specification: These track issues with a detailed description, this is like a proposal.
bug: These track bugs with the code
docs: These track problems with the documentation (i.e. missing or incomplete)
maintenance: These tracks problems, update and migration for dependencies / third-party tools
refactoring: These tracks internal improvement with no direct impact on the product
need review: this status must be set when you feel confident with your submission
in progress: some important change has been requested on your submission, so you can toggle from need review to in progress
under discussion: it's time to take a break, think about this submission and try to figure out how we can implement this or this

Submit a contribution

Setup your git repository

If you want to contribute to an existing issue, you can start by forking this repository, then clone your fork on your machine.

$ git clone https://github.com/<your-username>/terracognita.git
$ cd terracognita

In order to stay updated with the upstream, it's highly recommended to add cycloidio/terracognita as a remote upstream.

$ git remote add upstream https://github.com/cycloidio/terracognita.git

Do not forget to frequently update your fork with the upstream.

$ git fetch upstream --prune
$ git rebase upstream/master

Play with the codebase

Build from sources

Since Terracognita is a Go project, Go must be installed and configured on your machine (really ?). We currently support Go1.12 and Go.13 and go modules as dependency manager. You can simply pull all necessaries dependencies by running an initial.

$ make build

This basically builds terracognita with the current sources.

You also need to install other code dependencies not mandatory in the runtime environment:

enumer is used to generate some code.
goimports is used to format / organize code and imports. CI will perform a check on this, we highly recommend to run $ goimports -w <your-modified-files>.

Add a component to an existing provider (AWS or GCP)

We currently support three providers: Amazon Web Services (AWS), Google Cloud Platform (GCP) and Azure Resource Manager (AzureRM). If you want to play around with one of this provider, you can follow the following guidelines.

For both providers, you need to add your component to this places or equivalent:

As a const value:
- here for AWS.
- here for AzureRM.
- here for Google.

⚠️ your component name must exactly map the name as in Terraform documentation. Example: ComputeInstance will be later used as google_compute_instance ⚠️.

//go:generate enumer -type ResourceType -addprefix google_ -transform snake -linecomment

This means you will need to generate some code.

As a key/value, in this:
- array for AWS.
- array for AzureRM.
- array for Google.

This is where you map your component with a middleware function.

var (
	resources = map[ResourceType]rtFn{
		...
		ComputeInstance: computeInstances,
	}
)

Now, you can write your computeInstances function. Check-out the other functions, you basically fetch resources from a middleware layer and you add them as Terraform resources.

AWS Middleware layer

We have an aws/cmd that generates the aws/reader interface, which is then used by each resource. To add a new call you have to add a new Function to the list in aws/cmd/functions.go and run make generate, you'll have the code fully generated for that function. If it has a specific implementation, which is too different from the others you can check the ListBuckets Function.

Example with aws_db_parameter_group

Add your function

Functions are based on https://github.com/aws/aws-sdk-go/tree/master/service. For example with aws_db_parameter_group you should be able to find the Entity, Prefix, Service in https://github.com/aws/aws-sdk-go/blob/master/service/rds/rdsiface/interface.go#L343 In our case we want to read data, the dedicated function is DescribeDBParameterGroups.

Entity: use the function name without prefix "DBParameterGroups"
Prefix: use the function prefix (generally Describe, List or Get)
Service: SubDirectory from aws-sdk-go service service/rds

$ vim aws/cmd/functions.go

	Function{
		Entity:  "DBParameterGroups",
		Prefix:  "Describe",
		Service: "rds",
		Documentation: `
		// GetDBParameterGroups returns all DB parameterGroups based on the input given.
		// Returned values are commented in the interface doc comment block.
		`,
	},

Functions are used to generate Get methods in aws/reader/reader.go. After a make generate execution, we should find a GetDBParameterGroups method at the end of the file corresponding to our previous example.

Add your resource type

The naming is based on terraform resources like aws_db_parameter_group but without the aws prefix and Snake case like bla_foo needs to be replaced by Camel case blaFoo. The function should start with a lowercase and end with a s. The end result would be dbParameterGroups.

$ vim aws/resources.go

const (
	...
	DBParameterGroup
)

...

var (
	resources = map[ResourceType]rtFn{
		DBParameterGroup: dbParameterGroups,
        ...
	}
)

The const is used to generate aws/resourcetype_enumer.go.

Add the associated function to generate terraform codes.

func dbParameterGroups(ctx context.Context, a *aws, resourceType string, filters *filter.Filter) ([]provider.Resource, error) {
	dbParameterGroups, err := a.awsr.GetDBParameterGroups(ctx, nil)

	if err != nil {
		return nil, err
	}

	resources := make([]provider.Resource, 0)
	for _, i := range dbParameterGroups.DBParameterGroups {

		r, err := initializeResource(a, *i.DBParameterGroupName, resourceType)
		if err != nil {
			return nil, err
		}

		resources = append(resources, r)
	}

	return resources, nil
}

Make generate

Last step is to re-generate the following files with enumer and build/install.

aws/reader/reader.go
aws/resourcetype_enumer.go

$ make generate
$ make test

Update CHANGELOG

Don't forget to update the CHANGELOG.md

AzureRM Middleware layer

We have an azurerm/cmd that generates the azurerm/reader_generated.go methods used to get the resources from the Azure SDK. To add a new call you have to add a new Function to the list in aws/cmd/generate.go (and the corresponding AzureAPI if necessary) and run make generate, you'll have the code fully generated for that function.

Example with azurerm_virtual_machine

Add your function

Functions are based on https://github.com/Azure/azure-sdk-for-go/tree/master/services. For example with azurerm_virtual_machine you should be able to find the required information in https://github.com/Azure/azure-sdk-for-go/blob/master/services/compute/mgmt/2019-12-01/compute/virtualmachines.go The API to use is the compute service with the version 2019-12-01. In our case we want to list data from the ResourceGroup used when executing terracognita, the corresponding function is List, sometimes another List function is needed as it's depends if it based on a ResourceGroup, a Location or will only list all resources without those delimiters.

API: SubDirectory azure-sdk-for-go services compute
APIVersion: SubDirectory azure-sdk-for-go mgmt 2019-12-01
Resource: resource name in singular PascalCase VirtualMachine
ResourceGroup: true as the List function require the resource group name as a parameter

$ vim aws/cmd/generate.go

var azureAPIs = []AzureAPI{
	...
	AzureAPI{API: "compute", APIVersion: "2019-07-01"},
}

var functions = []Function{
	...
	Function{Resource: "VirtualMachine", API: "compute", ResourceGroup: true},
}

Functions are used to generate List methods in azurerm/reader_generated.go. After a make generate execution, we should find a ListVirtualMachines method at the end of the file corresponding to our previous example.

Add your resource type

The naming is based on terraform resources like azurerm_virtual_machine but without the azurerm prefix and Snake case like bla_foo needs to be replaced by Camel case blaFoo. The function should start with a lowercase and end with a s. The end result would be virtualMachines.

$ vim azurerm/resources.go

const (
	...
	VirtualMachine ResourceType = iota
)

...

var (
	resources = map[ResourceType]rtFn{
		...
		VirtualMachine: virtualMachines,
	}
)

The const is used to generate azurerm/resourcetype_enumer.go.

Add the associated function to generate terraform codes.

func virtualMachines(ctx context.Context, a *azurerm, resourceType string, tags []tag.Tag) ([]provider.Resource, error) {
	virtualMachines, err := a.azurer.ListVirtualMachines(ctx)
	if err != nil {
		return nil, errors.Wrap(err, "unable to list virtual machines from reader")
	}
	resources := make([]provider.Resource, 0)
	for _, virtualMachine := range virtualMachines {
		r := provider.NewResource(*virtualMachine.ID, resourceType, a)
		resources = append(resources, r)
	}
	return resources, nil
}

Make generate

Last step is to re-generate the following files with enumer and build/install.

azurerm/reader_generated.go
azurerm/resourcetype_enumer.go

$ make generate
$ make test

Update CHANGELOG

Don't forget to update the CHANGELOG.md.

GCP Middleware layer

In reader.go, you can add your middleware function ListInstances. You will need to be equipped with this documentation. Google SDK is pretty standard, APIs are most of the time used in a similar way. You only need to find out if your component belongs to a project or a project and a zone. It's highly recommended to base your code on the other functions (a method to generate this function will be provided soon).

Build and test your component

That's it ! You can now generate some code and build your binary:

$ make generate && make build
$ ./terracognita google resources
...
google_compute_instance
...

Add a new provider

For this, please open an issue to describe the provider that you want to add. We will discuss about the best way to help you in the implementation.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CONTRIBUTING.md

CONTRIBUTING.md

Contributing Guidelines

Reporting an issue

Issues

Issue Types

Submit a contribution

Setup your git repository

Play with the codebase

Build from sources

Add a component to an existing provider (AWS or GCP)

AWS Middleware layer

Example with aws_db_parameter_group

AzureRM Middleware layer

Example with azurerm_virtual_machine

GCP Middleware layer

Build and test your component

Add a new provider

Files

CONTRIBUTING.md

Latest commit

History

CONTRIBUTING.md

File metadata and controls

Contributing Guidelines

Reporting an issue

Issues

Issue Types

Submit a contribution

Setup your git repository

Play with the codebase

Build from sources

Add a component to an existing provider (AWS or GCP)

AWS Middleware layer

Example with aws_db_parameter_group

AzureRM Middleware layer

Example with azurerm_virtual_machine

GCP Middleware layer

Build and test your component

Add a new provider