EKS

Prerequisites

• VPC with enough IP address space. See the requirements

Resources Provisioned

• EKS Cluster
• KMS Key is used to encrypt K8S Secrets
• IAM Role for Service Account is enabled
• KMS Key for EBS volumes is created and ASG is given permission to use the key
• A "default" self managed node group on Bottlerocket is provisioned.

In addition, it is expected that all worker nodes will share one IAM Role and a common security group.

• IAM Role contains the necessary policies for nodes to join the EKS cluster and optionally manage ENI for CNI purposes. IAM Role for Service Account usage is strongly recommended.
• The Security Group has default rules to allow the cluster to function. Additional groups can be added for additional node groups.

Instance refresh on ASG is handled by AWS Node Termination Handler. For the purposes of instance refresh, the following resources are created:

• SQS Queue where events published to be consumed by Node Termination Handler is published.

The Queue ARN will be subsequently used by the eks_self_managed_nodes module to provision additional node groups.

Usage

Defining Providers

Definining providers in reusable modules is deprecated and causes features like modules for_each and count to not work. In addition to the aws providers, the main module and submodules can require additional Kubernetes providers to be configured.

Main Module

The main module uses the kubernetes provider.

provider "aws" {
  # ...
}

module "eks" {
  # ...
}

data "aws_eks_cluster_auth" "this" {
  name = module.eks.cluster_name
}

provider "kubernetes" {
  host                   = module.eks.cluster_endpoint
  cluster_ca_certificate = base64decode(module.eks.cluster_certificate_authority_data)
  token                  = data.aws_eks_cluster_auth.this.token
}

Sub-modules

Other modules might make use of the kubernetes or helm providers

provider "aws" {
  # ...
}

data "aws_eks_cluster" "this" {
  name = var.cluster_name
}

data "aws_eks_cluster_auth" "this" {
  name = var.cluster_name
}

provider "kubernetes" {
  host                   = data.aws_eks_cluster.this.endpoint
  cluster_ca_certificate = base64decode(data.aws_eks_cluster.this.certificate_authority[0].data)
  token                  = data.aws_eks_cluster_auth.this.token
}

provider "helm" {
  kubernetes {
    host                   = data.aws_eks_cluster.this.endpoint
    cluster_ca_certificate = base64decode(data.aws_eks_cluster.this.certificate_authority[0].data)
    token                  = data.aws_eks_cluster_auth.this.token
  }

  experiments {
    manifest = true
  }
}

Requirements

Name	Version
terraform	>= 1.0
aws	>= 4.0
kubernetes	>= 2.10

Providers

Name	Version
aws	>= 4.0
kubernetes	>= 2.10

Modules

Name	Source	Version
ebs_csi_irsa_role	terraform-aws-modules/iam/aws//modules/iam-role-for-service-accounts-eks	~> 4.21.1
eks	terraform-aws-modules/eks/aws	~> 18.29.0
kms_ebs	SPHTech-Platform/kms/aws	~> 0.1.0
kms_secret	SPHTech-Platform/kms/aws	~> 0.1.0
node_groups	./modules/eks_managed_nodes	n/a
vpc_cni_irsa_role	terraform-aws-modules/iam/aws//modules/iam-role-for-service-accounts-eks	~> 4.21.1

Resources

Name	Type
aws_eks_addon.coredns	resource
aws_eks_addon.ebs_csi	resource
aws_eks_addon.kube_proxy	resource
aws_eks_addon.vpc_cni	resource
aws_iam_role.cluster	resource
aws_iam_role.workers	resource
aws_iam_role_policy.ebs_csi_kms	resource
aws_iam_role_policy_attachment.cluster	resource
aws_iam_role_policy_attachment.workers	resource
aws_iam_service_linked_role.autoscaling	resource
kubernetes_config_map.amazon_vpc_cni	resource
aws_ami.eks_default_bottlerocket	data source
aws_caller_identity.current	data source
aws_iam_policy_document.ec2_assume_role_policy	data source
aws_iam_policy_document.eks_assume_role_policy	data source
aws_iam_policy_document.kms_csi_ebs	data source
aws_iam_policy_document.kms_ebs	data source
aws_partition.current	data source

Inputs

Name	Description	Type	Default	Required
cluster_additional_security_group_ids	List of additional, externally created security group IDs to attach to the cluster control plane	list(string)	[]	no
cluster_enabled_log_types	A list of the desired control plane logs to enable. For more information, see Amazon EKS Control Plane Logging documentation (https://docs.aws.amazon.com/eks/latest/userguide/control-plane-logs.html)	list(string)	[ "audit", "api", "authenticator" ]	no
cluster_endpoint_private_access	Indicates whether or not the Amazon EKS private API server endpoint is enabled	bool	true	no
cluster_endpoint_public_access	Indicates whether or not the Amazon EKS public API server endpoint is enabled	bool	true	no
cluster_endpoint_public_access_cidrs	List of CIDR blocks which can access the Amazon EKS public API server endpoint	list(string)	[ "0.0.0.0/0" ]	no
cluster_iam_boundary	IAM boundary for the cluster IAM role, if any	string	null	no
cluster_iam_role	Cluster IAM Role name. If undefined, is the same as the cluster name	string	""	no
cluster_name	EKS Cluster Name	string	n/a	yes
cluster_security_group_additional_rules	List of additional security group rules to add to the cluster security group created. Set source_node_security_group = true inside rules to set the node_security_group as source	any	{}	no
cluster_service_ipv4_cidr	The CIDR block to assign Kubernetes service IP addresses from. If you don't specify a block, Kubernetes assigns addresses from either the 10.100.0.0/16 or 172.20.0.0/16 CIDR blocks	string	null	no
cluster_version	EKS Cluster Version	string	"1.22"	no
create_aws_auth_configmap	Determines whether to create the aws-auth configmap. NOTE - this is only intended for scenarios where the configmap does not exist (i.e. - when using only self-managed node groups). Most users should use manage_aws_auth_configmap	bool	true	no
default_group_ami_id	The AMI from which to launch the defualt group instance. If not supplied, EKS will use its own default image	string	""	no
default_group_instance_types	Instance type for the default node group	list(string)	[ "m5a.xlarge", "m5.xlarge", "m5n.xlarge", "m5zn.xlarge" ]	no
default_group_launch_template_name	Name of the default node group launch template	string	"default"	no
default_group_max_size	Configuration for max default node group size	number	5	no
default_group_min_size	Configuration for min default node group size	number	1	no
default_group_name	Name of the default node group	string	"default"	no
default_group_node_labels	Additional node label for default group	map(string)	{}	no
default_group_subnet_ids	Subnet IDs to create the default group ASGs in	list(string)	[]	no
default_group_volume_size	Size of the persistentence volume for the default group	number	50	no
eks_managed_node_group_defaults	Map of EKS managed node group default configurations	any	{ "create_iam_role": false, "create_security_group": false, "disk_size": 50, "ebs_optimized": true, "enable_monitoring": true, "metadata_options": { "http_endpoint": "enabled", "http_put_response_hop_limit": 1, "http_tokens": "required", "instance_metadata_tags": "disabled" }, "protect_from_scale_in": false, "update_launch_template_default_version": true }	no
eks_managed_node_groups	Map of EKS managed node group definitions to create	any	{}	no
enable_cluster_windows_support	Determines whether to create the amazon-vpc-cni configmap and windows worker roles into aws-auth.	bool	false	no
force_imdsv2	Force IMDSv2 metadata server.	bool	true	no
force_irsa	Force usage of IAM Roles for Service Account	bool	true	no
iam_role_additional_policies	Additional policies to be added to the IAM role	list(string)	[]	no
manage_aws_auth_configmap	Determines whether to manage the contents of the aws-auth configmap	bool	true	no
node_security_group_additional_rules	List of additional security group rules to add to the node security group created. Set source_cluster_security_group = true inside rules to set the cluster_security_group as source	any	{}	no
only_critical_addons_enabled	Enabling this option will taint default node group with CriticalAddonsOnly=true:NoSchedule taint. Changing this forces a new resource to be created.	bool	false	no
role_mapping	List of IAM roles to give access to the EKS cluster	list(object({ rolearn = string username = string groups = list(string) }))	[]	no
skip_asg_role	Skip creating ASG Service Linked Role if it's already created	bool	false	no
subnet_ids	A list of subnet IDs where the EKS cluster (ENIs) will be provisioned along with the nodes/node groups. Node groups can be deployed within a different set of subnet IDs from within the node group configuration	list(string)	n/a	yes
user_mapping	List of IAM Users to give access to the EKS Cluster	list(object({ userarn = string username = string groups = list(string) }))	[]	no
vpc_id	VPC ID to deploy the cluster into	string	n/a	yes
workers_iam_boundary	IAM boundary for the workers IAM role, if any	string	null	no
workers_iam_role	Workers IAM Role name. If undefined, is the same as the cluster name suffixed with 'workers'	string	""	no

Outputs

Name	Description
cluster_certificate_authority_data	Base64 Encoded Cluster CA Data
cluster_endpoint	Endpoint of the EKS Cluster
cluster_name	EKS Cluster name created
cluster_oidc_issuer_url	The URL on the EKS cluster for the OpenID Connect identity provider
cluster_security_group_id	Security Group ID of the master nodes
ebs_kms_key_arn	KMS Key ARN used for EBS encryption
ebs_kms_key_id	KMS Key ID used for EBS encryption
oidc_provider_arn	OIDC Provider ARN for IRSA
worker_iam_role_arn	IAM Role ARN used by worker nodes
worker_iam_role_name	IAM Role Name used by worker nodes
worker_security_group_id	Security Group ID of the worker nodes