devops-stack-module-cluster-eks

A DevOps Stack module to deploy a Kubernetes cluster on Amazon Web Services EKS.

This module creates EKS managed nodes by default, bug also supports self-managed node groups by setting the variable use_self_managed_node_groups to true. It uses the Terraform module "eks" by AWS to manage the cluster itself.

The node_groups variable is a map of objects. Each value of the map creates a node group referenced by its label. The value object can take any input accepted by the eks-managed-node-group or self-managed-node-group submodule.

Here is an example that creates one EKS managed node group with 3 instances of type r5a.large, attaches them to the cluster Load Balancer(s), and customizes the base EBS volume size to 100 GB:

  node_groups = {
    "my-node-group" = {
      instance_types  = ["r5a.xlarge"]
      min_size        = 3
      max_size        = 3
      desired_size    = 3
      nlbs_attachment = true

      block_device_mappings = {
        "default" = {
          device_name = "/dev/xvda"
          ebs = {
            volume_size = 100
          }
        }
      }
    },
  }

When using self-managed node groups, the instance type is instead given as a single string value instead of an array:

      instance_type     = "r5a.xlarge"

You can check out available instance types on the official AWS documentation pages.

Depending on the create_public_nlb and create_private_nlb variables, it creates a public and/or public Network Load Balancer(s). The node groups that have the target_group_arns = module.my_cluster.nlb_target_groups value set will be added as backends of the LBs. By default, the LBs only forward traffic on the ports 80 and 443, but you can customize this using the extra_lb_target_groups and extra_lb_http_tcp_listeners variables to add other ports. Look at the lb.tf file in this module for the syntax.

This module needs a Route 53 DNS zone in the same AWS account to create a wildcard CNAME record that points to the Network LB. The DNS zone must be passed in the base_domain variable. This record is used by other DevOps Stack modules as default URLs for their applications.

Upgrading from versions earlier than 3.x

The versions 2.x and earlier of this module created self-managed node groups exclusively. Upgrading to versions 3.x or later requires setting the variable use_self_managed_node_groups to true to avoid breaking existing clusters.

When going from self-managed node groups to EKS managed ones, note the following changes in node groups arguments:

The instance_type argument is replaced with instance_types and accepts a list of strings,
The target_group_arns argument isn’t supported anymore. Instead set nlbs_attachment = true.

Switching an existing cluster to EKS managed node groups is possible, but requires manually draining the existing nodes after the new node group is created and before the old one is destroyed (by doing a terraform apply -target <resource> in multiple steps).

Technical Reference

Requirements

The following requirements are needed by this module:

terraform (>= 1.0)
aws (>= 4)

Providers

The following providers are used by this module:

dns
aws (>= 4)

Modules

The following Modules are called:

cluster

Source: terraform-aws-modules/eks/aws

Version: ~> 20.0

nlb

Source: terraform-aws-modules/alb/aws

Version: ~> 8.0

nlb_private

Source: terraform-aws-modules/alb/aws

Version: ~> 8.0

Resources

The following resources are used by this module:

aws_autoscaling_attachment.node_groups_to_nlbs_target_groups (resource)
aws_route53_record.wildcard (resource)
aws_eks_cluster_auth.cluster (data source)
aws_region.current (data source)
aws_route53_zone.this (data source)
dns_a_record_set.nlb (data source)
dns_a_record_set.nlb_private (data source)

Required Inputs

The following input variables are required:

cluster_name

Description: Name of the EKS cluster. Must be unique in the AWS account.

Type: string

vpc_id

Description: ID of the VPC where the cluster and nodes will be deployed.

Type: string

private_subnet_ids

Description: List of IDs of private subnets that the EKS instances will be attached to.

Type: list(string)

Optional Inputs

The following input variables are optional (have default values):

base_domain

Description: The base domain for the cluster.

This module needs a Route 53 zone matching this variable with permission to create DNS records. It will create a wildcard CNAME record *.<subdomain>.<base_domain> that points to an Elastic Load Balancer routing ingress traffic to all cluster nodes. Such urls will be used by default by other DevOps Stack modules for the applications they deploy (e.g. Argo CD, Prometheus, etc).

Type: string

Default: null

subdomain

Description: The subdomain used for ingresses.

Type: string

Default: "apps"

kubernetes_version

Description: Kubernetes <major>.<minor> version to use for the EKS cluster.

See AWS EKS documentation for a list of available versions.

If you do not specify a value, the latest available version at creation is used and no upgrades will occur except those automatically triggered by EKS.

The value can be set and increased on an existing cluster to upgrade it. Note that this triggers a rolling replacement of the compute nodes, so all pods will be recreated.

Type: string

Default: null

cluster_endpoint_public_access_cidrs

Description: List of CIDR blocks which can access the Amazon EKS public API server endpoint.

Type: list(string)

Default:

[
  "0.0.0.0/0"
]

public_subnet_ids

Description: List of IDs of public subnets the public NLB will be attached to if enabled with 'create_public_nlb'.

Type: list(string)

Default: []

node_groups

Description: A map of node group configurations to be created.

Type: any

Default: {}

use_self_managed_node_groups

Description: Whether to use self-managed node groups instead of EKS managed node groups.

EKS managed node groups have the advantage of automatically draining the nodes when instances are being replaced.

You should set this variable to true on clusters deployed with a module earlier than v3 because it created self-managed node groups exclusively.

Changing this on an existing cluster is not supported (although it is possible with some downtime and manual fixes to the load balancer target groups).

Type: bool

Default: false

create_public_nlb

Description: Whether to create an internet-facing NLB attached to the public subnets

Type: bool

Default: true

create_private_nlb

Description: Whether to create an internal NLB attached the private subnets

Type: bool

Default: false

nlb_attached_node_groups

Description: List of node_groups indexes that the NLB(s) should be attached to

Type: list(any)

Default: []

extra_lb_target_groups

Description: Additional Target Groups to attach to Network LBs.

A list of maps containing key/value pairs that define the target groups. Required key/values: name, backend_protocol, backend_port.

Type: list(any)

Default: []

extra_lb_http_tcp_listeners

Description: Additional Listeners to attach to Network LBs.

A list of maps describing the HTTP listeners. Required key/values: port, protocol. Optional key/values: target_group_index (defaults to http_tcp_listeners[count.index]).

Type: list(any)

Default: []

Outputs

The following outputs are exported:

cluster_name

Description: Name of the EKS cluster.

cluster_arn

Description: The Amazon Resource Name (ARN) of the cluster

base_domain

Description: The base domain for the cluster.

cluster_oidc_issuer_url

Description: The URL on the EKS cluster for the OpenID Connect identity provider

node_security_group_id

Description: ID of the node shared security group

node_groups

Description: Map of attribute maps for all node groups created.

kubernetes_host

Description: Endpoint for your Kubernetes API server.

kubernetes_cluster_ca_certificate

Description: Certificate data required to communicate with the cluster.

kubernetes_token

Description: Token to use to authenticate with the cluster.

nlb_dns_name

Description: Map of the DNS names of the load balancers (public and/or private if enabled). Returns null if the respective load balancer is disabled.

nlb_zone_id

Description: Map of the zone_id of the load balancer to assist with creating DNS records (public and/or private if enabled). Returns null if the respective load balancer is disabled.

nlb_target_groups

Description: List of the target groups ARNs (public and/or private if enabled).

kubernetes

Description: Kubernetes API endpoint and CA certificate as a structured value.

Reference in table format

Show tables

= Requirements

Name	Version
terraform	>= 1.0
aws	>= 4

Name

Version

terraform

>= 1.0

aws

>= 4

= Providers

Name	Version
aws	>= 4
dns	n/a

Name

Version

aws

>= 4

dns

n/a

= Modules

Name	Source	Version
nlb	terraform-aws-modules/alb/aws	~> 8.0
nlb_private	terraform-aws-modules/alb/aws	~> 8.0
cluster	terraform-aws-modules/eks/aws	~> 20.0

Name

Source

Version

nlb

terraform-aws-modules/alb/aws

~> 8.0

nlb_private

terraform-aws-modules/alb/aws

~> 8.0

cluster

terraform-aws-modules/eks/aws

~> 20.0

= Resources

Name	Type
aws_autoscaling_attachment.node_groups_to_nlbs_target_groups	resource
aws_route53_record.wildcard	resource
aws_eks_cluster_auth.cluster	data source
aws_region.current	data source
aws_route53_zone.this	data source
dns_a_record_set.nlb	data source
dns_a_record_set.nlb_private	data source

Name

Type

aws_autoscaling_attachment.node_groups_to_nlbs_target_groups

resource

aws_route53_record.wildcard

resource

aws_eks_cluster_auth.cluster

data source

aws_region.current

data source

aws_route53_zone.this

data source

dns_a_record_set.nlb

data source

dns_a_record_set.nlb_private

data source

= Inputs

Name Description Type Default Required

Name	Description	Type	Default	Required
cluster_name	Name of the EKS cluster. Must be unique in the AWS account.	`string`	n/a	yes
base_domain	The base domain for the cluster. This module needs a Route 53 zone matching this variable with permission to create DNS records. It will create a wildcard CNAME record `*.<subdomain>.<base_domain>` that points to an Elastic Load Balancer routing ingress traffic to all cluster nodes. Such urls will be used by default by other DevOps Stack modules for the applications they deploy (e.g. Argo CD, Prometheus, etc).	`string`	`null`	no
subdomain	The subdomain used for ingresses.	`string`	`"apps"`	no
kubernetes_version	Kubernetes `<major>.<minor>` version to use for the EKS cluster. See AWS EKS documentation for a list of available versions. If you do not specify a value, the latest available version at creation is used and no upgrades will occur except those automatically triggered by EKS. The value can be set and increased on an existing cluster to upgrade it. Note that this triggers a rolling replacement of the compute nodes, so all pods will be recreated.	`string`	`null`	no
cluster_endpoint_public_access_cidrs	List of CIDR blocks which can access the Amazon EKS public API server endpoint.	`list(string)`	`[ "0.0.0.0/0" ]`	no
vpc_id	ID of the VPC where the cluster and nodes will be deployed.	`string`	n/a	yes
private_subnet_ids	List of IDs of private subnets that the EKS instances will be attached to.	`list(string)`	n/a	yes
public_subnet_ids	List of IDs of public subnets the public NLB will be attached to if enabled with 'create_public_nlb'.	`list(string)`	`[]`	no
node_groups	A map of node group configurations to be created.	`any`	`{}`	no
use_self_managed_node_groups	Whether to use self-managed node groups instead of EKS managed node groups. EKS managed node groups have the advantage of automatically draining the nodes when instances are being replaced. You should set this variable to `true` on clusters deployed with a module earlier than v3 because it created self-managed node groups exclusively. Changing this on an existing cluster is not supported (although it is possible with some downtime and manual fixes to the load balancer target groups).	`bool`	`false`	no
create_public_nlb	Whether to create an internet-facing NLB attached to the public subnets	`bool`	`true`	no
create_private_nlb	Whether to create an internal NLB attached the private subnets	`bool`	`false`	no
nlb_attached_node_groups	List of node_groups indexes that the NLB(s) should be attached to	`list(any)`	`[]`	no
extra_lb_target_groups	Additional Target Groups to attach to Network LBs. A list of maps containing key/value pairs that define the target groups. Required key/values: `name`, `backend_protocol`, `backend_port`.	`list(any)`	`[]`	no
extra_lb_http_tcp_listeners	Additional Listeners to attach to Network LBs. A list of maps describing the HTTP listeners. Required key/values: `port`, `protocol`. Optional key/values: `target_group_index` (defaults to `http_tcp_listeners[count.index]`).	`list(any)`	`[]`	no
tags	Tags to apply to all resources created by the EKS Terraform module.	`map(string)`	`{}`	no

cluster_name

Name of the EKS cluster. Must be unique in the AWS account.

string

n/a

yes

base_domain

The base domain for the cluster.

string

null

subdomain

The subdomain used for ingresses.

string

"apps"

kubernetes_version

Kubernetes <major>.<minor> version to use for the EKS cluster.

See AWS EKS documentation for a list of available versions.

If you do not specify a value, the latest available version at creation is used and no upgrades will occur except those automatically triggered by EKS.

The value can be set and increased on an existing cluster to upgrade it. Note that this triggers a rolling replacement of the compute nodes, so all pods will be recreated.

string

null

cluster_endpoint_public_access_cidrs

List of CIDR blocks which can access the Amazon EKS public API server endpoint.

list(string)

[
  "0.0.0.0/0"
]

vpc_id

ID of the VPC where the cluster and nodes will be deployed.

string

n/a

yes

private_subnet_ids

List of IDs of private subnets that the EKS instances will be attached to.

list(string)

n/a

yes

public_subnet_ids

List of IDs of public subnets the public NLB will be attached to if enabled with 'create_public_nlb'.

list(string)

[]

node_groups

A map of node group configurations to be created.

any

{}

use_self_managed_node_groups

Whether to use self-managed node groups instead of EKS managed node groups.

EKS managed node groups have the advantage of automatically draining the nodes when instances are being replaced.

You should set this variable to true on clusters deployed with a module earlier than v3 because it created self-managed node groups exclusively.

Changing this on an existing cluster is not supported (although it is possible with some downtime and manual fixes to the load balancer target groups).

bool

false

create_public_nlb

Whether to create an internet-facing NLB attached to the public subnets

bool

true

create_private_nlb

Whether to create an internal NLB attached the private subnets

bool

false

nlb_attached_node_groups

List of node_groups indexes that the NLB(s) should be attached to

list(any)

[]

extra_lb_target_groups

Additional Target Groups to attach to Network LBs.

A list of maps containing key/value pairs that define the target groups. Required key/values: name, backend_protocol, backend_port.

list(any)

[]

extra_lb_http_tcp_listeners

Additional Listeners to attach to Network LBs.

A list of maps describing the HTTP listeners. Required key/values: port, protocol. Optional key/values: target_group_index (defaults to http_tcp_listeners[count.index]).

list(any)

[]