DevOps Stack Modules

The DevOps Stack is separated into multiple Terraform modules, each of them containing a set of related resources.

In order to improve the readability and maintenance of the code, this page contains some guidelines and explanations behind the creation and development of DevOps Stack modules. There is also a repository template on GitHub that can be used as a starting point for new modules and you can refer to it while reading this page.

Basic Modules

These kinds of modules are typically the ones that provision clusters and related resources. Good examples of this are the Amazon EKS module and KinD module.

A basic DevOps Stack module will contain the following files and folders:

devops-stack-module-template
├── .github
│   └── workflows
│       ├── linters.yaml
│       ├── release-please.yaml
│       └── terraform-docs.yaml
├── CHANGELOG.md
├── CODEOWNERS
├── docs
│   └── ...
├── LICENSE
├── locals.tf
├── main.tf
├── outputs.tf
├── README.adoc
├── terraform.tf
├── variables.tf
└── version.txt

Quick overview of each file/folder:

  1. .github - Contains the GitHub Actions workflows that are used to lint the code, generate the documentation and release the module. They are stored on the main repository and each module calls the same workflows.

  2. CHANGELOG.md - Contains the changelog of the module. It is automatically updated by the Release Please GitHub Action and you do not need to create this file manually.

  3. CODEOWNERS - Contains the list of GitHub users that will be automatically assigned as reviewers for pull requests on the module. In our case it is the @camptocamp/is-devops-stack team.

  4. docs - This is a folder that contains a precise substructure needed for the rendering of these documentation pages by Antora. The actual documentation is contained on the README.adoc files. You will find these and some other explanations about the docs on the Documentation page.

  5. LICENSE - The license of the module. In our case it is the Apache 2.0 license.

  6. README.adoc - The documentation of the module. It is written in AsciiDoc and contains the example usage along with some explanations as well as the automatic documentation generated by Terraform Docs.

  7. locals.tf - Contains the definition of the local variables used in the module.

  8. main.tf - Contains the definition of the resources that are created by the module. This can be any type of Terraform resource, depending on the use case.

  9. outputs.tf - Contains the definition of the output variables of the module.

  10. terraform.tf - Contains the versions of the required providers.

The terraform.tf file should only contain the minimum required version of the required providers. This is to avoid incompatibilities between modules and it is the recommended best practices by Terraform.

  1. variables.tf - Contains the definition of the input variables of the module.

  2. version.txt - Contains the version of the module. You should only create it if you are creating a new module, after that it is automatically updated by the Release Please GitHub Action.

Take care to properly describe each entry on the variables.tf and outputs.tf files. These descriptions are taken into account by Terraform Docs for the automatic documentation of the module.

Modules With Embedded Helm Charts

These are the more typical modules of the DevOps Stack and are used to deploy the remaining components of the stack. Good examples of this are the Argo CD module and Cert-manager module.

The Argo CD module is a special case, as it is used to deploy the other modules. A bootstrap Argo CD is deployed using resources of the type helm_release. This Argo CD is then responsible to deploy the remaining modules, which use resources of the type argocd_project and argocd_application.

A typical file/folder structure for a module with embedded Helm charts is the following:

devops-stack-module-template
├── .github
│   └── ...
├── CHANGELOG.md
├── charts
│   └── CHART_NAME
│       ├── Chart.lock
│       ├── charts
│       │   └── CHART_NAME.tar.gz
│       ├── Chart.yaml
│       ├── templates
│       │   └── RESOURCE.yaml
│       └── values.yaml
├── CODEOWNERS
├── docs
│   └── ...
├── LICENSE
├── locals.tf
├── main.tf
├── outputs.tf
├── README.adoc
├── terraform.tf
├── variables.tf
└── version.txt

Comparatively to a more basic module, note the following changes (all the other files are the same and are described above):

  1. charts - Contains the Helm chart(s) deployed by the module, if any. The chart itself refers to the chart that we really want to deploy as a dependency, which should be locate in the charts/CHART_NAME/charts folder. The chart package is simply downloaded manually using a helm dependency update and uploaded to the repository along with the rest of the code.

  2. locals.tf - Contains the definition of the local variables used in the module. It is here that we define the helm_values local that contains the default values for the Helm chart, as needed by the module. These should be written in HCL and not in YAML.

  3. main.tf - Contains the definition of the resources that are created by the module. It is here that we define the argocd_project and argocd_application resources that deploy the Helm chart.

Modules With Variants

Some modules have multiple variants. While the core module is the same, the variants deploy different resources or customize the Helm values in order to cater to a specific use case or a different platform. A good example is the Thanos module, which has variants for EKS, AKS and KinD.

These kinds of modules should be called from within their variant. The variant then recursively calls the root module ir order to apply its core resources.

A typical file/folder structure for a module with variants is the following:

devops-stack-module-template
├── aks
│   ├── extra-variables.tf
│   ├── locals.tf
│   ├── main.tf
│   ├── outputs.tf
│   ├── README.adoc
│   ├── variables.tf -> ../variables.tf
│   └── terraform.tf -> ../terraform.tf
├── CHANGELOG.md
├── charts
│   └── ...
├── CODEOWNERS
├── docs
│   └── ...
├── eks
│   ├── extra-variables.tf
│   ├── locals.tf
│   ├── main.tf
│   ├── outputs.tf
│   ├── README.adoc
│   ├── variables.tf -> ../variables.tf
│   └── terraform.tf -> ../terraform.tf
├── .github
│   └── ...
├── kind
│   ├── extra-variables.tf
│   ├── locals.tf
│   ├── main.tf
│   ├── outputs.tf
│   ├── README.adoc
│   ├── variables.tf -> ../variables.tf
│   └── terraform.tf -> ../terraform.tf
├── LICENSE
├── locals.tf
├── main.tf
├── outputs.tf
├── README.adoc
├── variables.tf
├── terraform.tf
└── version.txt
Note how the variables.tf and terraform.tf files are symbolic links to the root module. This is to avoid having to maintain the same variables and providers in multiple places.

Comparatively to a more basic module, note the following files inside the variants (all the other files are the same and are described above):

  1. extra-variables.tf - Contains the definition of the extra input variables of the variant. These are the variables that are specific to the variant and are not present in the root module.

  2. locals.tf - Contains the definition of the local variables used in the variant. It is here that we define the helm_values local that contains only the values specific to the variant. These should be written in HCL and not in YAML. They will be merged with the ones coming from the helm_values variable and then passed on to the root module. Afterwards, they will be merged once again, translated to YAML and then passed to the argocd_application resource.

  3. main.tf - Usually, this file only contains the call to the root module and passes along all the variables received as well as the modified entries. In specific cases it could also contain other resources specific to the variant. Take a look at this example from the Loki module:

module "loki-stack" {
  source = "../"

  cluster_name     = var.cluster_name
  base_domain      = var.base_domain
  argocd_namespace = var.argocd_namespace
  target_revision  = var.target_revision
  namespace        = var.namespace
  app_autosync     = var.app_autosync
  dependency_ids   = var.dependency_ids

  distributed_mode = var.distributed_mode
  ingress          = var.ingress
  enable_filebeat  = var.enable_filebeat

  sensitive_values = merge({}, var.sensitive_values)

  helm_values = concat(local.helm_values, var.helm_values)
}

  1. outputs.tf - Contains the definition of the output variables of the variant. At the very least, it should contain the the same outputs present in the root module, in order to propagate them out. In addition, it can contain other outputs specific to the variant. See this example from the Loki module (note the id output, which only propagates the id output of the root module):

output "id" {
  description = "..."
  value       = module.loki-stack.id
}

output "loki_credentials" {
  description = "..."
  value       = module.loki-stack.loki_credentials
  sensitive   = true
}

  1. README.adoc - Contains the documentation for the variant. More explanations on the Documentation page.

  2. variables.tf and terraform.tf - These files are symbolic links to the root module.

Documentation

The specific documentation for each modules is located in its README.adoc file. If a module contains a variant (e.g. eks or aks), the documentation should be split into multiple files, one per variant. See the Documentation page for more information.

Release

Each module is released and versioned separately. We use Semantic Versioning for versioning the modules. The release process is described in more detail in the Release page.