Deploying with pipelines

This document describes how to set up a CI/CD environment using Devops Stack and GitLab/GitHub pipelines.

Prerequisites

  • Access to API keys allowing to create required resources cloud environment,

  • Access to GitLab or GitHub (only supported CI/CD for now),

  • Knowledge of Terraform basics

Standard Workflow

When deploying with pipelines, the DevOps Stack runs a dry-run on Merge Request and applies the modification on Commit on a Protected Branch. Depending on the nature of the pipeline trigger we will find the following behaviors:

  • If pipeline is being triggered by a Merge Request then it will run a terraform plan, this will provide user with the output of the planned changes in the infrastructure. This will not touch your infra-structure.

  • If pipeline is being triggered by a Commit on a Protected Branch (e.g. by an approved merge request into your main branch) then it will run a self-approved terraform apply. This will update your infrastructure according to the the terraform plan.

It is highly recommended to prohibit (at an admin level) committing changes directly to your protected branch, hence forcing developers to review the terraform plan in the MR before applying any changes to the infrastructure.

Terraform Resource Migration Workflow

This only applies to GiLab Pipelines

The GitLab pipeline integrates [tfmigrate](https://github.com/minamijoyo/tfmigrate). It can handle automatically terraform state mv for renaming terraform resources when using remote state.

Why do we need this? Please see [official documentation](https://github.com/minamijoyo/tfmigrate#why)

The workflow for moving terraform state is the following:

  1. Push your tf renamed resources and create a merge request (MR). Make sure introduced changes do not introduce a new tf resource (for every planned ADD there should be a DELETE and vice-versa). Other type of planned changes will end up on a pipeline failure.

  2. Check pipeline output (terraform plan output), see planned ADD and DELETE resources of MR, verify previous statement, else remove whatever extra bit you added, and go back to step 1.

  3. Create tfmigrate.hcl file and describe those operations, each MOVE operation should correspond to a ADD and DELETE planned operation.

    For example:
    ```hcl
     migration "state" "test" {
         actions = [
             "mv module.my_old_module.azurerm_key_vault.vault_test module.my_new_module.azurerm_key_vault.this",
         ]
     }
     ```
  4. Commit and push file into MR, check pipeline (tfmigrate plan output).

  5. Merge MR (this corresponds to a tfmigrate apply, equivalent to moving the states).

  6. Delete tfmigrate.hcl file, push and merge.

  7. Proceed with other tf config modifications.