devops-stack-module-argocd

A DevOps Stack module to deploy and configure Argo CD.

The Argo CD chart used by this module is shipped in this repository as well, in order to avoid any unwanted behaviors caused by unsupported versions.

Current Chart Version Original Repository Default Values

Current Chart Version	Original Repository	Default Values
5.27.1	Chart	`values.yaml`

5.27.1

Chart

values.yaml

Usage

The root of this repository contains the final Argo CD module to be deployed, which uses a Terraform provider for Argo CD to deploy the Argo CD chart. On the first deployment of a cluster, you’ll want to use the bootstrap module instead. Check the bootstrap module’s documentation for more information.

To deploy the final Argo CD module, you’ll need to add the following declaration on your Terraform configuration:

module "argocd" {
  source = "git::https://github.com/camptocamp/devops-stack-module-argocd.git?ref=<RELEASE>"

  cluster_name   = local.cluster_name
  base_domain    = local.base_domain
  cluster_issuer = local.cluster_issuer

  admin_enabled            = "true"
  namespace                = module.argocd_bootstrap.argocd_namespace
  accounts_pipeline_tokens = module.argocd_bootstrap.argocd_accounts_pipeline_tokens
  server_secretkey         = module.argocd_bootstrap.argocd_server_secretkey

  dependency_ids = {
    argocd                = module.argocd_bootstrap.id
    traefik               = module.traefik.id
    cert-manager          = module.cert-manager.id
    oidc                  = module.oidc.id
    kube-prometheus-stack = module.kube-prometheus-stack.id
  }
}

A more complex declaration, that includes a OIDC configuration for the login (this way you avoid having to login using the admin password found on a Kubernetes secret) and the configuration of some other source repositories, would be:

module "argocd" {
  source = "git::https://github.com/camptocamp/devops-stack-module-argocd.git?ref=<RELEASE>"

  ...

  oidc = {
    name         = "OIDC"
    issuer       = module.oidc.oidc.issuer_url
    clientID     = module.oidc.oidc.client_id
    clientSecret = module.oidc.oidc.client_secret
    requestedIDTokenClaims = {
      groups = {
        essential = true
      }
    }
    requestedScopes = [
      "openid", "profile", "email"
    ]
  }

  repositories = {
    cluster-apps = {
      ...
    }
    user-apps = {
      ...
    }
  }

  ...
}

The sources can also be configured using the Application and ApplicationSet modules.

You can also overload the policy.csv as shown in the following example:

module "argocd" {
  source = "git::https://github.com/camptocamp/devops-stack-module-argocd.git?ref=<RELEASE>"

  ...

  rbac = {
    policy_csv = <<-EOT
      g, pipeline, role:admin
      g, argocd-admin, role:admin
      p, role:read-sync, applications, get, */*, allow
      p, role:read-sync, applications, sync, */*, allow
      p, role:read-sync, repositories, get, *, allow
      p, role:read-sync, projects, get, *, allow
    EOT
  }

  ...
}

By default, this module has a policy.csv that is configured to give administrator access to any user belonging to the groups argocd-admin or devops-stack-admins.

Custom Plugins

This module installs two custom plugins:

"kustomized-helm" is just a combination of Kustomize and Helm that allows using Helm charts and then applying Kustomize overrides to the result.
"helmfile-sops" adds support for Helmfile managed applications, and integrates the support for SOPS. This is a custom plugin developed by Camptocamp (source code available here).

When using SOPS, you will want to give it access to an external secrets management or encryption service. For this, you will want to pass an AWS IAM role, and Azure Workload Identity Client-ID, or an Azure AAD Pod Identity using the variables repo_server_iam_role_arn, repo_server_azure_workload_identity_clientid`, or repo_server_aadpodidbinding.

Troubleshooting

`connection_error` during `terraform apply`

When deploying this module of Argo CD you may experience connection errors (which is normal given that argocd-server pod could have been redeployed).

│ Error: Error while waiting for application argocd to be created
│
│   with module.argocd.argocd_application.this,
│   on .terraform/modules/argocd/main.tf line 55, in resource "argocd_application" "this":
│   55: resource "argocd_application" "this" {
│
│ error while waiting for application argocd to be synced and healthy: rpc error: code = Unavailable desc = connection error: desc = "transport: error while dialing: dial tcp 127.0.0.1:44461: connect:
│ connection refused"

When bootstrapping a cluster for the first time, you can simply run terraform apply again and the deployment should finish correctly.

There are some settings that force the automatic redeployment of the argocd-server pod during the first bootstrap of the cluster.

A good example is the admin_enabled variable which is set as true for the bootstrap but not for the final Argo CD module. If you pass the admin_enabled = true to the final Argo CD module you will most likely not encounter this error on the first deployment of you cluster, however you could run into the error of the interface looping whenever you click login (see the other troubleshoot sections).

However, on some cases (notably when upgrading the Argo CD module), this error could leave the Terraform resource tainted.

terraform plan
  # module.devops_stack_blue.module.argocd.argocd_application.this is tainted, so must be replaced
-/+ resource "argocd_application" "this" {
      ~ id      = "argocd:argocd" -> (known after apply)
        # (2 unchanged attributes hidden)

      ~ metadata {
        ...
        }

      ~ spec {
        ...
        }
    }

Plan: 1 to add, 1 to change, 1 to destroy.

Untainting the resource with the command terraform untaint module.argocd.argocd_application.this should solve the issue.

Argo CD interface reload loop when clicking on login

If you encounter a loop when clicking on the login button on the Argo CD interface, you can try to delete the Argo CD server pod and let it be recreated.

This error usually happens because there are some unapplied settings between the bootstrap Argo CD and the final Argo CD module. If you already encountered the error from the previous step, then this error should not happen, as it means the pod was already recreated.

Technical Documentation

Dependencies

`module.argocd_bootstrap.id`

Obviously, this module needs an already working Argo CD (the bootstrap), so it depends on module.argocd_bootstrap.

`module.traefik.id`

Since there is an ingress deployed with this module, it needs to be deployed after Traefik so it depends on module.ingress.

`module.traefik.id`

For the same reason as the previous dependency, it needs to be deployed after cert-manager so it depends on module.cert-manager.

`module.oidc.id`

Only for the platforms that deploy a OIDC module, such as EKS, KinD or SKS, there is also a the dependency on module.oidc.id.

`module.kube-prometheus-stack.id`

Finally, the kube-prometheus-stack is a requirement because this Argo CD module requires the ServiceMonitor CRD so it depends on module.kube-prometheus-stack.

Requirements

The following requirements are needed by this module:

terraform (>= 1.2)
argocd (>= 5)
htpasswd (>= 1)
jwt (>= 1.1)
null (>= 3)
random (>= 3)
time (>= 0.9)
utils (>= 1.6)

Providers

The following providers are used by this module:

null (>= 3)
jwt (>= 1.1)
time (>= 0.9)
random (>= 3)
utils (>= 1.6)
argocd (>= 5)

Resources

The following resources are used by this module:

argocd_application.this (resource)
argocd_project.this (resource)
jwt_hashed_token.tokens (resource)
null_resource.dependencies (resource)
null_resource.this (resource)
random_uuid.jti (resource)
time_static.iat (resource)
utils_deep_merge_yaml.values (data source)

Required Inputs

The following input variables are required:

cluster_name

Description: Name given to the cluster. Value used for the ingress' URL of the application.

Type: string

base_domain

Description: Base domain of the cluster. Value used for the ingress' URL of the application.

Type: string

accounts_pipeline_tokens

Description: API token for pipeline account.

Type: string

server_secretkey

Description: Signature key for session validation. Must reuse the bootstrap output containing the secretkey.

Type: string

Optional Inputs

The following input variables are optional (have default values):

subdomain

Description: Subdomain of the cluster. Value used for the ingress' URL of the application.

Type: string

Default: "apps"

argocd_project

Description: Name of the Argo CD AppProject where the Application should be created. If not set, the Application will be created in a new AppProject only for this Application.

Type: string

Default: null

argocd_labels

Description: Labels to attach to the Argo CD Application resource.

Type: map(string)

Default: {}

target_revision

Description: Override of target revision of the application chart.

Type: string

Default: "v4.0.0"

cluster_issuer

Description: SSL certificate issuer to use. Usually you would configure this value as letsencrypt-staging or letsencrypt-prod on your root *.tf files. You can use ca-issuer when using the self-signed variant of cert-manager.

Type: string

Default: "selfsigned-issuer"

helm_values

Description: Helm chart value overrides. They should be passed as a list of HCL structures.

Type: any

Default: []

app_autosync

Description: Automated sync options for the Argo CD Application resource.

Type:

object({
    allow_empty = optional(bool)
    prune       = optional(bool)
    self_heal   = optional(bool)
  })

Default:

{
  "allow_empty": false,
  "prune": true,
  "self_heal": true
}

dependency_ids

Description: n/a

Type: map(string)

Default: {}

resources

Description: Resource limits and requests for the Argo CD components. Follow the style on official documentation to understand the format of the values.

The repo_server requests and limits will be applied to all the extra containers that are deployed with the argocd-repo-server component (each container has the same requests and limits as the main container, so it is cumulative).

If you enable the HA mode using the high_availability variable, the values for Redis will be applied to the Redis HA chart instead of the default one.

Type:

object({

    application_set = optional(object({
      requests = optional(object({
        cpu    = optional(string, "100m")
        memory = optional(string, "128Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string, "100m")
        memory = optional(string, "128Mi")
      }), {})
    }), {})

    controller = optional(object({
      requests = optional(object({
        cpu    = optional(string, "500m")
        memory = optional(string, "512Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string, "1")
        memory = optional(string, "2Gi")
      }), {})
    }), {})

    notifications = optional(object({
      requests = optional(object({
        cpu    = optional(string, "100m")
        memory = optional(string, "128Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string, "200m")
        memory = optional(string, "256Mi")
      }), {})
    }), {})

    repo_server = optional(object({
      requests = optional(object({
        cpu    = optional(string, "200m")
        memory = optional(string, "128Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string, "400m")
        memory = optional(string, "256Mi")
      }), {})
    }), {})

    server = optional(object({
      requests = optional(object({
        cpu    = optional(string, "50m")
        memory = optional(string, "128Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string, "100m")
        memory = optional(string, "256Mi")
      }), {})
    }), {})

    redis = optional(object({
      requests = optional(object({
        cpu    = optional(string, "200m")
        memory = optional(string, "64Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string, "300m")
        memory = optional(string, "128Mi")
      }), {})
    }), {})

  })

Default: {}

high_availability

Description: Argo CD High Availability settings. By default, the HA is disabled.

To enable HA using the default replicas, simply set the value high_availability.enabled to true. This will deploy Argo CD in HA without autoscaling.

You can enable autoscaling of the argocd-server and argocd-repo-server components by setting the high_availability.server.autoscaling.enabled and high_availability.repo_server.autoscaling.enabled values to true. You can also configure the minimum and maximum replicas desired or leave the default values.

Activating the HA mode automatically enables the Redis HA chart which requires at least 3 worker nodes, as this chart enforces Pods to run on separate nodes.

Since this variable uses the optional argument to forcing the user to define all the values, there is a side effect you can pass any other bogus value and Terraform will accept it, but they won’t be used in the chart behind the module.

Type:

object({
    enabled = bool

    controller = optional(object({
      replicas = optional(number, 1)
    }), {})

    application_set = optional(object({
      replicas = optional(number, 2)
    }), {})

    server = optional(object({
      replicas = optional(number, 2)
      autoscaling = optional(object({
        enabled      = bool
        min_replicas = optional(number, 2)
        max_replicas = optional(number, 5)
        }), {
        enabled = false
      })
    }), {})

    repo_server = optional(object({
      replicas = optional(number, 2)
      autoscaling = optional(object({
        enabled      = bool
        min_replicas = optional(number, 2)
        max_replicas = optional(number, 5)
        }), {
        enabled = false
      })
    }), {})

  })

Default:

{
  "enabled": false
}

oidc

Description: OIDC settings for the log in to the Argo CD web interface.

Type: any

Default: null

rbac

Description: RBAC settings for the Argo CD users.

Type:

object({
    scopes         = optional(string, "[groups, cognito:groups, roles]")
    policy_default = optional(string, "")
    policy_csv = optional(string, <<-EOT
                                    g, pipeline, role:admin
                                    g, argocd-admin, role:admin
                                    g, devops-stack-admins, role:admin
                                  EOT
    )
  })

Default: {}

repositories

Description: List of repositories to add to Argo CD.

Type: map(map(string))

Default: {}

ssh_known_hosts

Description: List of SSH known hosts to add to Argo CD.

Check the official values.yaml to get the format to pass this value.

If you set this variable, the default known hosts will be overridden by this value, so you might want to consider adding the ones you need here."

Type: string

Default: null

exec_enabled

Description: Flag to enable the web-based terminal on Argo CD. Do not forget to set the appropriate RBAC configuration to your users/groups.

Type: bool

Default: false

admin_enabled

Description: Flag to indicate whether to enable the administrator user.

Type: bool

Default: false

extra_accounts

Description: List of accounts for which tokens will be generated.

Type: list(string)

Default: []

repo_server_iam_role_arn

Description: IAM role ARN to associate with the argocd-repo-server ServiceAccount. This role can be used to give SOPS access to AWS KMS.

Type: string

Default: null

repo_server_azure_workload_identity_clientid

Description: Azure AD Workload Identity Client-ID to associate with argocd-repo-server. This role can be used to give SOPS access to a Key Vault.

Type: string

Default: null

repo_server_aadpodidbinding

Description: Azure AAD Pod Identity to associate with the argocd-repo-server Pod. This role can be used to give SOPS access to a Key Vault.

Type: string

Default: null

helmfile_cmp_version

Description: Version of the helmfile-cmp plugin.

Type: string

Default: "0.1.1"

helmfile_cmp_env_variables

Description: List of environment variables to attach to the helmfile-cmp plugin, usually used to pass authentication credentials. Use an explicit format or take the values from a Kubernetes secret.

Type:

list(object({
    name  = optional(string)
    value = optional(string)
    valueFrom = optional(object({
      secretKeyRef = optional(object({
        name = optional(string)
        key  = optional(string)
      }))
    }))
  }))

Default: []

Outputs

The following outputs are exported:

id

Description: ID to pass other modules in order to refer to this module as a dependency.

extra_tokens

Description: Map of extra accounts that were created and their tokens.

Reference in table format

Show tables

= Requirements

Name	Version
terraform	>= 1.2
argocd	>= 5
htpasswd	>= 1
jwt	>= 1.1
null	>= 3
random	>= 3
time	>= 0.9
utils	>= 1.6

Name

Version

>= 1.2

>= 5

>= 1

>= 1.1

>= 3

>= 3

>= 0.9

>= 1.6

= Providers

Name	Version
null	>= 3
jwt	>= 1.1
time	>= 0.9
random	>= 3
utils	>= 1.6
argocd	>= 5

Name

Version

>= 3

>= 1.1

>= 0.9

>= 3

>= 1.6

>= 5

= Resources

Name	Type
argocd_application.this	resource
argocd_project.this	resource
jwt_hashed_token.tokens	resource
null_resource.dependencies	resource
null_resource.this	resource
random_uuid.jti	resource
time_static.iat	resource
utils_deep_merge_yaml.values	data source

Name

Type

argocd_application.this

resource

argocd_project.this

resource

jwt_hashed_token.tokens

resource

null_resource.dependencies

resource

null_resource.this

resource

random_uuid.jti

resource

time_static.iat

resource

utils_deep_merge_yaml.values

data source

= Inputs

Name Description Type Default Required

cluster_name

Name given to the cluster. Value used for the ingress' URL of the application.

string

n/a

yes

base_domain

Base domain of the cluster. Value used for the ingress' URL of the application.

string

n/a

yes

subdomain

Subdomain of the cluster. Value used for the ingress' URL of the application.

string

"apps"

argocd_project

Name of the Argo CD AppProject where the Application should be created. If not set, the Application will be created in a new AppProject only for this Application.

string

null

argocd_labels

Labels to attach to the Argo CD Application resource.

map(string)

{}

target_revision

Override of target revision of the application chart.

string

"v4.0.0"

cluster_issuer

SSL certificate issuer to use. Usually you would configure this value as letsencrypt-staging or letsencrypt-prod on your root *.tf files. You can use ca-issuer when using the self-signed variant of cert-manager.

string

"selfsigned-issuer"

helm_values

Helm chart value overrides. They should be passed as a list of HCL structures.

any

[]

app_autosync

Automated sync options for the Argo CD Application resource.

object({
    allow_empty = optional(bool)
    prune       = optional(bool)
    self_heal   = optional(bool)
  })

{
  "allow_empty": false,
  "prune": true,
  "self_heal": true
}

dependency_ids

n/a

map(string)

{}

resources

Resource limits and requests for the Argo CD components. Follow the style on official documentation to understand the format of the values.

If you enable the HA mode using the high_availability variable, the values for Redis will be applied to the Redis HA chart instead of the default one.

object({

    application_set = optional(object({
      requests = optional(object({
        cpu    = optional(string, "100m")
        memory = optional(string, "128Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string, "100m")
        memory = optional(string, "128Mi")
      }), {})
    }), {})

    controller = optional(object({
      requests = optional(object({
        cpu    = optional(string, "500m")
        memory = optional(string, "512Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string, "1")
        memory = optional(string, "2Gi")
      }), {})
    }), {})

    notifications = optional(object({
      requests = optional(object({
        cpu    = optional(string, "100m")
        memory = optional(string, "128Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string, "200m")
        memory = optional(string, "256Mi")
      }), {})
    }), {})

    repo_server = optional(object({
      requests = optional(object({
        cpu    = optional(string, "200m")
        memory = optional(string, "128Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string, "400m")
        memory = optional(string, "256Mi")
      }), {})
    }), {})

    server = optional(object({
      requests = optional(object({
        cpu    = optional(string, "50m")
        memory = optional(string, "128Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string, "100m")
        memory = optional(string, "256Mi")
      }), {})
    }), {})

    redis = optional(object({
      requests = optional(object({
        cpu    = optional(string, "200m")
        memory = optional(string, "64Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string, "300m")
        memory = optional(string, "128Mi")
      }), {})
    }), {})

  })

{}

high_availability

Argo CD High Availability settings. By default, the HA is disabled.

To enable HA using the default replicas, simply set the value high_availability.enabled to true. This will deploy Argo CD in HA without autoscaling.

Activating the HA mode automatically enables the Redis HA chart which requires at least 3 worker nodes, as this chart enforces Pods to run on separate nodes.

object({
    enabled = bool

    controller = optional(object({
      replicas = optional(number, 1)
    }), {})

    application_set = optional(object({
      replicas = optional(number, 2)
    }), {})

    server = optional(object({
      replicas = optional(number, 2)
      autoscaling = optional(object({
        enabled      = bool
        min_replicas = optional(number, 2)
        max_replicas = optional(number, 5)
        }), {
        enabled = false
      })
    }), {})

    repo_server = optional(object({
      replicas = optional(number, 2)
      autoscaling = optional(object({
        enabled      = bool
        min_replicas = optional(number, 2)
        max_replicas = optional(number, 5)
        }), {
        enabled = false
      })
    }), {})

  })

{
  "enabled": false
}

oidc

OIDC settings for the log in to the Argo CD web interface.

any

null

rbac

RBAC settings for the Argo CD users.

object({
    scopes         = optional(string, "[groups, cognito:groups, roles]")
    policy_default = optional(string, "")
    policy_csv = optional(string, <<-EOT
                                    g, pipeline, role:admin
                                    g, argocd-admin, role:admin
                                    g, devops-stack-admins, role:admin
                                  EOT
    )
  })

{}

repositories

List of repositories to add to Argo CD.

map(map(string))

{}

ssh_known_hosts

List of SSH known hosts to add to Argo CD.

Check the official values.yaml to get the format to pass this value.

If you set this variable, the default known hosts will be overridden by this value, so you might want to consider adding the ones you need here."

string

null

exec_enabled

Flag to enable the web-based terminal on Argo CD. Do not forget to set the appropriate RBAC configuration to your users/groups.

bool

false

admin_enabled

Flag to indicate whether to enable the administrator user.

bool

false

accounts_pipeline_tokens

API token for pipeline account.

string

n/a

yes

server_secretkey

Signature key for session validation. Must reuse the bootstrap output containing the secretkey.

string

n/a

yes

extra_accounts

List of accounts for which tokens will be generated.

list(string)

[]

repo_server_iam_role_arn

IAM role ARN to associate with the argocd-repo-server ServiceAccount. This role can be used to give SOPS access to AWS KMS.

string

null

repo_server_azure_workload_identity_clientid

Azure AD Workload Identity Client-ID to associate with argocd-repo-server. This role can be used to give SOPS access to a Key Vault.

string

null

repo_server_aadpodidbinding

Azure AAD Pod Identity to associate with the argocd-repo-server Pod. This role can be used to give SOPS access to a Key Vault.

string

null

helmfile_cmp_version

Version of the helmfile-cmp plugin.

string

"0.1.1"

helmfile_cmp_env_variables

List of environment variables to attach to the helmfile-cmp plugin, usually used to pass authentication credentials. Use an explicit format or take the values from a Kubernetes secret.

list(object({
    name  = optional(string)
    value = optional(string)
    valueFrom = optional(object({
      secretKeyRef = optional(object({
        name = optional(string)
        key  = optional(string)
      }))
    }))
  }))

[]

= Outputs

Name	Description
id	ID to pass other modules in order to refer to this module as a dependency.
extra_tokens	Map of extra accounts that were created and their tokens.

Name

Description

ID to pass other modules in order to refer to this module as a dependency.

extra_tokens

Map of extra accounts that were created and their tokens.

devops-stack-module-argocd

Usage

Custom Plugins

Troubleshooting

connection_error during terraform apply

Argo CD interface reload loop when clicking on login

Technical Documentation

Dependencies

module.argocd_bootstrap.id

module.traefik.id

module.traefik.id

module.oidc.id

module.kube-prometheus-stack.id

Requirements

Providers

Resources

Required Inputs

Optional Inputs

Outputs

Reference in table format

`connection_error` during `terraform apply`

`module.argocd_bootstrap.id`

`module.traefik.id`

`module.traefik.id`

`module.oidc.id`

`module.kube-prometheus-stack.id`