SKS Variant

This folder contains the variant to use when deploying in Exoscale using an SKS cluster.

Usage

This module can be declared by adding the following block on your Terraform configuration:

module "loki-stack" {
  source = "git::https://github.com/camptocamp/devops-stack-module-loki-stack//sks?ref=<RELEASE>"

  cluster_id       = module.sks.cluster_id
  argocd_namespace = module.argocd_bootstrap.argocd_namespace

  logs_storage = {
    bucket_name = resource.aws_s3_bucket.this["loki"].id
    region      = resource.aws_s3_bucket.this["loki"].region
    access_key  = resource.exoscale_iam_access_key.s3_iam_key["loki"].key
    secret_key  = resource.exoscale_iam_access_key.s3_iam_key["loki"].secret
  }

  dependency_ids = {
    argocd   = module.argocd_bootstrap.id
    longhorn = module.longhorn.id
  }
}
You are in charge of creating a S3 bucket for Loki to store the archived logs. We’ve decided to keep the creation of this bucket outside of this module, mainly because the persistence of the data should not be related to the instantiation of the module itself.
Check the SKS deployment example to see how to create the S3 bucket and to better understand the values passed on the example above.

If there is a need to configure something besides the common settings that we have provided, you can customize the chart’s values.yaml by adding an Helm configuration as an HCL structure:

module "loki-stack" {
  source = "git::https://github.com/camptocamp/devops-stack-module-loki-stack//sks?ref=<RELEASE>"

  cluster_id       = module.sks.cluster_id
  argocd_namespace = module.argocd_bootstrap.argocd_namespace

  logs_storage = {
    bucket_name = resource.aws_s3_bucket.this["loki"].id
    region      = resource.aws_s3_bucket.this["loki"].region
    access_key  = resource.exoscale_iam_access_key.s3_iam_key["loki"].key
    secret_key  = resource.exoscale_iam_access_key.s3_iam_key["loki"].secret
  }

  helm_values = [{ # Note the curly brackets here
    loki-distributed = {
      map = {
        string = "string"
        bool   = true
      }
      sequence = [
        {
          key1 = "value1"
          key2 = "value2"
        },
        {
          key1 = "value1"
          key2 = "value2"
        },
      ]
      sequence2 = [
        "string1",
        "string2"
      ]
    }
  }]

  dependency_ids = {
    argocd   = module.argocd_bootstrap.id
    longhorn = module.longhorn.id
  }

Technical Reference

Dependencies

module.argocd_bootstrap.id

Obviously, the module depends on an already running Argo CD in the cluster in order for the application to be created.

module.longhorn.id

This module requires a Persistent Volume so it needs to be deployed after the module Longhorn.

Requirements

The following requirements are needed by this module:

Modules

The following Modules are called:

loki-stack

Source: ../

Version:

Required Inputs

The following input variables are required:

logs_storage

Description: Exoscale SOS bucket configuration values for the bucket where the logs will be stored.

Type:

object({
    bucket_name = string
    region      = string
    access_key  = string
    secret_key  = string
  })

Optional Inputs

The following input variables are optional (have default values):

argocd_project

Description: Name of the Argo CD AppProject where the Application should be created. If not set, the Application will be created in a new AppProject only for this Application.

Type: string

Default: null

argocd_labels

Description: Labels to attach to the Argo CD Application resource.

Type: map(string)

Default: {}

destination_cluster

Description: Destination cluster where the application should be deployed.

Type: string

Default: "in-cluster"

target_revision

Description: Override of target revision of the application chart.

Type: string

Default: "v10.0.0"

helm_values

Description: Helm chart value overrides. They should be passed as a list of HCL structures.

Type: any

Default: []

app_autosync

Description: Automated sync options for the Argo CD Application resource.

Type:

object({
    allow_empty = optional(bool)
    prune       = optional(bool)
    self_heal   = optional(bool)
  })

Default:

{
  "allow_empty": false,
  "prune": true,
  "self_heal": true
}

dependency_ids

Description: IDs of the other modules on which this module depends on.

Type: map(string)

Default: {}

resources

Description: Resource limits and requests for Loki’s components. Follow the style on official documentation to understand the format of the values.

These are not production values. You should always adjust them to your needs.

Type:

object({

    ingester = optional(object({
      requests = optional(object({
        cpu    = optional(string, "100m")
        memory = optional(string, "256Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string)
        memory = optional(string, "512Mi")
      }), {})
    }), {})

    distributor = optional(object({
      requests = optional(object({
        cpu    = optional(string, "100m")
        memory = optional(string, "256Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string)
        memory = optional(string, "512Mi")
      }), {})
    }), {})

    querier = optional(object({
      requests = optional(object({
        cpu    = optional(string, "100m")
        memory = optional(string, "256Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string)
        memory = optional(string, "512Mi")
      }), {})
    }), {})

    query_frontend = optional(object({
      requests = optional(object({
        cpu    = optional(string, "100m")
        memory = optional(string, "256Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string)
        memory = optional(string, "512Mi")
      }), {})
    }), {})

    query_scheduler = optional(object({
      requests = optional(object({
        cpu    = optional(string, "100m")
        memory = optional(string, "256Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string)
        memory = optional(string, "512Mi")
      }), {})
    }), {})

    compactor = optional(object({
      requests = optional(object({
        cpu    = optional(string, "100m")
        memory = optional(string, "512Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string)
        memory = optional(string, "1Gi")
      }), {})
    }), {})

    index_gateway = optional(object({
      requests = optional(object({
        cpu    = optional(string, "100m")
        memory = optional(string, "256Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string)
        memory = optional(string, "1Gi")
      }), {})
    }), {})

    memcached_chunks = optional(object({
      requests = optional(object({
        cpu    = optional(string, "100m")
        memory = optional(string, "256Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string)
        memory = optional(string, "512Mi")
      }), {})
    }), {})

    memcached_frontend = optional(object({
      requests = optional(object({
        cpu    = optional(string, "100m")
        memory = optional(string, "256Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string)
        memory = optional(string, "512Mi")
      }), {})
    }), {})

    memcached_index_queries = optional(object({
      requests = optional(object({
        cpu    = optional(string, "100m")
        memory = optional(string, "256Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string)
        memory = optional(string, "512Mi")
      }), {})
    }), {})

    memcached_index_writes = optional(object({
      requests = optional(object({
        cpu    = optional(string, "100m")
        memory = optional(string, "256Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string)
        memory = optional(string, "512Mi")
      }), {})
    }), {})

    promtail = optional(object({
      requests = optional(object({
        cpu    = optional(string, "100m")
        memory = optional(string, "256Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string)
        memory = optional(string, "512Mi")
      }), {})
    }), {})

    grafana_eventhandler = optional(object({
      requests = optional(object({
        cpu    = optional(string, "100m")
        memory = optional(string, "128Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string)
        memory = optional(string, "256Mi")
      }), {})
    }), {})

  })

Default: {}

ingress

Description: Loki frontend ingress configuration.

Type:

object({
    hosts          = list(string)
    cluster_issuer = string
    allowed_ips    = optional(list(string), [])
  })

Default: null

retention

Description: Logs retention period. To deactivate retention, pass 0s.

Type: string

Default: "30d"

Outputs

The following outputs are exported:

id

Description: ID to pass other modules in order to refer to this module as a dependency.

loki_credentials

Description: Credentials to access the Loki ingress, if activated.

Reference in table format

Show tables

= Requirements

Name Version

argocd

>= 6

htpasswd

>= 1

null

>= 3

random

>= 3

utils

>= 1

= Modules

Name Source Version

loki-stack

= Inputs

Name Description Type Default Required

logs_storage

Exoscale SOS bucket configuration values for the bucket where the logs will be stored.

 
object({
    bucket_name = string
    region      = string
    access_key  = string
    secret_key  = string
  })

n/a

yes

argocd_project

Name of the Argo CD AppProject where the Application should be created. If not set, the Application will be created in a new AppProject only for this Application.

string

null

no

argocd_labels

Labels to attach to the Argo CD Application resource.

map(string)

{}

no

destination_cluster

Destination cluster where the application should be deployed.

string

"in-cluster"

no

target_revision

Override of target revision of the application chart.

string

"v10.0.0"

no

helm_values

Helm chart value overrides. They should be passed as a list of HCL structures.

any

[]

no

app_autosync

Automated sync options for the Argo CD Application resource.

 
object({
    allow_empty = optional(bool)
    prune       = optional(bool)
    self_heal   = optional(bool)
  })
 
{
  "allow_empty": false,
  "prune": true,
  "self_heal": true
}

no

dependency_ids

IDs of the other modules on which this module depends on.

map(string)

{}

no

resources

Resource limits and requests for Loki’s components. Follow the style on official documentation to understand the format of the values.

These are not production values. You should always adjust them to your needs. 
object({

    ingester = optional(object({
      requests = optional(object({
        cpu    = optional(string, "100m")
        memory = optional(string, "256Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string)
        memory = optional(string, "512Mi")
      }), {})
    }), {})

    distributor = optional(object({
      requests = optional(object({
        cpu    = optional(string, "100m")
        memory = optional(string, "256Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string)
        memory = optional(string, "512Mi")
      }), {})
    }), {})

    querier = optional(object({
      requests = optional(object({
        cpu    = optional(string, "100m")
        memory = optional(string, "256Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string)
        memory = optional(string, "512Mi")
      }), {})
    }), {})

    query_frontend = optional(object({
      requests = optional(object({
        cpu    = optional(string, "100m")
        memory = optional(string, "256Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string)
        memory = optional(string, "512Mi")
      }), {})
    }), {})

    query_scheduler = optional(object({
      requests = optional(object({
        cpu    = optional(string, "100m")
        memory = optional(string, "256Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string)
        memory = optional(string, "512Mi")
      }), {})
    }), {})

    compactor = optional(object({
      requests = optional(object({
        cpu    = optional(string, "100m")
        memory = optional(string, "512Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string)
        memory = optional(string, "1Gi")
      }), {})
    }), {})

    index_gateway = optional(object({
      requests = optional(object({
        cpu    = optional(string, "100m")
        memory = optional(string, "256Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string)
        memory = optional(string, "1Gi")
      }), {})
    }), {})

    memcached_chunks = optional(object({
      requests = optional(object({
        cpu    = optional(string, "100m")
        memory = optional(string, "256Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string)
        memory = optional(string, "512Mi")
      }), {})
    }), {})

    memcached_frontend = optional(object({
      requests = optional(object({
        cpu    = optional(string, "100m")
        memory = optional(string, "256Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string)
        memory = optional(string, "512Mi")
      }), {})
    }), {})

    memcached_index_queries = optional(object({
      requests = optional(object({
        cpu    = optional(string, "100m")
        memory = optional(string, "256Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string)
        memory = optional(string, "512Mi")
      }), {})
    }), {})

    memcached_index_writes = optional(object({
      requests = optional(object({
        cpu    = optional(string, "100m")
        memory = optional(string, "256Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string)
        memory = optional(string, "512Mi")
      }), {})
    }), {})

    promtail = optional(object({
      requests = optional(object({
        cpu    = optional(string, "100m")
        memory = optional(string, "256Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string)
        memory = optional(string, "512Mi")
      }), {})
    }), {})

    grafana_eventhandler = optional(object({
      requests = optional(object({
        cpu    = optional(string, "100m")
        memory = optional(string, "128Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string)
        memory = optional(string, "256Mi")
      }), {})
    }), {})

  })

{}

no

ingress

Loki frontend ingress configuration.

 
object({
    hosts          = list(string)
    cluster_issuer = string
    allowed_ips    = optional(list(string), [])
  })

null

no

retention

Logs retention period. To deactivate retention, pass 0s.

string

"30d"

no

= Outputs

Name Description

id

ID to pass other modules in order to refer to this module as a dependency.

loki_credentials

Credentials to access the Loki ingress, if activated.