K3s on Docker Quickstart

Prerequisites

  • Access to a functional Docker Engine,

  • Knowledge of Terraform basics

  • Minimal Terraform version: 0.14

  • jq binary

  • kubectl binary

  • argocd CLI

Create your Terraform root module

Camptocamp’s DevOps Stack is instantiated using a Terraform composition module.

Here is a minimal working example:

# terraform/main.tf

module "cluster" {
  source = "git::https://github.com/camptocamp/devops-stack.git//modules/k3s/docker?ref=master"

  cluster_name = "my-cluster"
}

If your docker setup doesn’t support the bridge0 like on MacOSX, you cannot access to the container IP so the computed base domain isn’t reachable. You can specify the published ports of the K3S master and the base domain to use your computer’s IP.

# terraform/main.tf

module "cluster" {
  source = "git::https://github.com/camptocamp/devops-stack.git//modules/k3s/docker?ref=master"

  cluster_name = "my-cluster"
  cluster_endpoint = "192-168-1-118.nip.io"
  base_domain      = "192-168-1-118.nip.io"
  server_ports = [
    {
      internal = 6443
      external = 6443
    },
    {
      internal = 80
      external = 80
    },
    {
      internal = 443
      external = 443
    },
  ]
}

Terraform Outputs

Define outputs:

# terraform/outputs.tf

output "kubeconfig" {
  sensitive = true
  value     = module.cluster.kubeconfig
}

output "argocd_url" {
  value = format("https://argocd.apps.%s.%s", var.cluster_name, module.cluster.base_domain)
}

output "keycloak_url" {
  value = format("https://keycloak.apps.%s.%s", var.cluster_name, module.cluster.base_domain)
}

output "grafana_url" {
  value = format("https://grafana.apps.%s.%s", var.cluster_name, module.cluster.base_domain)
}

output "prometheus_url" {
  value = format("https://prometheus.apps.%s.%s", var.cluster_name, module.cluster.base_domain)
}

output "alertmanager_url" {
  value = format("https://alertmanager.apps.%s.%s", var.cluster_name, module.cluster.base_domain)
}

output "argocd_server_admin_password" {
  sensitive = true
  value     = module.cluster.argocd_server_admin_password
}

output "keycloak_admin_password" {
  sensitive = true
  value     = module.cluster.keycloak_admin_password
}

output "grafana_admin_password" {
  sensitive = true
  value     = module.cluster.grafana_admin_password
}

output "keycloak_users" {
  value     = module.cluster.keycloak_users
  sensitive = true
}

Deploy the cluster

$ terraform init
$ terraform apply

You should see the services URL as Terraform outputs.

Get kubeconfig and Keycloak users credentials

Retrieve the Kubeconfig file:

$ terraform output -json kubeconfig | jq -r . > kubeconfig.yaml
$ export KUBECONFIG=kubeconfig.yaml

By default, two users are defined in Keycloak:

user Keycloak role Keycloak realm terraform output comment

admin

Administrator

all

keycloak_admin_password

This user has admin rights only in Keycloak. Use keycloak_url and select "Administration Console" to login.

jdoe

applications

devops-stack

keycloak_users

This user has related applications rights within Kubernetes realm. Use keycloak_url/auth/realms/devops-stack/account/ to login.

To retrieve password:

$ terraform output keycloak_admin_password
$ terraform output keycloak_users
# a user map is displayed that includes jdoe password

Wait for Keycloak to be ready

$ kubectl -n keycloak get sts
NAME       READY   AGE
keycloak   1/1     8m58s

Wait until the READY column says 1/1.

Inspect the DevOps Stack Applications

You can view the ingress routes for the various DevOps Stack Applications with:

$ kubectl get ingress --all-namespaces

Access the URLs in https or use the URL output from terraform (see table below).

Application URL user password comment

Argo CD

argocd_url

admin

argocd_server_admin_password

Argo CD is a declarative, GitOps continuous delivery tool for Kubernetes. The argo CD web application allows you to visualise the application deployment, configurations, status to name a few.

Grafana

grafana_url

admin

grafana_admin_password

Grafana is a visualisation web application for metrics or log data. The devops-stack provides pre-defined dashboards ready to use. The devops-stack provides pre-defined dashboards ready to use.

Grafana

grafana_url

jdoe

jdoe_password

To visualise logs in Grafana (use "Explore" menu), users need "Editor" Grafana rights. By default in devops-stack, Grafana user rights is set to "Editor" such as John Doe (our user example).

Prometheus

prometheus_url

n/a

n/a

Prometheus web app is mainly used to test queries, a one time metrics visualisation. This application is not used for dashboarding but Grafana instead.

Alertmanager

alertmanager_url

n/a

n/a

The Alertmanager handles alerts sent by client applications such as the Prometheus server. It takes care of deduplicating, grouping, and routing them to the correct receiver integration (e.g. email, PagerDuty, etc). It also takes care of silencing and inhibition of alerts.

Access the Keycloak dashboard

The keycloak dashboard uses the devops-stack realm. You can log in to it using the /auth/realms/devops-stack/account/ path with the Keycloak ingress.

Destroy the cluster

$ terraform destroy

Reference

See the K3s Docker reference page.