K3s on Libvirt Quickstart

Prerequisites

Terraform composition module

To bootstrap the Camptocamp’s DevOps Stack you just have to create these 3 following Terraform code files:

  • terraform/main.tf: the main cluster module block with the source code for the desired DevOps Stack version

  • `terraform/variables.tf: an input variable with the default cluster name

  • terraform/outputs.tf: return values for accessing the deployed cluster/tools (admin password and URLs)

Here are the contents of these files:

# terraform/main.tf

module "cluster" {
  source = "git::https://github.com/camptocamp/devops-stack.git//modules/k3s/libvirt?ref=master"
  cluster_name  = var.cluster_name
  node_count    = 1
  server_memory = 8192
}
# variables/variables.tf

variable "cluster_name" {
  type    = string
  default = "default"
}
# terraform/outputs.tf

output "kubeconfig" {
  sensitive = true
  value     = module.cluster.kubeconfig
}

output "argocd_url" {
  value = format("https://argocd.apps.%s.%s", var.cluster_name, module.cluster.base_domain)
}

output "keycloak_url" {
  value = format("https://keycloak.apps.%s.%s", var.cluster_name, module.cluster.base_domain)
}

output "grafana_url" {
  value = format("https://grafana.apps.%s.%s", var.cluster_name, module.cluster.base_domain)
}

output "prometheus_url" {
  value = format("https://prometheus.apps.%s.%s", var.cluster_name, module.cluster.base_domain)
}

output "alertmanager_url" {
  value = format("https://alertmanager.apps.%s.%s", var.cluster_name, module.cluster.base_domain)
}

output "argocd_server_admin_password" {
  sensitive = true
  value     = module.cluster.argocd_server_admin_password
}

output "keycloak_admin_password" {
  sensitive = true
  value     = module.cluster.keycloak_admin_password
}

output "grafana_admin_password" {
  sensitive = true
  value     = module.cluster.grafana_admin_password
}

output "keycloak_users" {
  value     = module.cluster.keycloak_users
  sensitive = true
}

Deploy the cluster

$ terraform init
$ terraform apply

You should see the services URL as Terraform outputs.

Get kubeconfig and Keycloak users credentials

Retrieve the Kubeconfig file:

$ terraform output -json kubeconfig | jq -r . > kubeconfig.yaml
$ export KUBECONFIG=kubeconfig.yaml

By default, two users are defined in Keycloak:

user Keycloak role Keycloak realm terraform output comment

admin

Administrator

all

keycloak_admin_password

This user has admin rights only in Keycloak. Use keycloak_url and select "Administration Console" to login.

jdoe

applications

devops-stack

keycloak_users

This user has related applications rights within Kubernetes realm. Use keycloak_url/auth/realms/devops-stack/account/ to login.

To retrieve password:

$ terraform output keycloak_admin_password
$ terraform output keycloak_users
# a user map is displayed that includes jdoe password

Wait for Keycloak to be ready

$ kubectl -n keycloak get sts
NAME       READY   AGE
keycloak   1/1     8m58s

Wait until the READY column says 1/1.

Inspect the DevOps Stack Applications

You can view the ingress routes for the various DevOps Stack Applications with:

$ kubectl get ingress --all-namespaces

Access the URLs in https or use the URL output from terraform (see table below).

Application URL user password comment

Argo CD

argocd_url

admin

argocd_server_admin_password

Argo CD is a declarative, GitOps continuous delivery tool for Kubernetes. The argo CD web application allows you to visualise the application deployment, configurations, status to name a few.

Grafana

grafana_url

admin

grafana_admin_password

Grafana is a visualisation web application for metrics or log data. The devops-stack provides pre-defined dashboards ready to use. The devops-stack provides pre-defined dashboards ready to use.

Grafana

grafana_url

jdoe

jdoe_password

To visualise logs in Grafana (use "Explore" menu), users need "Editor" Grafana rights. By default in devops-stack, Grafana user rights is set to "Editor" such as John Doe (our user example).

Prometheus

prometheus_url

n/a

n/a

Prometheus web app is mainly used to test queries, a one time metrics visualisation. This application is not used for dashboarding but Grafana instead.

Alertmanager

alertmanager_url

n/a

n/a

The Alertmanager handles alerts sent by client applications such as the Prometheus server. It takes care of deduplicating, grouping, and routing them to the correct receiver integration (e.g. email, PagerDuty, etc). It also takes care of silencing and inhibition of alerts.

Access the Keycloak dashboard

The keycloak dashboard uses the devops-stack realm. You can log in to it using the /auth/realms/devops-stack/account/ path with the Keycloak ingress.

Destroy the cluster

$ terraform destroy

Reference

See the K3s Libvirt reference page.