K3s on Libvirt Quickstart

Setting up extra terraform provider

Due to an issue in the terraform provider system, you need to download and place the libvirt provider at a very specific location in your home directory before deploying K3s. You can find the libvirt provider version in modules/k3s/libvirt/versions.tf. To keep things simple this version will be referred to as $LIBVIRT_PROVIDER_VERSION in this documentation. Go to https://github.com/dmacvicar/terraform-provider-libvirt/releases/tag/v$LIBVIRT_PROVIDER_VERSION and find the correct link for your OS/CPU_ARCH (an example is linux_amd64, referred to as $OS_CPU_ARCH in the rest of this documentation).
mkdir -p ~/.local/share/terraform/plugins/registry.terraform.io/dmacvicar/libvirt/$LIBVIRT_PROVIDER_VERSION/$OS_CPU_ARCH/
mv terraform-provider-libvirt ~/.local/share/terraform/plugins/registry.terraform.io/dmacvicar/libvirt/$LIBVIRT_PROVIDER_VERSION/$OS_CPU_ARCH/terraform-provider-libvirt

Prerequisites

  • Access to a functional Libvirt daemon

  • Knowledge of Terraform basics

  • jq binary

  • argocd CLI

Create your Terraform root module

Camptocamp’s DevOps Stack is instantiated using a Terraform composition module.

Here is a minimal working example:

# terraform/main.tf

module "cluster" {
  source = "git::https://github.com/camptocamp/devops-stack.git//modules/k3s/libvirt?ref=master"

  cluster_name = "my-cluster"
  node_count   = 2
}

Terraform Outputs

Define outputs:

# terraform/outputs.tf

output "argocd_server_admin_password" {
  sensitive = true
  value     = module.cluster.argocd_server_admin_password
}

output "argocd_auth_token" {
  sensitive = true
  value     = module.cluster.argocd_auth_token
}

output "kubeconfig" {
  sensitive = true
  value     = module.cluster.kubeconfig
}

output "argocd_server" {
  value = module.cluster.argocd_server
}

output "grafana_admin_password" {
  sensitive = true
  value     = module.cluster.grafana_admin_password
}

Deploy the cluster

$ terraform init
$ terraform apply

You should see the services URL as Terraform outputs.

Get kubeconfig and admin password

Retrieve the Kubeconfig file:

$ terraform output -json kubeconfig | jq -r .

Retrieve the Keycloak password for the jdoe user of the devops-stack realm:

$ terraform output admin_password

You will use this user and password to log in to applications.

Wait for Keycloak to be ready

$ kubectl -n keycloak get sts
NAME       READY   AGE
keycloak   1/1     8m58s

Wait until the READY column says 1/1.

Inspect the DevOps Stack Applications

You can view the ingress routes for the various DevOps Stack Applications with:

$ kubectl get ingress --all-namespaces

Access the URLs in https, and use the OIDC/OAuth2 to log in, using the admin account with the password previously retrieved.

Access the Keycloak dashboard

The keycloak dashboard uses the devops-stack realm. You can log in to it using the /auth/realms/devops-stack/account/ path with the Keycloak ingress.

Destroy the cluster

$ terraform destroy

Reference