k3s — Lightweight Kubernetes

k3s is a certified, production-ready Kubernetes distribution packaged as a single binary (~100 MB). It is optimized for small machines: homelabs, edge and IoT.

Why k3s

Feature	Value
Single binary	No Docker daemon required
Built-in containerd	Reduced resource footprint
ARM + x86 support	Pi, mini PC, VM or VPS
Certified Kubernetes	100% API compatible
Automatic TLS	Internal cluster PKI included

Architecture

┌─────────────────────────────────┐    ┌─────────────────────────────────┐
│         Server (node 1)         │    │         Agent (node 2)          │
│                                 │    │                                 │
│  k3s server                     │    │  k3s agent                      │
│  ├─ kube-apiserver  :6443       │◄───│  ├─ kubelet                     │
│  ├─ etcd (embedded)             │    │  ├─ kube-proxy                  │
│  ├─ controller-manager          │    │  └─ containerd                  │
│  └─ scheduler                   │    │                                 │
│                                 │    │  Flannel VXLAN  :8472/udp       │
│  Traefik (ingress)  :80/:443    │    │  Kubelet API    :10250/tcp      │
│  cert-manager                   │    │                                 │
└─────────────────────────────────┘    └─────────────────────────────────┘
           │                                         │
           └─────────── Flannel VXLAN ───────────────┘
                        (pod overlay network)

Prerequisites

Requirement	Notes
Ubuntu 22.04+ / Debian 12+	Both nodes
2 vCPU / 2 GB RAM (server)	1 GB minimum for agent
Reachable IP on each node	Required for TLS SAN + UFW rules
SSH access as root or sudo user	Bootstrap uses `INITIAL_USER=root`
Port `6443` open on server	k3s API server
Ports `80`, `443` open on server	HTTP + HTTPS traffic

Install flags explained

The k3s server Ansible role (ansible/roles/k3s_server) installs k3s with these key flags:

bash

curl -sfL https://get.k3s.io | \
  INSTALL_K3S_VERSION="${K3S_VERSION}" \
  K3S_TOKEN="${K3S_NODE_TOKEN}" \
  sh -s - server \
    --disable=traefik \          # Managed via Helm for full control
    --disable=servicelb \        # No built-in LB — use externalIPs instead
    --node-ip="${NODE_IP}" \     # Internal NIC IP (Flannel overlay)
    --advertise-address="${NODE_IP}" \
    --tls-san="${PUBLIC_IP}" \   # Public IP added to API server certificate SAN
    --flannel-backend=vxlan \    # Stable VXLAN overlay (UDP 8472)
    --protect-kernel-defaults \  # Enforces sysctl requirements
    --secrets-encryption \       # Encrypts Kubernetes Secrets at rest
    --write-kubeconfig-mode=600  # Restrict kubeconfig permissions

--tls-san is critical — without the public IP in the TLS SAN, your local kubectl will get a certificate error when connecting remotely.

Sysctl requirements

k3s requires specific kernel parameters. These are written to /etc/sysctl.d/99-z-k3s.conf:

ini

net.bridge.bridge-nf-call-iptables  = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward                 = 1
net.ipv4.conf.all.forwarding        = 1
vm.panic_on_oom                     = 0
vm.overcommit_memory                = 1
kernel.panic                        = 10
kernel.panic_on_oops                = 1

The 99-z- prefix ensures these values are applied after any hardening configs (e.g. 99-security.conf), so ip_forward=1 is the final value.

Firewall rules (UFW)

Server node

Port	Protocol	Purpose
`80`	TCP	HTTP (Traefik + ACME HTTP-01 challenge)
`443`	TCP	HTTPS (Traefik TLS termination)
`6443`	TCP	Kubernetes API server
`10.42.0.0/16`	any	k3s pod CIDR (Flannel)
`10.43.0.0/16`	any	k3s service CIDR
`8472` from AGENT_IP	UDP	Flannel VXLAN tunnel
`10250` from AGENT_IP	TCP	kubelet API

Agent node

Port	Protocol	Purpose
`8472` from SERVER_IP	UDP	Flannel VXLAN tunnel
`10250` from SERVER_IP	TCP	kubelet API

Installation workflow

1. Bootstrap server

bash

task provision:server

This runs the Ansible k3s-server.yml playbook which:

Applies common setup (packages, kernel modules, sysctl, UFW)
Installs k3s server with configured flags
Waits for the node to be Ready
Reads the node token (used to join agents)
Fetches kubeconfig with public IP replaced
Optionally sets up WireGuard VPN

2. Bootstrap agent

bash

task provision:agents

This runs the Ansible k3s-agent.yml playbook which:

Applies common setup on agent nodes
Installs k3s agent with server URL and node token
Configures UFW for cluster communication

3. Fetch kubeconfig

bash

task kubeconfig:fetch

Fetches /etc/rancher/k3s/k3s.yaml from the server, replaces 127.0.0.1 with the public IP, and merges it into ~/.kube/config under the context name KUBECONFIG_CONTEXT.

Uninstall

bash

task provision:reset       # Uninstall k3s from all nodes (DESTRUCTIVE)

The Ansible reset.yml playbook:

Runs the official k3s-uninstall.sh or k3s-agent-uninstall.sh
Cleans up CNI interfaces (flannel.1, cni0)
Flushes iptables rules
Unmounts k3s bind mounts

Node token

The node token is a shared secret that agents use to authenticate with the server API.

Auto-generated during server install if K3S_NODE_TOKEN is empty
Automatically read from the server by the Ansible site.yml playbook
Stored on the server at /var/lib/rancher/k3s/server/node-token

To rotate: uninstall both nodes and reinstall with a new token.

Datastore backup

A single server keeps all cluster state in SQLite (/var/lib/rancher/k3s/server/db/state.db). The k3s_server role installs a daily k3s-db-backup.timer that writes /var/backups/k3s/k3s-<UTC timestamp>.tar.gz (mode 0600, newest 7 kept). Each archive holds:

File	Why it is needed
`state.db`	the cluster state, snapshotted online
`token`	k3s refuses to start on a datastore with a different server token
`encryption-config.json`	Secrets are encrypted at rest (`--secrets-encryption`) — unreadable without it

Tune with k3s_db_backup_* in group_vars (k3s_db_backup_enabled: false to turn it off). Copy the archives off the node from your own config repo (restic, rclone, …): a backup on the same disk does not survive that disk.

Restore on a server with the same k3s version:

bash

systemctl stop k3s
mkdir -p /tmp/k3s-restore && tar -C /tmp/k3s-restore -xzf k3s-<stamp>.tar.gz
cp /tmp/k3s-restore/state.db               /var/lib/rancher/k3s/server/db/state.db
cp /tmp/k3s-restore/token                  /var/lib/rancher/k3s/server/token
cp /tmp/k3s-restore/encryption-config.json /var/lib/rancher/k3s/server/cred/
rm -f /var/lib/rancher/k3s/server/db/state.db-{wal,shm}
systemctl start k3s

PersistentVolume data (local-path) is not in this archive — back up volumes separately (e.g. Velero) from your config repo.

Kubernetes

k3s — Lightweight Kubernetes ​

Why k3s ​

Architecture ​

Prerequisites ​

Install flags explained ​

Sysctl requirements ​

Firewall rules (UFW) ​

Server node ​

Agent node ​

Installation workflow ​

1. Bootstrap server ​

2. Bootstrap agent ​

3. Fetch kubeconfig ​

Uninstall ​

Node token ​

Datastore backup ​

References ​

k3s — Lightweight Kubernetes

Why k3s

Architecture

Prerequisites

Install flags explained

Sysctl requirements

Firewall rules (UFW)

Server node

Agent node

Installation workflow

1. Bootstrap server

2. Bootstrap agent

3. Fetch kubeconfig

Uninstall

Node token

Datastore backup

References