Building a Highly Available k3s Edge Cluster on Raspberry Pi 5

With the Raspberry Pi 5, we gain sufficient CPU headroom and I/O throughput to run a genuine High Availability (HA) Kubernetes cluster on edge hardware. Here is how to set up a 3-node HA k3s cluster using embedded etcd and Kube-VIP.

Prerequisites & OS Setup

This setup uses three Pi 5 nodes (pi-master-1, pi-master-2, pi-master-3) running Ubuntu 24.04 Server. Prior to installing k3s, enable cgroups and disable swap so that kubelet operates reliably.

# On all nodes: Edit boot parameters
sudo sed -i '$ s/$/ cgroup_enable=cpuset cgroup_enable=memory cgroup_memory=1 swapaccount=1/' /boot/firmware/cmdline.txt
# Disable swap right away and keep it disabled after reboot
sudo swapoff -a
sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab
sudo reboot

Why this matters: Appending cgroup flags to cmdline.txt allows k3s to enforce pod memory and CPU limits. Disabling swap is required by Kubernetes so kubelet starts up reliably without throwing errors.

Initializing the First Control Plane Node

We will use k3s's built-in etcd datastore for HA. This saves us from managing a separate MySQL or PostgreSQL database.

# On pi-master-1 (10.0.0.11)
curl -sfL https://get.k3s.io | sh -s - server \
 --cluster-init \
 --tls-san 10.0.0.100 \
 --node-taint CriticalAddonsOnly=true:NoExecute
# Grab the cluster token needed for joining nodes
cat /var/lib/rancher/k3s/server/node-token

Details: --cluster-init instructs k3s to spin up a new etcd cluster. --tls-san 10.0.0.100 adds an extra SAN entry for the Virtual IP (VIP) we will set up with Kube-VIP. The CriticalAddonsOnly taint keeps general workloads off this control plane node.

Joining Additional Control Plane Nodes

Now that node 1 is running etcd, join the remaining two nodes to establish a proper etcd quorum.

# On pi-master-2 & pi-master-3
export K3S_TOKEN=""
export K3S_URL="https://10.0.0.11:6443"
curl -sfL https://get.k3s.io | sh -s - server \
 --server $K3S_URL \
 --token $K3S_TOKEN \
 --tls-san 10.0.0.100 \
 --node-taint CriticalAddonsOnly=true:NoExecute

Note: Even though we are joining existing nodes, we still pass server rather than agent. Adding --server tells k3s to attach as an additional control plane peer instead of a worker node.

Setting Up Kube-VIP for API Server HA

For true high availability, client traffic and kubectl should point to a shared virtual IP rather than an individual node's IP address. We can deploy Kube-VIP as a static pod manifest.

# Generate the Kube-VIP manifest on pi-master-1
KVVERSION=$(curl -sL https://api.github.com/repos/kube-vip/kube-vip/releases | jq -r ".[0].name")
sudo ctr image pull ghcr.io/kube-vip/kube-vip:$KVVERSION
# Generate static pod YAML
sudo ctr run --rm --net-host ghcr.io/kube-vip/kube-vip:$KVVERSION vip \
 /kube-vip manifest pod \
 --interface eth0 \
 --address 10.0.0.100 \
 --controlplane \
 --services \
 --arp \
 --leaderElection | sudo tee /var/lib/rancher/k3s/agent/pod-manifests/kube-vip.yaml

How it works: We use k3s's built-in ctr tool to fetch the Kube-VIP container and generate the pod spec directly into /var/lib/rancher/k3s/agent/pod-manifests/. kubelet automatically picks up files in this directory and runs them as static pods. --arp enables Layer 2 load balancing, and --leaderElection guarantees only one control plane node broadcasts ARP packets for 10.0.0.100 at a time.