Edge computing demands lightweight, resilient infrastructure. With the release of the Raspberry Pi 5, we finally have the CPU and I/O performance required to run a proper High Availability (HA) Kubernetes cluster at the edge using k3s. This guide covers deploying an embedded etcd HA k3s cluster across three Raspberry Pi 5 nodes.

Prerequisites & OS Configuration

Assume three Pi 5 nodes (pi-master-1, pi-master-2, pi-master-3) running Ubuntu 24.04 Server. First, we must enable cgroups and disable swap to ensure kubelet functions correctly.

# On all nodes: Edit boot parameters
sudo sed -i '$ s/$/ cgroup_enable=cpuset cgroup_enable=memory cgroup_memory=1 swapaccount=1/' /boot/firmware/cmdline.txt

# Disable swap immediately and persistently
sudo swapoff -a
sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab

sudo reboot

Annotation: We append cgroup configurations to cmdline.txt which is necessary for k3s pod resource limits. Disabling swap prevents kubelet from failing on startup, a strict requirement for standard Kubernetes deployments.

Initializing the First Control Plane Node

We use the embedded etcd datastore for HA, which eliminates the need for an external database like MySQL or PostgreSQL.

# On pi-master-1 (10.0.0.11)
curl -sfL https://get.k3s.io | sh -s - server \
  --cluster-init \
  --tls-san 10.0.0.100 \
  --node-taint CriticalAddonsOnly=true:NoExecute

# Retrieve the node token for joining other nodes
cat /var/lib/rancher/k3s/server/node-token

Annotation: --cluster-init instructs k3s to bootstrap a new etcd cluster. --tls-san 10.0.0.100 adds a Subject Alternative Name for our eventual Virtual IP (VIP) managed by Kube-VIP. The taint ensures regular workloads don't schedule on this control plane node.

Joining Additional Control Plane Nodes

With the cluster initialized and the token retrieved, we join the remaining two nodes to form an etcd quorum.

# On pi-master-2 & pi-master-3
export K3S_TOKEN=""
export K3S_URL="https://10.0.0.11:6443"

curl -sfL https://get.k3s.io | sh -s - server \
  --server $K3S_URL \
  --token $K3S_TOKEN \
  --tls-san 10.0.0.100 \
  --node-taint CriticalAddonsOnly=true:NoExecute

Annotation: Notice we are running the server command, not agent. Passing --server to the server command tells k3s to join an existing cluster as a control plane peer rather than bootstrapping a new one.

Implementing Kube-VIP for the API Server

To achieve true HA, kube-config must point to a highly available endpoint, not a single node. We deploy Kube-VIP via a static pod manifest.

# Generate Kube-VIP manifest on master-1
KVVERSION=$(curl -sL https://api.github.com/repos/kube-vip/kube-vip/releases | jq -r ".[0].name")
sudo ctr image pull ghcr.io/kube-vip/kube-vip:$KVVERSION

# Create the static pod manifest
sudo ctr run --rm --net-host ghcr.io/kube-vip/kube-vip:$KVVERSION vip \
  /kube-vip manifest pod \
  --interface eth0 \
  --address 10.0.0.100 \
  --controlplane \
  --services \
  --arp \
  --leaderElection | sudo tee /var/lib/rancher/k3s/agent/pod-manifests/kube-vip.yaml

Annotation: We use containerd (ctr) bundled with k3s to pull and execute the kube-vip binary to generate the YAML. Placing this file in /var/lib/rancher/k3s/agent/pod-manifests/ tells the kubelet to run this as a static pod. --arp enables Layer 2 load balancing, and --leaderElection ensures only one node broadcasts the ARP for the VIP 10.0.0.100.