K3s on Raspberry Pi

Introduction

K3s is a lightweight Kubernetes distribution for resource-constrained systems. Unlike full Kubernetes (1.5 GB+ RAM), K3s runs in ~500 MB. Perfect for learning Kubernetes, running edge workloads, or building small clusters. This guide covers single-node K3s setup, kubectl basics, and when K3s beats Docker Compose.

Prerequisites

• Raspberry Pi 4 (4GB RAM minimum, 8GB recommended) or Pi 5
• 32GB+ SSD (critical for etcd performance)
• Static IP address configured
• SSH access
• Raspberry Pi OS Bookworm or Ubuntu 22.04+

When to Use K3s vs Docker Compose

Feature	Docker Compose	K3s
Setup time	5 minutes	15 minutes
Memory overhead	~0 MB	~500 MB
Service mesh	No	Yes (Traefik)
Multi-node scaling	Manual	Built-in orchestration
Rolling updates	Manual	Automatic
Health checks	Manual	Automatic restart
Helm charts	No	Yes
Complexity	Low	Medium

Use Docker Compose for 1-3 services. Use K3s for 5+ services or multi-node clusters.

Step 1 — Prepare System

Update packages:

sudo apt update && sudo apt upgrade -y

Disable memory limits cgroup for proper Kubernetes operation:

sudo nano /boot/firmware/cmdline.txt

Find the line starting with console= and append at the end:

cgroup_memory=1 cgroup_enable=memory

Reboot:

sudo reboot

Step 2 — Install K3s

K3s provides an install script:

curl -sfL https://get.k3s.io | sh -

This installs:

• k3s server (control plane + worker on single-node)
• kubectl CLI
• systemd service for auto-start

Verify installation:

sudo k3s kubectl get nodes

Expected output:

NAME          STATUS   ROLES                  AGE
raspberrypi   Ready    control-plane,master   2m

Step 3 — Configure kubectl

Copy kubeconfig to user home:

mkdir -p ~/.kube
sudo cp /etc/rancher/k3s/k3s.yaml ~/.kube/config
sudo chown $USER:$USER ~/.kube/config
chmod 600 ~/.kube/config

Verify access:

kubectl get nodes
kubectl get pods --all-namespaces

Step 4 — Deploy First Application

Create a simple Nginx deployment:

kubectl create deployment nginx --image=nginx:latest

Expose via service:

kubectl expose deployment nginx --port=80 --target-port=80 --type=NodePort

Get the port:

kubectl get service nginx

Example output:

NAME    TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)
nginx   NodePort   10.43.100.50    <none>        80:30123/TCP

Access via http://pi-ip:30123.

Step 5 — Deploy via YAML Manifest

Create a manifest file:

nano nginx-deploy.yaml

Paste:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.26-alpine
        ports:
        - containerPort: 80
        resources:
          requests:
            memory: "64Mi"
            cpu: "100m"
          limits:
            memory: "128Mi"
            cpu: "200m"

---
apiVersion: v1
kind: Service
metadata:
  name: nginx-service
spec:
  selector:
    app: nginx
  type: NodePort
  ports:
  - protocol: TCP
    port: 80
    targetPort: 80
    nodePort: 30080

Deploy:

kubectl apply -f nginx-deploy.yaml

Check status:

kubectl get deployments
kubectl get pods

Step 6 — K3s Built-in Features

Traefik Ingress Controller

K3s includes Traefik for ingress. Create ingress:

cat <<EOF | kubectl apply -f -
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-ingress
spec:
  rules:
  - host: nginx.local
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: nginx-service
            port:
              number: 80
EOF

Access via http://nginx.local (after DNS/hosts entry).

Storage

K3s bundles local-path-provisioner for persistent volumes:

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: data-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
EOF

Data stored in /var/lib/rancher/k3s/storage/.

Step 7 — Install Helm Charts

Helm packages pre-configured apps. Install Helm:

curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash

Add repository:

helm repo add bitnami https://charts.bitnami.com/bitnami
helm repo update

Install a chart (e.g., MariaDB):

helm install my-mariadb bitnami/mariadb \
  --set auth.rootPassword=changeme \
  --set primary.persistence.enabled=false

List installed releases:

helm list

Step 8 — Resource Monitoring

View resource usage:

kubectl top nodes
kubectl top pods --all-namespaces

If metrics unavailable, install metrics-server:

kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

Step 9 — Useful kubectl Commands

View logs

kubectl logs <pod-name>
kubectl logs -f <pod-name>    # Follow

Exec into container

kubectl exec -it <pod-name> -- /bin/sh

Port forward (local access)

kubectl port-forward svc/nginx-service 8080:80
# Access via http://localhost:8080

Scale deployment

kubectl scale deployment nginx --replicas=3

Rolling update

kubectl set image deployment/nginx nginx=nginx:1.27-alpine

Multi-Node Cluster (Optional)

To add worker nodes, get the token from master:

sudo cat /var/lib/rancher/k3s/server/node-token

On worker Pi, install K3s agent:

curl -sfL https://get.k3s.io | K3S_URL=https://master-ip:6443 K3S_TOKEN=<token> sh -

Verify on master:

kubectl get nodes

Resource Usage

Metric	Single-Node
RAM (idle)	~500 MB
RAM (with workload)	~800 MB - 1.2 GB
Disk (system)	~500 MB
CPU (idle)	<5%

Limit resources in pod specs to prevent runaway containers.

Troubleshooting

Pods won't start

Check events: kubectl describe pod <name>. Look for resource requests exceeding available memory.

Service unreachable

Verify service and endpoints: kubectl get svc and kubectl get endpoints. Check firewall for NodePort range (30000-32767).

K3s service won't start

Check journal: sudo journalctl -u k3s -f. Common issue: incompatible kernel (enable cgroup memory).

Metrics-server pending

Some Pi images lack required kernel features. Check: kubectl describe pod -n kube-system metrics-server-*. Falls back to manual monitoring if unavailable.

Summary

K3s brings Kubernetes to Pi without the overhead. Start with single-node for learning, scale to multi-node when ready. Use Helm for faster deployments, and monitor with kubectl top to avoid memory exhaustion. For small homelabs, Docker Compose remains simpler; switch to K3s when you need automatic restarts, rolling updates, or service discovery across multiple machines.