Skip to content

Kubernetes (K3s + Rancher)

Deployed Versions

Component Version Notes
K3s v1.34.6 Server on Hub, agent on DMZ + Beast
Cilium 1.17.4 CNI, kube-proxy replacement, Hubble
Rancher 2.14.0 Community Edition, single replica
cert-manager 1.17.1 Prerequisite for Rancher TLS
Fleet 0.15.0 GitOps, bundled with Rancher

K3s Installation

K3s is installed via the official install script with specific flags to harden and customize the cluster.

Server Node (Hub)

curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION="v1.34.6+k3s1" \
    INSTALL_K3S_EXEC="server" sh -s - \
    --flannel-backend=none \
    --disable-network-policy \
    --disable=traefik \
    --disable=servicelb \
    --tls-san=91.98.121.97 \
    --tls-san=10.0.1.1 \
    --tls-san=10.0.2.1 \
    --node-ip=10.0.1.1 \
    --advertise-address=10.0.1.1 \
    --cluster-cidr=10.42.0.0/16 \
    --service-cidr=10.43.0.0/16 \
    --kube-apiserver-arg="audit-log-path=/var/log/k3s-audit.log" \
    --kube-apiserver-arg="audit-log-maxage=30" \
    --kube-apiserver-arg="audit-log-maxbackup=3" \
    --protect-kernel-defaults=true \
    --secrets-encryption

Key flags explained:

Flag Purpose
--flannel-backend=none Disable Flannel; Cilium will be installed instead
--disable-network-policy Disable K3s built-in policy; Cilium handles this
--disable=traefik Disable Traefik; Caddy on DMZ handles ingress
--disable=servicelb Disable ServiceLB; not needed
--tls-san Add SANs for public, private, and WireGuard IPs
--node-ip Advertise private IP for inter-node communication
--secrets-encryption Encrypt secrets at rest in etcd
--protect-kernel-defaults Enforce required kernel parameters

Agent Nodes (DMZ, Beast)

curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="agent" sh -s - \
    --server=https://10.0.1.1:6443 \
    --token=<node-token> \
    --node-ip=10.0.1.x \
    --node-label="lron/role=dmz"   # or "lron/role=beast"

Node labels

Each node gets a lron/role label (hub, dmz, beast) used by node selectors and tolerations to pin workloads to the correct VM.

Cilium Installation

Cilium 1.17.4 is installed via Helm after K3s server is up but before agent nodes join:

helm repo add cilium https://helm.cilium.io/
helm install cilium cilium/cilium \
    --version 1.17.4 \
    --namespace kube-system \
    --set operator.replicas=1 \
    --set tunnel=vxlan \
    --set hubble.enabled=true \
    --set hubble.relay.enabled=true \
    --set hubble.ui.enabled=true \
    --set ipam.operator.clusterPoolIPv4PodCIDRList=10.42.0.0/16 \
    --set bpf.masquerade=true \
    --set kubeProxyReplacement=true \
    --set k8sServiceHost=91.98.121.97 \
    --set k8sServicePort=6443

k8sServiceHost must be the real Hub IP

Setting k8sServiceHost=127.0.0.1 works on the server node but crashes Cilium init on agent nodes (DMZ, Beast). The agent's localhost is not the API server. Always use the Hub's actual IP (public or private, depending on network topology). See ADR-015.

Feature Status
VXLAN tunnel Enabled (private network underlay)
Hubble Enabled (network flow visibility)
Hubble UI Enabled (accessible via WireGuard)
kube-proxy replacement Enabled (eBPF-based)
BPF masquerade Enabled
NetworkPolicy CiliumNetworkPolicy CRDs

Rancher Installation

Rancher CE 2.14.0 is installed via Helm on the Hub node. cert-manager must be installed first -- Rancher depends on it for TLS certificate management.

Step 1: cert-manager (prerequisite)

helm repo add jetstack https://charts.jetstack.io
helm repo update

# Install cert-manager v1.17.1 with CRDs
helm install cert-manager jetstack/cert-manager \
    --version v1.17.1 \
    --namespace cert-manager \
    --create-namespace \
    --set crds.enabled=true

cert-manager must be healthy before Rancher install

Verify all three cert-manager pods are running before proceeding: kubectl get pods -n cert-manager. If cert-manager webhooks are not ready, the Rancher Helm install will fail with admission webhook errors.

Step 2: Rancher

helm repo add rancher-latest https://releases.rancher.com/server-charts/latest
helm repo update

helm install rancher rancher-latest/rancher \
    --version 2.14.0 \
    --namespace cattle-system \
    --create-namespace \
    --set hostname=rancher.vdhome.be \
    --set replicas=1 \
    --set ingress.tls.source=letsEncrypt \
    --set letsEncrypt.email=dev@vdhome.be

Single replica

Rancher runs as a single replica -- this is a personal lab, not HA. Acceptable for the use case.

Rancher 2.14.0 provides:

  • Cluster management UI
  • Fleet 0.15.0 GitOps controller
  • RBAC management
  • Cluster monitoring integration
  • Backup/restore via rancher-backup operator

Fleet GitOps Structure

Fleet watches the fleet/ directory in the GitLab repository and auto-deploys changes:

fleet/
  ingress/
    fleet.yaml          # Bundle targeting DMZ node
    caddy-config.yaml
    authelia-config.yaml
    ttyd-deployment.yaml
  monitoring/
    fleet.yaml          # Bundle targeting Hub node
    victoriametrics.yaml
    grafana.yaml
    loki.yaml
    alloy.yaml
  dev/
    fleet.yaml          # Bundle targeting Beast node
    sample-workloads.yaml

Each fleet.yaml specifies:

  • Target cluster: lron-local
  • Target namespace: workload-specific
  • Node selector: matches lron/role label
# Example fleet/ingress/fleet.yaml
defaultNamespace: ingress
targets:
  - clusterSelector:
      matchLabels:
        management.cattle.io/cluster-name: local
helm: {}

Namespace Isolation

Cilium NetworkPolicies enforce namespace boundaries:

# Deny all ingress/egress by default in each namespace
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: default-deny
  namespace: dev
spec:
  endpointSelector: {}
  ingress: []
  egress: []
---
# Allow dev pods to reach internet (outbound only)
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: allow-egress-internet
  namespace: dev
spec:
  endpointSelector: {}
  egress:
    - toEntities:
        - world
    - toEndpoints:
        - matchLabels:
            io.kubernetes.pod.namespace: kube-system
            k8s-app: kube-dns
      toPorts:
        - ports:
            - port: "53"
              protocol: UDP

Namespace layout:

Namespace Node Purpose Internet Ingress
cattle-system Hub Rancher No (WireGuard only)
cattle-fleet-system Hub Fleet controller No
monitoring Hub VictoriaMetrics, Grafana, Loki No (WireGuard only)
ingress DMZ Caddy, Authelia, ttyd Yes (443/tcp)
dev Beast Dev workloads No
kube-system All Cilium, CoreDNS, metrics-server No