Live build log

Kubernetes homelab

Why

A Kubernetes platform built on bare metal, from the hypervisor up to the workloads — owning every layer rather than delegating it to a managed service.

Each layer is deployed by hand first, then codified. The goal is to understand how it actually works before automating it.

01

Stack

deployed in progress planned
LAYER 01

Hardware and hypervisor

Dual Xeon Silver 4116, 128 GB RAM, running Proxmox VE. Key-only SSH, deny-by-default firewall, TOTP on the root account.

DECISION: WHY AN EXTERNAL SSD

Proxmox runs from a dedicated external SSD, booted on demand. The Windows install on the internal drives stays untouched. No dual boot, no partitioning. It also means the lab is not running around the clock: every session starts with a full boot of the cluster, which makes it a reproducibility test as much as a lab.

DEPLOYED
LAYER 02

k3s cluster

Three Debian VMs provisioned from the Proxmox CLI with cloud-init. One server, two agents, SQLite datastore.

DEPLOYED
LAYER 03

Ingress and TLS

Traefik as ingress controller, cert-manager issuing and renewing Let's Encrypt certificates over the DNS-01 challenge on Route 53. The cluster stays private, but every service is reachable over HTTPS on a real domain, with an IAM user scoped to a single hosted zone.

DEPLOYED
LAYER 04

GitOps

Gitea deployed via Helm as the internal forge, ArgoCD watching a single repository as the source of truth and reconciling the cluster against it.

DEPLOYED
LAYER 05

Continuous integration

Gitea Actions enabled, a self-hosted Gitea Runner registered against the organisation. It runs as two containers — the runner and a docker:dind sidecar sharing the socket through an emptyDir volume — with per-job container isolation.

DECISION: RUNNER TOPOLOGY, AND WHAT'S NEXT

The runner currently relies on a privileged docker:dind sidecar — a deliberate, documented trade-off, since the pipeline already depends on the Docker socket. A pod-per-job runner would be the cloud-native answer, but it ships only with the Enterprise edition. The planned improvement is to move to a daemonless, rootless builder (Buildah or Kaniko): no Docker socket, no privileged container.

DEPLOYED
LAYER 06

Supply chain security

Hadolint and Grype vulnerability scanning (--fail-on critical) are integrated into the CI pipeline, ahead of the image build and push to the registry. Still to come: image signing (cosign) and a move to rootless builds (Buildah / Kaniko) to remove the Docker socket dependency.

BUILDING
LAYER 07

Observability

kube-prometheus-stack (Prometheus + Grafana) deployed via Helm and reconciled by ArgoCD, running inside the cluster.

DEPLOYED
LAYER 08

Infrastructure as Code

Terraform and Ansible, once the manual flow is fully understood.

PLANNED
02

Architecture

How the stack is built, from the disk it boots off to the cluster on top.

Proxmox VE 2 x Xeon Silver 4116, 128 GB RAM, 3 Debian VMs, 192.168.1.200 k3s cluster SQLite datastore, no HA k3s-server-1 .211, 4 vCPU, 8 GB k3s-agent-1 .212, 4 vCPU, 8 GB k3s-agent-2 .213, 4 vCPU, 8 GB External SSD Proxmox root, booted on demand Key-only SSH, deny-by-default firewall, TOTP on root

How a change reaches the cluster. CI writes to Git and to the registry, and stops there.

Gitea Manifests, source of truth writes image tag watches CI No cluster credentials Gitea Runner Privileged DinD, 2 containers Lint, build, scan Hadolint, Grype Delivery Runs inside the cluster ArgoCD Applies what Git declares Workloads Converge to declared state Registry Gitea built-in, then Harbor pushes pulls
← Back to projects