guest@afavaccio: ~/blog

guest@afavaccio:~/blog$ cat homelab-kubernetes-on-a-mac-mini.md

A Kubernetes homelab on a Mac mini

tags: homelab, kubernetes, gitops

I am building a home server: a three-node Kubernetes cluster on a Mac mini (M4, 16 GB), created entirely from code and managed with GitOps. The repository is hyesr/homelab. It is at an early stage, and this post covers what runs today and why it is built the way it is.

The stack

LayerToolRole
Virtual machinesLimaThree plain Ubuntu arm64 VMs that can reach each other
ProvisioningAnsibleInstalls and configures K3s on the VMs
KubernetesK3sOne control plane and two workers
GitOpsFluxReconciles the cluster from the repository
TLScert-managerInternal certificate authority for the apps
First appHomepageA dashboard for the homelab

Each VM has 2 vCPUs, 2 GiB of RAM and a 20 GiB disk. The whole cluster comes up with one command:

make up                        # create the VMs, install K3s
eval "$(make -s kubeconfig)"   # point kubectl at the cluster
kubectl get nodes              # three nodes, all Ready

Real nodes, one machine

I wanted a real multi-node cluster, where each node is a full operating system, and not containers pretending to be nodes. Lima runs arm64 Ubuntu VMs natively on Apple Silicon and puts them on a network where they reach each other by name.

The trade-off is plain: it is one physical machine, so there is no real high availability. Three nodes teach scheduling and failure handling, but they do not survive a hardware failure.

Lima creates, Ansible installs

Lima templates can install K3s on their own, and that would be quicker. I kept Lima to creating plain Ubuntu VMs and gave the installation to Ansible, so the same playbooks can later run unchanged on a mini PC or a VPS. It is more code, and I maintain the install logic myself.

The workers join with an agent-only token that the server generates and the playbook reads at run time. It is never stored in Git, and unlike the server token it cannot be used to take over the cluster.

Flux without flux bootstrap

Flux fetches the repository every minute and applies what is under clusters/homelab/. Its own manifests live there too, so upgrading Flux is a pull request like any other change.

I did not use flux bootstrap. It needs a personal access token that can push to the repository, and it commits straight to main, which my branch rules reject: every change goes through a pull request and a lint check. So I generated the same manifests with the Flux CLI, merged them through a pull request, and created the SSH key inside the cluster. GitHub holds only the public half, as a read-only deploy key.

The cost is a first install of a few manual commands instead of one, and a new deploy key after every rebuild, because the private key exists only in the cluster.

An internal CA that can only sign home.arpa

The apps use home.arpa, the domain reserved for home networks. No public CA issues certificates for it, so cert-manager runs a CA inside the cluster.

A device that trusts a CA accepts whatever it signs, and this CA’s key is a Secret in the cluster. To limit the damage if the cluster were compromised, the CA carries X.509 name constraints: home.arpa only, no IP addresses. I tested it with OpenSSL and the macOS verifier: dashboard.home.arpa passes, github.com and an IP address are rejected.

The first app

The first app is Homepage, a start page for the homelab. It is stateless, so it needs no persistent volume or backup, yet it exercises the whole path: Flux, Ingress through Traefik, a certificate from the internal CA and a NetworkPolicy. New apps add themselves to it with annotations on their Ingress.

Its login is off for now, because the password would be the first secret in the repository and secret encryption is not set up yet. Until then the dashboard is reachable only from the Mac mini itself.

What comes next

The roadmap is ticked only for things that actually run. Still open:

  • Reach the apps from the LAN
  • Pi-hole
  • Secrets with SOPS and age
  • Backups of the persistent volumes
  • Lightweight monitoring
  • Gateway API instead of Ingress
  • A kubeadm-based cluster built the same way, to compare with K3s

Every choice above, with its trade-offs, is recorded in docs/decisions.md.