Lab 4 · GitOps with Argo CD

#

The cluster becomes a mirror of a Git repo — Kubernetes From The Ground Up

#

David Chan, Claude Opus 4.8 AI-Symbiosis Research · July 2026

Labs 1–3 studied a cluster. This lab changes how you change one. Every edit so far — kubectl apply, edit, scale, patch — you typed imperatively, straight at the cluster. That works on your laptop and is a liability in a team.

The problem with kubectl-as-a-workflow

#

The GitOps idea (one move fixes all of it)

#

Make Git the single source of truth for what the cluster should look like. The desired state (YAML) lives in a repo. A controller — Argo CD — runs in the cluster and continuously:

1. watches the Git repo (desired state),
2. watches the cluster (actual state),
3. reconciles — makes the cluster match Git.

🔁 It’s the Lab 0 reconcile loop, one layer up

#

This is the same idea that runs all of Kubernetes, lifted:

Lab 0:   a Deployment reconciles  →  PODS match the spec
Lab 4:   Argo CD reconciles       →  the CLUSTER matches a Git repo

Desired-vs-actual, continuously converged — just promoted from “pods” to “the whole cluster.”

Setup

#

Install Argo CD, and point it at a repo:

kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml
# UI: kubectl port-forward -n argocd svc/argocd-server 8080:443   → https://localhost:8080
#     user: admin,  pw: kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath='{.data.password}' | base64 -d

The Git repo holds a plain manifest — a demo app declared at 2 replicas:

# deployment.yaml  (github.com/you/gitops-lab)
spec:
  replicas: 2          # ← the source of truth for how many pods should run

Then the object that ties them together — an Argo CD Application:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata: { name: gitops-demo, namespace: argocd }
spec:
  project: default
  source:      { repoURL: https://github.com/you/gitops-lab, targetRevision: main, path: . }
  destination: { server: https://kubernetes.default.svc, namespace: gitops-demo }
  syncPolicy:
    syncOptions: [ CreateNamespace=true ]

The moment it registers, Argo reports OutOfSync / Missing — Git says 2 pods, the cluster has nothing. That status line is the reconcile loop talking.

1. Sync: Git → cluster

#

Hit Sync and Argo creates the namespace, Deployment, and 2 pods. The tree goes green — Synced / Healthy. You deployed an app and never ran kubectl apply.

2. Change by commit, not by command

#

The core move — scale to 3 by editing Git, not the cluster:

sed -i 's/replicas: 2/replicas: 3/' deployment.yaml
git commit -am "Scale gitops-demo to 3 replicas" && git push

Argo detects the new commit (hit Refresh to skip the poll interval), flips to OutOfSync — and you can open the Deployment’s Diff to see replicas: 2 → 3 in black and white. Sync, and a third pod appears. The change is now version-controlled, reviewable, and auditable — git log is your change log.

3. Self-heal: you cannot drift

#

Turn on automated self-heal, then do the forbidden thing — change the cluster behind Git’s back:

syncPolicy: { automated: { selfHeal: true, prune: true } }

kubectl scale deployment gitops-demo -n gitops-demo --replicas=5   # bypass Git

Watch the tree: Argo sees the cluster (5) diverge from Git (3) and reverts it — in seconds, before the extra pods finish booting:

14:34:30  kubectl scale → 5
+6s       spec.replicas = 3     ← Argo already snapped it back to Git's value

That manual kubectl scale was overridden by the platform. The only change that sticks is one that goes through Git. No more mystery hotfixes, no snowflake clusters where someone tweaked something during an incident and nobody remembers.

4. Rollback = git revert

#

The scariest operation in ops becomes the most boring one. Undo the scale-up by reverting its commit:

git revert HEAD --no-edit && git push

Two things make this beautiful:

• The revert is itself a new commit — your history shows the scale-up and the rollback. The audit trail records not just what changed but that you un-changed it, when, and why.
• With self-heal on, you don’t even sync — Argo sees Git now says 2, and rolls the cluster back on its own:

push revert  →  Refresh  →  argocd=Synced, running=2   (the 3rd pod terminated, automatically)

Rolling back production was one git command. No reversing steps by hand, no remembering what the previous state was — the previous state is a commit.

What you built

#

Git repo (desired state)  ──watched by──▶  Argo CD  ──reconciles──▶  cluster (actual state)
      ▲                                                                     │
      └───────  every change is a commit: reviewed, audited, revertible ────┘

The shift is as much cultural as technical: the cluster stops being something you operate and becomes something you declare. Changes are pull requests. Rollbacks are reverts. Drift is impossible. And it’s all the same reconcile loop from Lab 0 — desired vs actual, converged forever — just pointed at the whole cluster.

Next — Lab 5 · Blast Radius: the series finale. A bad deploy and an instant rollback, a failing readiness probe, and a deliberate OOMKill — each with a five-line postmortem you can tell in an interview.