I'm Betting My Entire Stack on One Linux Distro. No Turning Back.
In this article
> **Bottom line:** After a 2024 config-drift incident took down three of our staging environments in the same week, I migrated our entire infrastructure — CI runners, Kubernetes nodes, and my own laptop — to NixOS over five months in 2025.
The result: zero "works on my machine" tickets since January 2026, and our mean-time-to-rebuild-a-broken-node dropped from 40 minutes to under four.
The tradeoff is real: NixOS has a brutal learning curve and a smaller hiring pool.
If your team is drowning in environment drift, the bet is worth it — but go in knowing you're signing up to write Nix expressions for a while before it pays off.
I deleted our Ansible playbooks. All 14,000 lines of them.
Then I told my team we were rebuilding every server, every CI runner, and every dev environment on a single Linux distribution most of them had never heard of — and if it didn't work, it was on me.
That was February 2025. I want to tell you it was a calculated, low-risk decision backed by a spreadsheet. It wasn't.
It was the decision of someone who'd spent three years patching over the same category of bug on a Friday night and finally snapped.
The Config Drift That Broke Three Environments in One Week
Here's the confession part.
For years I ran what I'd now call "distro tourism" — Ubuntu on the web tier because it had the best package support, Alpine in containers because the images were small, CentOS on a couple of legacy database boxes nobody wanted to touch, and Debian for anything in between.
Every choice was locally justified. Collectively, it was a disaster waiting for a trigger.
In November 2024, the trigger arrived. A routine `apt upgrade` on one staging box pulled in a newer OpenSSL than what was baked into our container base images.
Three services that had been working for months started failing TLS handshakes — but only in staging, and only on Tuesdays and Thursdays when our nightly rebuild job ran.
It took us four days to figure out the images and the hosts had quietly diverged.
Four days, across three environments, chasing a bug that didn't exist in any single file — it existed in the *gap* between two machines that were supposed to be identical and weren't.
That's the part nobody tells you about distro sprawl. It's not that any one distro is bad.
It's that "close enough" package versions across a fleet is a silent, compounding liability, and you don't feel it until the week it costs you.
Why I Bet Everything on NixOS
The contrarian take I'd push back on: the fix isn't better documentation or a stricter runbook. Documentation drifts too.
The fix is making drift *structurally impossible*, and that's the one thing NixOS actually does differently from every other distro I've run in production.
NixOS builds your entire system — packages, services, kernel config, even user accounts — from a single declarative file, and it hashes every dependency into the build.
Change one version number, and you get a new, isolated system generation. Nothing is "upgraded in place." Nothing silently drifts, because there's no mutable state to drift from.
I call the approach **the Convergence Bet**, and it rests on three ideas:
1.
**One source of truth, not a fleet of similar-but-not-identical machines.** A `flake.nix` file defines a server the same way a Dockerfile defines a container — except it covers the whole OS, not just the app layer.
2. **Reproducibility is a build property, not a discipline.** You don't need engineers to remember to pin versions. The build fails if it can't reproduce exactly.
3. **Rollback is a first-class operation.** Every deploy creates a new generation you can boot back into in seconds — no snapshot restore, no re-provisioning.
Here's roughly what one of our web-tier node configs looks like now:
```nix { config, pkgs, ... }: { services.nginx.enable = true;
environment.systemPackages = [ pkgs.nodejs_20 pkgs.postgresql_16 ]; system.stateVersion = "24.11"; } ```
That file is the entire server. Every teammate who runs `nixos-rebuild switch` on it gets byte-identical results, whether it's their laptop, a CI runner, or the production box.
There's no "it worked on staging" gap left to hide in.
The Migration Framework We Actually Used
I didn't rebuild the whole stack at once — that's the fastest way to torch a team's trust in you. We ran it in four phases over five months:
- **Phase 1 — Contain (weeks 1-3):** New CI runners only. Lowest blast radius, fastest feedback loop.
- **Phase 2 — Convert (weeks 4-10):** Staging environments, one service at a time, with the old Ansible-managed box kept alive as a fallback for two weeks after each cutover.
- **Phase 3 — Commit (weeks 11-16):** Production web tier and worker nodes, cut over during our lowest-traffic window.
- **Phase 4 — Consolidate (weeks 17-20):** Legacy database hosts last, because they had the least room for error and the most institutional fear attached to them.
The fallback window in Phase 2 mattered more than anything else in that list.
Twice we caught a misconfigured systemd service before it hit production, because we could diff the old box against the new one line by line.
What the Data Actually Says
I'm wary of vibes-based infra claims, so we tracked this. Comparing the six months before the migration to the six months after:
- **Environment-drift incidents:** 11 → 0 - **Mean time to rebuild a failed node:** ~40 minutes → under 4 minutes (`nixos-rebuild` from a known-good generation, versus reprovisioning through Ansible and hoping package mirrors hadn't shifted) - **"Works on my machine" tickets:** roughly 2-3 a month → zero since January 2026 - **Onboarding time for a new engineer's dev environment:** 1.5 days → about 45 minutes
That last number surprised me the most.
A new hire runs one command against our shared flake and gets the exact toolchain — same compiler version, same linter, same Postgres build — as everyone else on the team, on day one.
Where People Push Back — and They're Not Wrong
I'd be lying if I said this was a clean win with no cost. The engineer on our team who resisted hardest, our lead SRE, made a point I still think about: Nix's learning curve isn't steep, it's a cliff.
The language is functional, the error messages are dense, and StackOverflow-style answers for it are thinner than for Ubuntu or Debian.
She was right. It took most of the team six to eight weeks before they stopped needing me to review every Nix expression.
Hiring is the other real cost — you're pulling from a smaller pool of engineers who already know it, or budgeting real ramp-up time for the ones who don't.
There's also a philosophical tension worth naming: NixOS optimizes hard for reproducibility at the expense of the "just SSH in and fix it" workflow a lot of ops culture is built around.
If your team leans heavily on ad-hoc production debugging, that muscle atrophies fast, and you have to relearn how to debug through generations and rebuilds instead.
What This Means If You're Weighing the Same Bet
If your incidents trace back to package version mismatches, "it worked in staging," or a runbook nobody trusts anymore, the underlying disease is drift — and NixOS treats the disease, not the symptom.
If your incidents are about scale, latency, or application logic, a distro migration will cost you five months and solve nothing. Know which problem you actually have before you delete your playbooks.
Start where the blast radius is smallest — CI runners, not production databases. Keep a fallback path alive for at least two weeks past every cutover.
And budget real calendar time, not sprint points, for the learning curve; ours took closer to two months per engineer than the "one weekend tutorial" the YouTube demos make it look like.
I'm not going back to mixed distros.
Not because NixOS is perfect — it isn't, and I still curse at eval errors more than I'd like to admit — but because I now know exactly what state every machine we run is in, at every point in time, and I've never been able to say that before in a decade of shipping infrastructure.
Have you made a similar bet on a single tool or platform for your whole stack — and did it pay off, or did you quietly walk it back? I'd genuinely like to know what broke for you.
---
**Marcus Webb** — Infrastructure engineer turned tech writer. Writes about AI, DevOps, and security.
