A thing that snuck up on me
That DevOps wishlist
you never got to?
It's the default now.
Every team I've been on kept a doc of infrastructure we'd build "when things calm down." Preview environments. Real end-to-end tests. Migrations checked both directions. Things calmed down exactly never. Then the price of building all of it fell through the floor, and now my weekend projects run the setup I couldn't get funded for products with actual revenue.
The list every team has
You know this list. It lives in a Confluence page called something like "Platform Improvements (Draft)." Here's the version I carried around for a decade.
- A fresh copy of the app for every pull request
- End-to-end tests on every commit, phone and desktop
- Database migrations replayed from zero before they touch prod
- Seeded test data that looks real without being anyone's real data
- Team conventions enforced by the build, not by a wiki nobody reads
- Permissions proven, not assumed
- Screenshots of the actual UI attached to every review
- Error triage that beats the customer to the ticket
- Dependency upgrades landing weekly instead of never
- A staging environment that actually matches production (still open)
Nine of ten. Some things are eternal.
Every one of those checkmarks is real, and most of them are running on side projects. Not a platform team. Not a quarter of roadmap. A guy at a kitchen table with a running app and a tabletop dice calculator.
Why it never happened
It wasn't laziness and it wasn't ignorance. Everybody on those teams knew exactly what good looked like. We could describe the pipeline in detail. We just couldn't pay for it.
Each line on that list was a week of the one person who actually understood the YAML, and that person was the same person the roadmap needed for the feature that had a date attached. So preview environments lost to the feature. Every quarter. Honestly? They should have. No sane manager slips a customer commitment to go build flaky-test quarantine.
And the list didn't hold still. It picked up new items faster than it ever lost old ones. Ten years later you've got a beautiful document, fully formatted, with a table of contents, describing a system that has never existed.
What changed is the price
Let's be clear about what didn't happen. The infrastructure did not get simpler. Kubernetes did not get kind. CI configuration is still cursed. What collapsed is the cost of producing it.
a week of the one person who knew Terraform → an afternoon and a code review
That single change reorders the whole priority list. Work that was too expensive to justify is now cheaper than the meeting where you'd argue about it. A few things I have running right now that I would never have built by hand:
A Monte Carlo test for a dice engine. ktcalc rolls a seeded hundred thousand dice inside one test and checks the simulation lands within 2% of the odds the calculator claims. Writing that used to mean going and learning enough statistics to pick a fair tolerance. I did not learn statistics. I asked for a sentence.
A throwaway database on every run. One project spins up a disposable Postgres, replays all 37 migrations from empty, and then fires 39 checks proving one user cannot read another user's rows. Migration confidence and tenant isolation, both, on a project with zero budget.
Conventions that argue back. Pheidi has a
test whose only job is to read the rest of the source and
fail the build if anybody writes DateTime.Now.
That's a whole family of timezone bugs, dead, permanently,
for about twenty minutes of work.
Proof you can look at. The end-to-end suite drives the real app on a desktop viewport and a phone one, records a screenshot at every step plus a video, and posts them into the pull request. I glance at the pictures before I merge. A test can pass while the page renders as three sad boxes. A screenshot can't hide that.
Fussy details I'd normally punt. This very site has a scheduled function that pings a Discord server about game night. It fires twice and gates on the local Eastern hour so it posts exactly once, correctly, across daylight saving. That is precisely the sort of thing you shrug at and let break twice a year.
None of that is exotic. That's the point. It's the boring stuff off the wishlist, finally affordable.
The bill still comes, just later
Cheap to build is not the same as free to own. Two ways this goes wrong, and I've done both.
Infra you didn't design is infra you can't debug. When the pipeline you generated in an afternoon breaks at 6pm on a Friday, you are reading it for the first time. The fix is not to write less of it. It's to make the agent explain the thing it just built, in your repo's own words, and to keep the pieces small enough that you can hold one in your head.
Noise is the same as nothing. Forty checks that are always a little yellow teach the team to stop looking, and a dashboard everybody has learned to ignore is worse than no dashboard, because it feels like coverage. So the rule I keep: every automated thing has to say something you would actually act on. If it can't, delete it.
Good news is the same news as before. Pruning got cheap too. The thing that used to take a week to build now takes an hour to rip out and replace, which means you're finally allowed to be picky about it.
Where I'd push this next
Everything above is just the old wishlist, finally affordable. The interesting part is what you build once you stop treating infrastructure as a thing you ration. Here's where I'd take it.
1. Infra that notices its own gaps. Right now a human decides that a module needs tests. That's a scheduled job: walk the codebase, find the route with no end-to-end coverage, the table with no isolation check, the job with no alert, and open the pull request that fills the hole. Coverage stops being a number on a report and starts being a queue of specific proposed diffs.
2. Review the running app, not the diff. Every pull request gets a live copy, and an agent drives it like a user, then writes up what it saw in plain English. "Signup works, the plan page renders, the date picker on mobile is now under the keyboard." I'd rather read that than 600 lines of diff. Especially when I didn't write the 600 lines.
3. Every incident becomes a permanent
check. The postmortem action item that nobody
does is "add a test for this." Make that the automatic
part: the outage ends when a check exists that would
have caught it. Same with review comments. Any note you
find yourself writing twice should graduate into a rule
the build enforces, like that DateTime.Now
guard. Your repo grows a memory instead of relying on
whoever's been there longest.
4. Give the night shift the runbook. I already have a job that reads production errors and opens fix pull requests while I sleep. The next step is the operational half: flip the flag, roll back the bad deploy, scale the thing up, with a human approval gate in front of anything that touches money or data. Not "self-healing" in the magic-brochure sense. More like a capable on-call who wakes you only when it matters.
5. Debug from evidence, not vibes. Hand the agent the same things a good engineer would open: logs, traces, deploy history, feature flag state, the last ten commits. MCP makes that plumbing boring. An agent guessing at a stack trace is a party trick. An agent that can check which deploy the error started after is a colleague.
6. The migrations nobody ever schedules. Framework upgrades, deprecation sweeps, the library that's three majors behind. These never get done because they're enormous, boring, and nobody gets promoted for them. That's the perfect standing overnight job: one upgrade at a time, tests and screenshots attached as proof, ready to merge or throw away in the morning.
7. Budgets as gates, everywhere. Color contrast, bundle size, p95 latency, query counts per request, cloud spend per environment. Any team could have written these checks ten years ago. Nobody had a spare week per check. Now each one is an hour, so the real question is which ones you want, not which ones you can afford.
What this does to the job
If a platform team's value was building the paved road, that value just got a lot cheaper to reproduce. The scarce thing is deciding what belongs on the road, reading what the agents built, and killing the pieces that generate noise instead of signal. Curation, not construction.
That's a better job, not a smaller one. Twenty years of watching pipelines rot teaches you which checks people actually respect and which ones they route around by Thursday. That taste used to be worth very little, because you never had the hours to act on it. Now it's the only input that matters, because the hours are handled.
So go reopen the doc. The one with the table of contents. Most of it is a Tuesday now.
Related reading, since I clearly can't stop: everyone ships 90%, and I spend half my build time proving it works.