A thing I believe about building with AI
I let the robot write it.
Then I make it prove
it actually works.
The code coming out of Claude is better than anything I'd write by hand, and it reviews better too. (Sorry to my own ego.) The catch: it shows up 100x faster than I can read it. So the job quietly changed. It's not writing anymore. It's trusting. And trust is something you build, on purpose, with at least half your time.
760
automated checks guarding one app
100k
dice rolls inside a single test
2screens
every end-to-end run checks
nightly
a robot opens bug-fix PRs while I sleep
The test that rolls the dice 100,000 times
ktcalc is a combat calculator for a tabletop game. The whole app answers one question: if I take this shot, what happens? And the honest answer is never a number. It's odds.
That breaks the normal way you'd test it. "The function returned 4" tells you nothing when the right answer was "4, about 38% of the time." A tidy little unit test can walk down one happy path, go green, and the actual probabilities underneath can be quietly, confidently wrong. Green tests, wrong app. The worst kind of wrong.
So one test doesn't check a path. It rolls the dice a hundred thousand times. Seeded, so it's the same hundred thousand rolls on every run, no flakes. Then it checks that the simulated reality lands within 2% of the math the calculator is claiming. If the engine ever drifts off the truth, that test lights up before anybody plays a game on bad numbers.
100,000 rolls vs. the claimed odds → within 2%
Now sit with how that test would've gotten written a few years ago. A seeded random generator, a hundred thousand trials, the statistics to know what tolerance is even fair. That's a real afternoon for someone who knows how. I'm not sure I'm that someone. I'd have to go read up on it, and then I probably still wouldn't bother. Too much yak-shaving for a side project.
With AI it's a sentence. "Write me a Monte Carlo test that validates the dice engine within 2%." Done, correct, seeded, before I finish my coffee. The verification that used to be too expensive to justify is now the cheap part. That changes what you're allowed to check.
That's not a unit test. That's proving the thing is true. And once you've felt the difference, "did it return a value" stops feeling like testing at all.
Unit tests are the floor, not the ceiling
Unit tests are table stakes. Fine. Necessary. But the checks that actually keep me safe are weirder than "call function, assert output," and they're where the real verification lives.
Rules you enforce as code. Pheidi has a
test whose entire job is to read the rest of the source
and fail the build if anyone writes
DateTime.Now. Sounds petty. It kills an
entire family of timezone bugs before they can exist. The
convention isn't a wiki page nobody reads. It's a test
that can't be argued with.
Security as a test, not a hope. Auspex Fatalis spins up a throwaway Postgres in memory, runs all 37 database migrations, and then fires 39 checks proving one user literally cannot read another user's data. "The permissions are probably fine" is now a thing the machine confirms, every run, instead of a thing I pray about at 2am.
And here's the part that should embarrass an entire industry, me included: this is a brand new side project, and it has better test coverage than production systems I've babysat for ten years. A weekend app, more trustworthy than software that runs real money. That's not a flex about Auspex Fatalis. It's how cheap this got.
The whole app, on two screens. Pheidi's end-to-end suite drives the real app like a user would, then does it again in a phone-sized viewport. Desktop and mobile, same proof, no "oops, looked great on my monitor."
Proof you can actually look at. Because those tests drive the real app, they record it: a screenshot at every step, plus a little video, posted straight into the pull request. A green check tells me the code ran. The screenshot tells me the button is where it's supposed to be and the page didn't render as three sad boxes. A test can pass and still look broken. A picture in the PR can't hide it. I open the PR, glance at the shots, and I can see if anything broke before I merge a thing.
Build the watchdog once. It works forever.
Here's the part I'd have called sci-fi two years ago, and it's just running in prod now. Verification doesn't have to stop when you ship. You can build a thing that keeps checking while you sleep, and it costs almost nothing to stand up.
A job wakes up, reads the last 24 hours of real production errors, and dedupes them so I don't get spammed about the same hiccup twice. Then it does the part that still makes me grin: for anything new, it doesn't file a ticket for future-me. It writes the fix and opens a pull request. Right when the bug happens. I wake up to a proposed patch, not a bug report.
So my error logs are clean. Not because nothing ever breaks. Things break constantly. They're clean because the bugs get caught and fixed while I sleep, and I review the PR over coffee instead of getting paged at 3am. Same when a test goes red in CI: an agent reads the failure, finds the root cause, and hands me a diagnosis instead of a red X and a shrug.
Why half your time. Not a tenth.
Because the math flipped. When I wrote all the code, I read all the code, so writing was the bottleneck. Now the model hands me more in an afternoon than I could carefully read in a week. Reading every line myself isn't a plan. It's a fantasy.
Verification is the one lever that scales with the size of the pile. A convention guard checks a rule across a thousand files as easily as ten. A hundred-thousand-roll test doesn't care that I generated the engine in five minutes. The checking machine keeps up with the writing machine, and I get to stay the person who decides what "correct" even means.
Building features is the fun half. The easy half, honestly. Skip the other half and you don't have a fast team. You have a fast pile of code nobody can trust, which is slower than slow. Every feature you didn't verify is a bug you'll re-debug later, at ten times the price.
Yes, writing tests used to be the chore you skipped. Now the same tools that write the app write the tests too, if you bother to ask. Most people won't bother. That's the opening.