jf

NightForge × every repo

Three robot sweeps
drifted. So I built
a factory.

Each of my projects had a nightly robot: read the error logs, figure out what broke, open a fix. I wrote each one by hand. In about two months they had drifted into three different pipelines, and one of them was quietly writing the same report every night. NightForge is the one version they all run now.

3

hand-written sweeps, three different pipelines

24KB

nightly report of the same old findings

1

pipeline they all share now

1 line

report on a quiet night

How it drifted

One sweep filed issues and opened pull requests. One only filed issues. One was a single sentence with no memory at all, so every night it rediscovered its whole backlog from scratch and wrote it all down again. Its reports grew to 24 KB of findings I'd already read.

None of them were wrong, exactly. They'd just each learned different lessons, and a fix in one never reached the others. (Robots, it turns out, also don't read each other's Slack.)

One pipeline, a card per project

The pipeline is the same everywhere. Each stack (Netlify, Supabase, Azure App Insights) gets an adapter. Each project gets a short card with its names and quirks. Adding a project is one card. A pipeline fix is one edit every project gets.

  1. Collect

    Pull the night's errors

    From whatever the project runs on, through that stack's adapter.

  2. Dedupe

    Check the ledger and the issue tracker

    Including closed issues. On the very first live run, an error that was new to the empty ledger turned out to be fixed already, by a PR merged fourteen minutes after the error last fired. That one check stopped a junk issue and a fix agent aimed at a fix that already shipped.

  3. Triage

    Read it against the real source

    Not the stack trace's opinion of the source. The actual code on the default branch.

  4. Fix

    File it, then hand it to a fix agent

    The agent works in its own worktree and opens a pull request. An issue only closes when a merged, deployed PR and quiet error logs both say so.

Every rule was paid for

A sweep that only watched for exceptions was blind to 404s. One app served them to a live subscriber for 21 hours while every scheduled run reported success.

An Azure CLI default quietly narrowed a query to the last hour, even though the query asked for a week. One project ran green every day for a week while missing six of its eight live problems.

For a month the sweeps wrote what they learned back into their own instructions. The cards passed 100 KB. Now lessons go to a database, and the skill only changes when a human edits it.

The rest of the night shift

The error sweep came first. The rest followed the same shape: one skill, a short card per project.

  • docs-sweep · weekly Finds every repo with its own sync-docs audit, runs it, and opens a draft PR where the docs drifted from the code. sync-docs itself was born in Pheidi and ported everywhere else. Pheidi, Merge & Tell, Auspex Fatalis, and NightForge all run their own copy.
  • coderabbit-sweep · hourly I get a limited number of CodeRabbit reviews, so this spends the next one on the pull request across all my repos that needs it most. Its rules come from real Pheidi and Merge & Tell PRs.
  • ci-cost-sweep Measures where CI minutes go, cuts them without cutting coverage, and proves the saving on real runs before claiming it.
  • pr-test Runs a PR's test plan in a real browser, ticks the boxes, and posts the screenshots.
  • unslop Cuts the AI tells out of writing, so the robots don't sound like robots.

NightForge is open source and public domain. It's a pile of Markdown and a little Python, and it has been running my repos since August. Take it: github.com/jfreal/nightforge ↗. Two commands install the lot. See how it plugs into the rest on the stack.

← The stack Next: Merge & Tell × Pheidi →