Nine months into using AI as my main work driver, review time has become the bottleneck. There’s a fix forming across senior practitioners, and a company-scale architecture question that follows.
I start most of my work days in Claude Code. Anything repetitive, anything codifiable, anything that takes more than ten minutes to do, I try to delegate to AI agents. At any given point I have five or six parallel agent sessions running, and my work is jumping between them to check their output, give direction, and then move on while they continue. As a result, I focus my limited bandwidth on the things that truly only I can do — for now.
Nine months in, the more I use it, the more I want to use it. But the math has a limit: I cannot 5x my output while 5x my review time.
The management analogy
Around January, a recurring frustration led to a realization: there should be a way to codify every time Claude makes a mistake. I hate being inefficient, and the beauty of code is that you can systematize it.
It’s the same discipline as management. When someone on your team makes a mistake, you walk through it in a one-on-one and help them internalize the lesson so the mistake hopefully doesn’t happen again. Code on top of a probabilistic machine can be more reliable than human internalization, if you know how to encode the rules into the system (rather than relying on human memory alone).
The result is a Claude Code skill I call learning-loop that I run after every session to capture the learnings. Without it, I’d be running in circles and Claude would still be making the same kind of mistakes five months from now.
A forming chorus
Not managed well, verification cost scales linearly with output, and the bottleneck shifts rather than disappears. Most responses (the wave of code-review startups, “human in the loop” at organizational scale, simply giving up on “using AI” at personal scale) relocate the cost without reducing it.
But a different response has been forming across senior practitioners, scattered through their essays and X posts over the past several months. Garry Tan calls his version “skillify”:
“My agent screwed up twice this week. Neither failure can happen again. Not because I asked nicely. Because I turned each failure into a permanent structural fix: a skill with tests that run every day, forever.”
Boris Cherny, Anthropic’s Claude Code lead, captured the same move in his public Jan 2026 Claude Code thread: “Update your CLAUDE.md so you don’t make that mistake again.” Mitchell Hashimoto and the OpenAI Codex team have made variations of it.
The move works because the harness is modifiable. By “harness,” I mean what Garry Tan calls “the program that runs the LLM.” Its four jobs: run the model in a loop, read and write your files, manage context, and enforce safety. The scaffolding has to exist somewhere: at the individual, team, or organization layer. Without it, every session starts blank and the verification tax compounds downstream.
Two levels of verification
The verification work splits into detection and upstream engineering.
Detection comes first: spot-checks, “show your work” discipline, and review at output time. These catch more mistakes. You can leverage some clever review techniques (e.g. expert or adversarial review subagents) to make review marginally faster and more thorough, but that’s not the same as improving system-level efficiency.
Upstream engineering is where compounding happens. Each captured mistake becomes a permanent rule that stops the same mistake from happening. Over time, fewer mistakes show up at the review step at all.
Five months of running the learning-loop skill I built produced one structural observation: the verification step doesn’t balloon, because most potential mistakes get engineered out at source. I spend a decent amount of time authoring the upstream engineering, but I treat it as worthwhile investments that compound downstream to help me scale my output without scaling review time.
The company-scale architecture
One of my partners at NextView and I were dividing up work of a project we’re collaborating on. Both of us use Claude Code as our main work driver, and there’s a piece of work that he was handling with Claude that was upstream of mine.
When I started building upon it, I ran a multi-pass review of my work and unexpectedly found that nine out of ten items from his portion had meaningful drift: fabricated statistics and citations that didn’t cover what they claimed to cover. My partner is smart and great at his job; the gap was the verification toolkit. Without the upstream-engineered detection layer, the mistakes would have compounded into my downstream work.
That’s the two-person-scale version. Scaled to a company, the pattern multiplies (shared agents, agent-to-agent handoffs, no centralized review bottleneck): two colleagues would hit this once; two hundred employees sharing the same set of agents will hit it constantly. The Build-vs-Buy Reset named the architectural question of which capabilities to build vs buy; verification infrastructure is now another item on that list. The default response, “human in the loop downstream,” not only does not scale your capacity to leverage AI in a meaningful way, it also makes the lowest-verification-capability person on the team the team’s effective ceiling.
This is solvable at two levels, although neither is straightforward.
At the individual level, the move is empowering and educating each builder to maintain their own learning-loop: every detected mistake becomes their permanent harness rule. At the organizational level, it’s a build-or-buy question of its own. Buy: a growing set of vendors in the verification-loop category. Build: stand up internal ownership of harness rules, mistake-codification, and the feedback path from production back into everyone’s AI stack.
Where capital is moving
Capital is starting to fund the vendor side, but two adjacent categories are worth separating. Traditional eval and observability platforms (Braintrust, Arize, LangSmith, Patronus AI) monitor and score model outputs against predefined success criteria. For general-purpose agents where defining “right” is itself the work, that’s necessary but not sufficient.
Self-improvement is the next layer up: catching in-session misfires, analyzing root causes, and retooling the prompt or harness with less human intervention than traditional eval/observability requires. Lemma claims to detect semantic failures and auto-generate prompt improvements. InsightFinder, an enterprise AIOps incumbent, has extended into automated root-cause analysis for AI workflows. Judgment Labs takes a different angle: breaking down where in an agent’s run things went wrong as a continuous-improvement layer. Braintrust’s “Loop agent” is the eval incumbent extending into the same layer. The category itself is too young to have defined winners.
Four standalones have been acquired in the last twelve months: Galileo by Cisco, Promptfoo by OpenAI, HumanLoop by Anthropic (sunset Sept 2025), and Langfuse by ClickHouse. Whether verification survives as a standalone layer or gets absorbed into the model platforms is the open question for the category.
METR, an AI-capability research org, tracks how long agentic tasks stay at the frontier; their latest update shows task time-horizons now doubling every 3.5 months, down from seven. As capability moats decay, verification-loop infrastructure is the layer that compounds in the other direction.
What compounds against the model
For most teams, the binding constraint of scaling AI-assisted output is the verification loop. The teams that will use AI well (at the individual or organizational level) are the ones whose harness gets harder to defeat every week. As capability keeps accelerating, the moat is in what compounds against it.
Previously in Ground Truth: The Confidence Gap established that AI confidence is not correlated with AI correctness, the foundation this post extends. Who Captures the Value? named verification as the scarce complement to execution; this post explores what verification infrastructure looks like in practice. The Judgment Layer decomposed unverifiable decisions into verifiable sub-decisions plus irreducible judgment; this post argues the verification work on the decomposable part needs its own architecture.