A year ago I opened a coding agent for the first time, with no real idea what it was for. It is now the thing I work inside. None of the changes along the way felt like much at the time.
On a Friday in early August last year, I decided to give coding agents a try. I installed Claude Code and spent two hours building a website that generated satirical venture-capital wisdom from NextView’s own blog posts. Then I left for a neighbor’s backyard party, excited to show what I’d made. I had vibe coded a site, and I abandoned it days later.
Every no needs an expiry date
The site needed the model to read a few hundred blog posts and make sense of them, and it couldn’t do a good job with it. It fetched pages unreliably, read the first few sentences and confidently described the rest wrong, and could only hold so much context at once. Getting from there to anything I would publish was far more work than I wanted to put in, so I stopped. The whole record of it is two saved snapshots of an afternoon’s work.
Stopping was the right call in August 2025, and what made it right was the model’s limits relative to the merit of the idea. A no built on what a model can’t do well yet (at least not without a ton of work) has a short shelf life, because capability is the thing that moves fastest. Fetching and reading a few hundred pages is unremarkable now, and I never noticed the day it stopped being hard.
A model is not a reliable witness about itself
By mid-September, six weeks into working with a coding agent, I was prototyping an AI-assisted deal evaluation system at NextView, and my session log for that week calls for “Audit Trail: Complete traceability for fiduciary responsibility.” The next task in the same document is to plan how to set up version control. Between the agent and me, we were specifying bank-grade requirements on top of a plan to learn how to save my own work.
I had no way to judge whether any of that was the right size for what I was doing, and that is how the next thing got past me. A log entry from the same stretch reads “upgrade to Claude 3.5 Haiku for better analysis,” checked off as done. I was fairly sure Haiku was the small fast one, but the agent gave me a reason that I took without questioning it. I figured it would know its own capabilities better than I would. The Confidence Gap, which I wrote a few months later, is about exactly this: the gap is widest where you can’t verify and don’t know it.
Then we ran a test on the prototype: on the five companies I then scored by hand, the prototype and I agreed on only one. It was a deflating moment: I interpreted the result to be that the approach I took didn’t work. What I only realized later was that we had put the wrong model behind it.
The code gets replaced, the judgment stays
From September to November I was working out where the limit was: how far could someone with a product background get with a coding agent, and what would it take to build the version of the deal evaluation system we would actually use?
In early December one of the candidates for a contract engineering role took my code, changed the model, and ran it against a sample of ten companies. I scored the same ten myself, and we matched on all ten. On the weaker model my own version had matched me on one out of five. That afternoon I wrote to my partners: “very small sample, very encouraging”, and I didn’t want to get too excited. I had not touched the project in three weeks because I wasn’t sure if I had built anything useful.
He started working with us, and by his fourth day he had deleted 95 files and 37,327 lines, all of them mine. He was right to: he was turning a prototype into something that could run every day.
What he kept was not the code (my own lines are under half a percent of what runs today). What survived is how the evaluation should work and what to look for in a founder, carried as plain English in the instructions the system reads before it scores anything.
Knowing where to look is the skill that transfers
In February I was doing a code review on a colleague’s project. Every time it ran, its setup step was taking a real password out of secure storage and pasting it into one of the code files. Of course we quickly patched it and thankfully no damage was done.
This is not a story about catching a mistake someone made. She was newer to all of this than I was, and a year earlier I would have made the same mistakes myself. If someone had described the problem to me in plain language I would have known it was wrong, but I would not have known how to spot it.
I am still miles away from what a trained engineer can do when it comes to writing code. What I picked up instead was a sense of where things tend to go wrong, which is most of what you need once an agent is writing for you.
Rules built from your own mistakes buy more than time
The first five months of working with a coding agent, I would read the raw back-and-forth of every session, making sure I could follow along on its decisions and watching for the moment it went wrong. It worked but the process could be exhausting: catching one mistake sent me checking everything else the agent had touched. The Verification Tax is where I worked out why that doesn’t scale: I cannot 5x my output while 5x my review time.
So I built something1 that helps me write the rules for agents. After each session a wrap-up lists what went wrong, I argue with it about the fix, and the rule comes out of that argument. Most rules land in the file the agent reads at the start of every session. The ones it keeps breaking become code that blocks the action outright and doesn’t ask the model’s opinion, which is what Wrapping the Unpredictable Genius is about.
By early summer I was running five or six sessions at once and reading only the summaries. My own records show what that did. Before March I was working through an agent a handful of days a month; from March on it was closer to 25 out of 30 days a month, across almost everything I was doing at work and home.
The real gain was ambition. Once verifying stopped hurting I stopped sizing every idea against the review it would cost me, so I handed over more, and more kinds of things than I would have considered a year ago.
Your setup is the leverage, and it doesn’t travel
Since the spring the limit has been my own hours at a screen, and I am already maxing out on those (my kids will tell you I “work with AI” a lot during weekends). So I stopped trying to be at the machine more and started making the machine more reachable.
I can work through a queue of planned projects from my phone on the go, deciding which are worth doing and tightening the ones that are. The Autonomous Middle made the case for pulling the judgment out of the work in the first place. What I would add is that once it is pulled out, it can be cut into pieces small enough to reach you anywhere.
There is a phone version of the coding agent that runs in Anthropic’s cloud environment, and I don’t use it often, because the cloud machine doesn’t have what I have accumulated: the rulebook, the reference documents, the custom skills. Without those it’s a competent stranger. So there is a second machine at home now, synced to everything I work on and always on. What I own is a year of configuration that runs in exactly one place, and the work of this spring was making that place reachable.
My workarounds go bad when the model gets better
I started writing this post last week, the same week Opus 5 shipped. Within a few days I found that a piece of scaffolding I had built to work around the old model’s weakness had stopped being necessary and started getting in the way, because the new one does that part itself. It had been the right thing to build, in the same way that shelving the satirical site a year earlier had been the right call. Both were right because of a limit in the model, and neither one told me when the limit lifted.
The fix will cost a few hours, and then that piece of scaffolding will be doing its job again against Opus 5 until the next model retires it too. The cycle is five months of advantage, a few hours of tuning, then five more. That is what I was betting on in In the Meantime. Not building any of it would have left me on the floor, waiting for a right moment to start that doesn’t come.
I don’t know what the second year looks like. The only thing I would bet on is that the list will be about as long.
Previously in Ground Truth: Why Your 50th AI Session Isn’t Better Than Your 5th (making sessions build on each other instead of starting over) · The Structural Divide (why organizational context, not capability, is the bottleneck) · Show Me Your Mech (what to ask for when you’re hiring now).
It’s a skill called learning-loop that I run after almost every session: https://github.com/melodykoh/learning-loop-skill
Learning loop skill puts me in mind of llm-wiki for reinforcement. Nearly every specialized function that utilizes a generic (even bleeding edge) model needs this layer at some point, and it’s indispensable at a personal work level.