Work in progress. This is a note, not a finished piece. It’s a live experiment and I’m publishing the state of it rather than waiting for a verdict that may not arrive. Companion to part one, part two and part three of my setup write-up.
Both my agents load a shared skill library - obra/superpowers - mounted read-only, so neither of them can quietly edit its own instructions. That read-only detail isn’t paranoia for its own sake: an instruction layer an agent can rewrite is not an instruction layer.
Skills are markdown files that trigger themselves when the agent recognises a situation they apply to. There’s nothing to install into the project, no runtime, no lock-in, and nothing that shows up in a dependency list. If I removed the mount tomorrow, everything would keep working - slightly worse.
What it actually changed
My honest read after a few months: it genuinely speeds things up, though not in the way I assumed it would. The code doesn’t arrive faster. Far less of it gets thrown away.
That distinction took me a while to see. I’d been measuring the wrong thing - how quickly something appeared - when the expensive part of my week was never typing. It was building the wrong thing carefully, discovering that two hours later, and having no clean way back.
The ones that changed my working day:
- brainstorming - refuses to start building until it has pulled a specification out of the conversation and I’ve signed off on it. For someone who thinks in requirements rather than code, this is the single most valuable one in the library. It also catches the thing I’m worst at: describing the outcome I want instead of the implementation I imagined.
- writing-plans and executing-plans - the plan lands in a file, broken into steps, and gets worked through with checkpoints. The alternative is an agent that vanishes for an hour and returns with a surprise, and a surprise is expensive even when it’s a good one.
- systematic-debugging - ends the “try a fix, try another fix, try the first fix again” spiral that can quietly consume an entire evening.
- verification-before-completion - the agent has to run the thing and show me the output before it’s allowed to tell me it works. I struggle to overstate how much this one is worth. “Done” and “I believe it’s done” are different claims, and before this I was being handed the second one dressed as the first.
- requesting-code-review - a second pass over the work before it ever reaches me.
There’s a pattern in that list, and it’s the reason I keep the library rather than writing five rules of my own. Every one of them is a discipline a good engineer already has and I don’t: insist on a spec, write the plan down, change one variable at a time, prove it before claiming it, get a second opinion. Encoded well enough, the agent supplies that discipline on my behalf. That’s a strange and rather useful thing to be able to buy, and it’s a much better description of what these libraries do than “prompt templates”.
Now the doubt
There’s a piece of advice I keep circling back to and haven’t taken: every so often, strip out all the extra configuration and work with the bare tool for a while.
Two reasons it nags at me.
The models move underneath the scaffolding. Structure built to compensate for a real weakness a year ago may now be getting in the way of something the model does better unaided. Skills are written against a snapshot of model behaviour, and nothing tells you when that snapshot expires. A skill that once prevented a failure mode can end up enforcing a detour around a pothole that was filled in months ago.
Every layer of instruction occupies context. Skills, rules, hooks, reminders - it’s all text that arrives before my question does.
Where I’ve actually got to
I don’t have a verdict. What I have is a setup that feels better with the skills loaded, and an awareness that “feels better” is precisely the kind of claim that survives longest without ever being tested. It’s the sort of thing I’d challenge immediately in someone else’s analysis, which is a reasonable sign that I should challenge it in my own.
Open questions I’m carrying:
- What does a fair test even look like? Same task, twice, with and without - but the second run isn’t independent, because by then I know the problem.
- Which of these are compensating for the model, and which for me? The second kind should stay regardless of how good the models get.
- Is there a smaller subset that captures most of the value? My instinct says the spec-first one and the prove-it-before-claiming-it one carry the majority, and the rest are refinements.
- How would I notice if a skill went stale?
I’ll update this note when I’ve run the experiment rather than when I’ve thought about running it. That distinction is, unfortunately, the whole point.