Work in progress. This is a note, not a finished piece. Several weeks of testing, no conclusion, and the honest interim result is “not what I hoped for”. I’m publishing it in that state deliberately. Companion to part one, part two and part three of my setup write-up.
What I wanted
An assistant that isn’t a coding tool.
Something holding the picture across all my projects at once: what’s a priority, what’s blocked, what I said I’d do three weeks ago in a conversation about a project I haven’t opened since. The coding agents are excellent inside one repository and have no idea the other four exist. That’s not a flaw in them - it’s the boundary I deliberately built in part one, working exactly as intended. But it leaves a real gap above the projects, and the gap is where most of my actual decisions live.
The plan was a division of labour: an orchestrator that holds the cross-project picture and helps me plan, with the IDE agents staying separate and doing the work inside each repository.
Why this particular one
I’ve been testing Hermes for several weeks, and the reason I chose it is worth stating precisely, because it’s the same reason someone else might: it has a layered memory model that neither coding agent ships with.
By the time I looked at it I already believed memory architecture was the bottleneck - that’s the whole argument of part three - and here was a system that appeared to agree, with structure built in rather than assembled by hand out of markdown files and habits. That was the entire attraction. Not the model behind it, not the interface. The memory.
How I contained it
The same way as everything else, and this part I’d do again without changes:
- its own container;
- both its ports bound to the loopback interface only, so nothing is reachable from the network;
- the projects tree mounted read-only, with exactly one project writable;
- no Docker socket, no home directory;
- a pinned image, so a rebuild reproduces what I tested rather than whatever shipped this week.
Worth saying plainly: an assistant with a view across all my projects is precisely the thing that most needs a boundary. Broad read access and narrow write access was the shape that made me comfortable turning it on at all.
What actually happened
It’s genuinely good at narrow, one-off research. Ask it a bounded question, get a solid answer. That part works and I still use it.
And it isn’t what I hoped for. It drives models through a chat interface rather than through a coding agent, and that difference turns out to be everything. There’s no real agentic loop over a repository: no tool cycle, no diff to review, no verification step, none of the “it tried, it failed, it adjusted, it tried again” that makes the IDE agents useful for actual work. For a one-shot answer, fine. For “go handle this in that project”, not close.
This holds regardless of which provider’s models are behind it, which surprised me. I’d assumed the model was the variable. The surface it talks through turned out to matter more.
So the delegation piece is still switched off, deliberately. Before an orchestrator hands work to a coding agent on my behalf, I need explicit project selection, approval gates, sane behaviour on timeouts and failures, and a rule for where the resulting transcript goes and under which project’s name. None of that is designed yet. Turning it on without it would be handing an unsupervised agent a keyboard and a list of my repositories, which is the exact scenario the rest of my setup exists to prevent.
There’s also a licensing question I want settled before building on this: whether driving one vendor’s coding agent from another orchestrator is within its terms of use. Unresolved, and not something to discover after the fact.
Where that leaves the actual question
Still open, and it’s the one I’d most like an answer to: what do you use as a life admin?
My requirements, as they currently stand:
- holds every project’s context without me pasting it in;
- not tied to a single vendor’s subscription, so my planning layer doesn’t move whenever my coding tools do;
- either a real agentic loop, or an honest acknowledgement that it’s a thinking surface rather than a doing one - the failure mode is a tool that looks like the second and behaves like the first;
- containable to the same standard as everything else.
I don’t have a candidate that meets all four. I may end up building the thin version myself: something that reads the per-project wikis from part three and maintains a small cross-project state file, with no autonomy at all. Less impressive, and it would answer the question I actually have.
I’m writing this up unfinished because “I tested this for several weeks and it didn’t do what I hoped” is more useful to somebody else than another architecture diagram - and because I’d have liked to read it before I started.
Open questions I’m carrying:
- Is the chat-surface limitation fundamental to this class of tool, or a current implementation detail?
- Does a life admin need to act at all, or is holding an accurate picture the whole job? I keep assuming the first and I’m no longer sure.
- If the answer is a thin self-built layer, what’s the smallest version worth having?