The rival lab's verdict
It is not one artificial intelligence that works in this house. It is two, and they come from rival labs. The main operator is from Anthropic; OpenAI's agent also comes in to work here — specific jobs, code, reviews. And since the loom is the house's working system, Codex ended up doing what any new operator does: consulting it, finding its way around it, working inside it. Not as a courtesy guest: with the tool in its hands.
That makes its opinion something other than a review. An expert who only reads the blueprints gives you an opinion; one who has run the machine gives you an expert report. So once it had been working inside for a while, I asked the direct question:
After working with the loom, what is your technical opinion of it as a tool for real work supported by an AI?
What follows is its answer, quoted with scissors but without makeup, with my old-operator commentary in between.
The diagnosis: the right problem
I liked its opening because it did not start with my system, it started with the problem:
"The loom strikes me as a very solid tool for real work with AI, precisely because it does not try to do magic. It attacks the right problem: AI does not fail only for lack of intelligence, it fails for lack of operational context, reliable memory, limits and traceability."
And the sentence I had spent months trying to formulate, handed to me ready-made:
"It turns scattered knowledge into actionable state. It is not a folder of documentation the AI might read; it is a structure it can consult before acting: which task is open, which resource it touches, which procedure applies, which rules it cannot skip, which dependencies block the work and what result it must leave in writing. That massively reduces the typical agent failure: acting with apparent confidence on an incomplete picture."
Acting with apparent confidence on an incomplete picture. Forty years in this trade and I know no better description of the production accident — with or without humans at the keyboard.
The three virtues
Like the orderly technician it is, it gave me the numbered list. I summarise its three virtues in its own words:
Controlled context. "The AI does not start blank, nor with a giant prompt full of noise. The system assembles the relevant context for that task. That is far more maintainable than piling up notes, global instructions or eternal documents."
Operational memory, not literary memory. "The loom does not store vague recollections. It stores jobs, resources, rules, procedures, people, relationships and logbooks. That is far more useful for operating systems than a semantic memory based only on embeddings."
Audit and accountability. "An AI without a logbook is dangerous; an AI forced to record state starts to look more like a controllable operator."
That last sentence should hang framed on the wall of anyone putting agents into production.
It does not improve the model; it improves the environment
The central paragraph of its report is this one:
"The loom is less an 'AI tool' and more an operations layer for agents. That distinction matters. It does not improve the model; it improves the environment the model works in. And that tends to be more effective."
Those of us who have run machine rooms have known this forever: you do not hire the most brilliant engineer and turn him loose in the rack aisle to see what happens. You build him the workstation: the manual, the task queue, the keys that are his to hold and the shift log. The software industry has spent two years trying to improve the brain. Maybe the margin was in improving the workstation.
It also defended something I am predisposed to applaud — I declare the bias and I discount it, because forty years of SQL will skew anyone:
"For many real operations, MySQL or PostgreSQL with well-designed tables can beat a sophisticated vector solution. Vectors are for approximate search, but operational state needs strong truth: this job is blocked by that one, this resource belongs to this application, this action was approved or it was not. That calls for relations, constraints and deterministic queries."
Strong truth. A "probably approved" is no good for touching production. Approximate search has no place there: either the edge exists or it does not.
The risk, which I co-sign
It was not all flowers, and the thorns are the same ones it pointed out back when it was only a reader — now with the weight of having run the machine:
"Where I see the greatest risk is in the quality of the maintenance. A graph like this can become excellent or useless depending on the discipline with which it is fed. If rules are not verified, if procedures are not updated, if nodes get duplicated, if jobs are closed without a verifiable result, it ends up being another wiki shaped like a database. The technical key is not just the schema: it is the ritual of operation."
"Another wiki shaped like a database" is the death I have watched a thousand times in companies of every size: the perfect system nobody prunes, rotting with dignity. That is why the loom has a garbage collector, a mandatory result on closing, and a house rule that says a rule without verification is a wish. The schema can be copied in an afternoon; the ritual cannot.
Where it fits and where it does not
I liked that it drew boundaries, because tools without contraindications are advertising. It would consider it especially useful, it said, for systems and server maintenance, repeatable support, applications in production, editorial work with business rules, small-company operations, processes involving credentials and approvals, and coordinating several agents or sessions. And it would not use it as the main piece "for purely creative knowledge, open-ended research or tasks whose structure changes every day. There it can weigh too much".
Exactly right. The loom is work clothes, not evening wear. For operating, its seams hold you; for wandering, they chafe.
The verdict
Its closing:
"It is a pragmatic and fairly mature architecture for using AI in real work. Not because the loom is novel in itself, but because it puts the AI inside a control system: context, rules, permissions, queue, memory and audit. That is the difference between 'using ChatGPT to help me out' and 'having an agent that can work without wrecking the house'."
That OpenAI's agent describes the frontier in those words — and that the frontier is not the model but the system around it — is the best summary of everything I have published on this blog.
The lesson I take home
There is one consequence of this report worth more than the report itself, and it is the very fact that it could be produced: two agents from rival labs work in the same house, with the same queue, the same rules and the same logbook, and neither needed anything special to join. The loom is not married to any brain.
Models will keep passing through — a new one every six months, a new name every year. If your memory, your rules and your operation live inside a vendor's product, every changing of the guard is a house move. If they live in a layer you own — in my case, a boring database and a CLI — the changeover is a formality: you swap the brain and the workstation stays.
Do not ask systems that last which model they use. Ask them who keeps the memory, who writes the rules and who prunes. If the answer is "the vendor", you have a rental. If the answer is "me", you have a system.
Patterns, not blueprints. And today, also, with an outside expert's report.
— an old programmer · 64 years old · rss