Francisco Acasuso · AI Strategist & Builder

Every few weeks a new model tops a benchmark, and every few weeks someone asks whether we should switch. It is the wrong first question. Across a fleet of products where coding agents write most of the code, the thing that decided whether the work was good was rarely the model. It was the harness around it: what the agent is allowed to touch, what is checked before its work counts as done, what it can see of cost and history, and who reviews it.

A model is a capable new colleague. The harness is the onboarding, the access rights, the definition of done and the review. Nobody would let a new colleague push to production on day one because they interviewed well.

Instructions are requests

We started where most teams start: rules in a CLAUDE.md file. Do not edit migrations on the main branch. Do not delete tests. Run the checks before you finish. The rules were followed most of the time.

Most of the time is not good enough. An inventory in September 2026 found the same stack in all three live revenue products, with each defense applied to a different subset of them. Whatever the building session happened to remember is what shipped. A rule that holds nine times out of ten is a rule that fails on the tenth commit, and the tenth commit is usually the one written late, in a hurry, in a parallel session.

So the rules moved out of prose and into the client harness, a Claude Code plugin that a repository declares once and every session loads:

A PreToolUse guard makes migrations immutable on the default branch and lets the test suite grow but never shrink.
A second guard blocks the git habits that cause most accidents: staging everything, committing everything, force-pushing without a lease.
A PostToolUse hook runs the project's own linter on every edited file and hands the errors straight back to the agent.
A Stop hook runs the repository's checks, and the turn cannot end while they fail.

Every hook answers its own self-test, and every block has a deliberate, visible and narrow escape hatch. A block with no way out does not make people careful. It makes them disable the hook.

A reviewer that cannot act

The second lesson was about review. An agent that writes code and then reviews its own code with the same permissions is marking its own homework. Our reviewer agent holds read, grep and glob, and nothing else. No shell. It is read-only by construction, not because a prompt asked it to behave. It reads the diff against the spec and reports only the gaps that affect correctness, which keeps the signal high enough that people actually read it.

The same thinking applies to CI. The harness writes a test-count check against a committed baseline, a secret scan that redacts what it finds, and a pull-request template that asks for evidence. The checks are required. A gate that is not required is advice.

Defaults instead of memories

The product template is the harness applied before the first line of code. Every module in it maps to an incident we would rather not repeat: webhook routes exempt from auth, environment keys checked at startup, login history in the database, and a daily cross-host check that scheduled jobs really fired. A new product starts with every lesson already applied, instead of depending on whoever builds it to remember them.

Visibility is part of the harness

A harness also decides what you can see. Every model call in our products is traced in Langfuse with the product, the feature and the user attached, and the agent fleet console turns that into cost per product. The dev panel in the same console connects the shared backlog to the coding agents, so work is claimed, started and tracked in one place rather than across terminals. If you cannot see what an agent spent or what it is working on, you are not running a fleet. You are hoping.

Agents with roles, people at the gates

Product Factory took the idea one step further: an orchestrator and seven specialist agents (planner, UX designer, prototyper, frontend, backend, QA and deploy) working a live Kanban, with a person approving at planning, prototype and review. It taught us where agents are fast and reliable, and where a human decision is cheaper than an agent's third attempt. Those lessons now shape how the harness splits work between agents and people.

What this means for clients

For a business adopting AI, the practical takeaway is the same at every scale. Choose a good model, but spend the real effort on the harness:

Decide what the system may touch, and enforce it in code rather than in a prompt.
Define done as checks that run every time, not as a request to be careful.
Make review independent of the work it reviews.
Instrument cost and activity from day one, so the conversation about value is based on numbers.
Put people at the decisions that matter, and let the agents do the rest.

Models will keep changing. A good harness lets you swap them without changing how much you can trust the result.