Inside the harness: shipping production software with LLMs —
without losing the plot.

Anyone can prompt a model into producing code. The hard part is producing software you’d bet a production system on. Our answer is an engineering harness: a pipeline where models check inputs and outputs at every boundary, and a human signs off on every decision that matters.

Why a harness at all

An unharnessed agent optimizes for looking done: implementation before the problem is specified, tests omitted when inconvenient, completion reported without evidence. These are predictable failure modes, and the harness addresses them structurally rather than by prompting harder. Each phase has entry and exit criteria — inputs checked on the way in, outputs reviewed on the way out, first by a model in an adversarial reviewer role, then by an engineer.

In practice: the spec is checked against the original problem before architecture begins. The architecture is checked against the spec before implementation begins. Every diff is checked against the plan, the test suite, and a security checklist before a human reads it — so reviewer attention is spent on judgment, not on catching mechanical errors.

Watch it run

The rules the pipeline enforces

Under the phase diagram, four rules do the real work:

• Specification before code. A rough idea becomes a written spec, reviewed in small, readable pieces and signed off before anything gets built.
• Tasks small enough to check. Work is broken into narrow steps, each with an explicit way to verify it — progress is auditable, not taken on trust.
• A clean slate per task, reviewed twice. Each task runs in isolation and is checked twice — against its spec, then for quality — before it reaches a person.
• Tests first, always. The failing test comes before the implementation; code without a test to justify it doesn’t stay.

The pipeline also enforces three hard constraints: models work with sandboxed access and scoped, time-limited credentials; automated security scanning runs on every change; and production secrets and sensitive data never reach a model. No exceptions, no overrides.

What it buys

We instrument every engagement; we don’t estimate. Three numbers tell us whether the harness is earning its keep:

• Human review time per merged change — how much reviewer attention each change actually costs, after the model has already validated it against the plan, the tests, and the security checklist.
• Defects caught before human review — the share of issues the pipeline stops before a person sees the diff.
• Idea → deployed MVP — calendar time from first conversation to a running system, against our pre-harness baseline.

It’s already done its job at least once: on the billing build, the architecture gate rejected a design that would have double-counted usage on retries — before a line of code existed.

We publish these as running averages once the sample is large enough to mean something. Numbers land here as they’re collected.

What it costs

None of this is free. Specs take time before any code appears, gates reject work and force loops, and the pipeline is slower on day one. We accept that, because the cost surfaces early — where it’s cheap — instead of in production.

The line that doesn’t move

The harness makes us faster and stricter at the same time — that’s the whole point. But it never decides. Every architecture choice, every merge, every deploy carries a human signature. You inherit a codebase where each of those signatures is on record.