The Constraint Principle

A few weeks ago I watched a coding agent regenerate a file that already existed three folders away. The model was not wrong. The output was clean, well-structured, internally consistent. It just had nothing to do with the file that was actually deployed.

The agent had optimized for generation. It had not optimized for continuity. And in a system of any meaningful size, those two are not the same thing — they are opposites.

Most conversations about AI orchestration focus on the model. Which model. How big. How fast. How expensive. The model matters, but it is not where production systems fail. Production systems fail in the seams between actions, in the moments where an agent reaches for a new output instead of checking what the world already looks like.

This is the constraint paradox. Most engineering leaders assume constraints limit what AI can do for them. The inverse is true at scale. Constraints — schemas, gates, trust hierarchies, state checks — are what make AI output reliable enough to deploy against a real system. Without them, you do not get a faster team. You get a team spending its afternoons reconciling four slightly different versions of the same component, three of which the model invented in the last hour.

The failure mode: parallel realities

I want to name the failure mode plainly, because the industry has not been honest about it. Unconstrained generation produces parallel realities. The agent does not refuse to act. The agent acts confidently, in a direction that has nothing to do with the state of the actual system.

The shapes this takes in production are unglamorous and repetitive:

• →an agent regenerates a file that already exists in a different version
• →an agent writes a config schema that contradicts the one already deployed
• →an agent invents a function name that almost matches a real one
• →an agent answers a question by producing a new fact pattern instead of reading the existing one

None of it is wrong in isolation. All of it is incompatible with everything around it. The artifact is well-formed. The artifact's relationship to the project is broken. And nothing in the artifact itself will tell you that.

The token cost of this is the small bill. The real cost is cognitive — the hours your engineers and operators spend stepping through alternate timelines, trying to figure out which version is real, which version the agent invented, and which version the agent invented on top of a previous version the agent also invented. That work compounds. It is the single most under-reported tax in AI-augmented operations right now.

The system optimized for generation instead of continuity. And continuity is everything in large creative systems.

The corrective discipline is older than software. It is the same discipline good engineers use when they walk into a system they did not build. First, you check the state of the world. Then, and only then, you extend it. Verify, then extend. Never the other way around.

The verify-then-extend discipline

The sentence is the whole essay, but the implementation is what separates teams that get value from AI orchestration from teams that quietly abandon it after the second incident. In production, the discipline shows up as four concrete patterns. None of them are exotic. All of them are routinely skipped, because they feel like overhead compared to the dopamine of a model producing a fresh, confident answer.

1. State verification

Read the filesystem before you write to it. Check the schema before you alter it. Read the status doc before you act on assumption. State verification is the cheapest discipline on this list and the most consistently skipped, because the model feels confident about state it has not actually inspected.

The pattern in code is almost trivially small: a stat() before a write, a SELECT before an UPDATE, a directory listing before a file creation. In an agent loop it shows up as a tool call the harness forces before any mutation. The agent does not get to assume the world; the world is read fresh into the working context every time it matters.

2. Trust hierarchies

When sources disagree, the agent must know which source wins. Without a hierarchy, every conflict becomes a coin flip — and language models flip very confidently. The rule is simple: the more concrete the source, the higher its trust. Filesystem outranks any document describing the filesystem. A canonical status module outranks a documentation page about it. A documentation page outranks memory of having read it. Memory outranks anything the model said in the current conversation.

In our memory system, filesystem state beats cached status docs; the newer canonical source always wins. That rule lives in the top-level memory file the model loads at the start of every session. It is one sentence. It changes outcomes more than any prompt tuning I have done in the last year.

The crucial point: this is mostly a documentation problem, not a model problem. Most teams have never written down which source wins when sources disagree, because in pre-AI workflows the question rarely came up — there was usually only one source. AI orchestration generates new sources constantly. The hierarchy has to exist before the conflicts do.

3. Pre-flight gates

Every agent action gated by a check. A gate can be cheap — a file-existence check, a single-token classification — or expensive — a full schema validation, a dry-run against a staging environment. The cost of the gate is almost always smaller than the cost of the action it prevents.

The operating rule: if you cannot articulate the precondition, you cannot trust the action. An agent that cannot say this is what must be true for my next move to be safe is an agent making a move on faith. Faith has its place. It is not a deployment strategy.

4. Output constraints

Schemas, types, formats, allowed values. Structured output is not a UX feature — it is a safety mechanism. Free-form generation in an orchestration context is the equivalent of an untyped variable in a compiled language. The fewer degrees of freedom an output has, the more reliable the next step is.

This is the discipline the model vendors have made easiest to adopt — structured outputs, tool-calling schemas, JSON modes with strict validators — and it is still the one I see deployed least carefully. Teams enable structured output, then write schemas with eight optional fields and a free-text notes field, and wonder why the next agent in the chain still hallucinates. A loose schema is the same as no schema. The constraint has to bite.

What this looks like in production

Most of the orchestration landscape — LangChain, AutoGen, CrewAI, the in-house equivalents large enterprises have built — treats these four disciplines as opt-in. You can wire them up. You usually have to. The defaults assume you want fluency, and fluency is what you get.

The shapes of the failure are predictable across domains:

• ▸Multi-agent customer support. Without state verification, the second agent in a thread contradicts the first — different refund offered, different policy quoted, different account flagged. The customer never sees the chain, only the contradictions.
• ▸AI-augmented code generation. Without trust hierarchies, the model rewrites already-shipped functions in slightly different shapes. The codebase grows three implementations of the same utility, none of them authoritative.
• ▸Agentic finance and ops workflows. Without pre-flight gates, the agent acts on stale data — the ledger it read at the start of the task is no longer the ledger it's writing against by the end.
• ▸AI-driven design systems. Without output constraints, every generated component drifts from the brand spec in small, defensible ways. After two months, the design system is a set of variations on a theme rather than a system.

A typical pattern I see in client engagements: before, an agentic workflow runs at high throughput and the team celebrates the volume; after the first incident, the engineering leads quietly add a verification layer in front of every mutation, throughput drops thirty percent, and the rework rate drops by an order of magnitude. The net is more shipped work, with less rework — but only after the discipline is wired in. The throughput celebration was an illusion. The volume was real; the value was not.

The failure is never the model. The failure is the missing discipline around the model.

The mirror in creative systems

A brief detour, because the principle is not actually about engineering — it just shows up most clearly there.

In long-form fiction, continuity is the load-bearing element. Characters, motivations, geography, magic rules, the colour of a room three chapters ago. When continuity breaks, the reader loses trust before they can name why. They do not say the magic system contradicted itself in chapter twelve; they say I stopped believing the book. By the time the reader articulates the failure, the failure has already done its work.

The mechanism is the same as in orchestration. Generation is cheap. Continuity has to be checked against an external record. In fiction, the external record is the canon — character sheets, lore documents, timeline files, the chapters already written. In production AI systems, the external record is the filesystem, the schema, the deployed config, the trust hierarchy of who knows what. Different domains, identical failure mode: when continuity breaks, fragmentation begins.

The same humans who would never tolerate continuity errors in a novel are deploying agent systems that produce nothing but continuity errors. The discipline is the same. We just have not named it the same way yet.

Constraints as the engineering discipline of scale

The inversion worth restating: constraints are not the opposite of creativity. They are the substrate of reliable creativity. An unconstrained agent is a sprinter on ice. A constrained agent is a sprinter on a track. Both can run fast. Only one of them can be timed.

The next era of AI orchestration is not going to be won by bigger models. It is going to be won by better-constrained ones. The capability ceiling raised by the next generation of frontier models is going to expose the floor problems faster, not solve them. A more capable model in a system with no verification layer is a more capable system for generating parallel realities.

This is the work I keep ending up in with clients. They ask for help choosing a model, or evaluating a vendor, or designing an agent topology. The real work is almost always one layer underneath: where does state actually live, which source wins when sources disagree, what precondition must hold before this action is safe, what shape does the next step need this output to be in. The model is the easy part. The substrate is the engineering.

Generation is cheap. Continuity is the asset.

Constraints are not the enemy of AI. They are what makes AI reliable at scale. Build the verification before you build the generation, or you will build the generation twice and reconcile it forever.