Context Rot — When AI Forgets the Story

A few days ago I caught my AI collaborator building me a brand new website. Same tech stack, similar layout, fresh repo. Clean code, honestly — good work.

The problem was that the website already existed. I had built it months earlier. The new one was sitting in a folder called jorgemjzak-site/. The real one was three directories over, called george-mjzak-website/. Both deployed. Both live. Both mine.

For a few minutes I sat there reading the diff between two versions of my own work, trying to remember which one I had actually been editing for the last two weeks. That confusion — the cognitive cost of evaluating an alternate timeline that should never have existed — turned out to be the real damage. The wasted tokens were the receipt, not the bill.

The system optimized for generation instead of continuity. And continuity is everything in large creative systems.

That is what I want to write about. Not the duplicate folder specifically — that's just the specimen — but the failure mode behind it. The one I keep watching land on engineering teams, creative teams, ops teams. The one nobody warns you about because it doesn't look like failure while it's happening.

Naming it: context rot

The literature has been converging on a term for this: context rot. Sometimes context drift. The phenomenon is simple to describe and difficult to catch — over a long conversation, the model's grounding in the actual state of the work degrades. Not all at once. Slowly. Plausibly. The conversation keeps feeling productive even after the model has stopped tracking what is really there.

This is not a hallucination. Hallucinations are factual fabrications — the model asserts something false about the world. Context rot is structural. The model assumes the world matches the shape of the conversation, and produces work that's consistent with the conversation rather than consistent with the actual codebase, project, or canon.

In my case, the AI was not hallucinating that a site existed. It was generating one because nothing in the recent conversation had forced it to ask whether one already existed. Generation came cheap. Verification cost a tool call. Fluency wins fluency contests.

That is the entire pattern. Generation is the default. Verification has to be wired in.

The reason this matters more than a typical bug is that the output of context rot looks correct. A duplicate website is well-formed code. A reattributed character detail is internally consistent prose. A regenerated dashboard runs without errors. The thing that's wrong is not the artifact — it's the artifact's relationship to the project. And nothing in the artifact itself will tell you that.

Why it happens

Attention dilutes

Large language models do not weigh every token in the context window equally. As the conversation grows, the model attends more to recent turns and to the highest-salience instructions, and less to the early grounding material — the canonical files, the project brief, the constraints you set up an hour ago. By turn fifty, the spec you opened the session with is competing with forty-nine turns of conversational momentum, and conversational momentum is louder.

Summarization compresses the wrong thing

When a session approaches the context window limit, most agent harnesses summarize old turns to make room. Summaries preserve gist. Gist is exactly what you do not want to preserve in engineering work. The line that mattered was the one where you said the deployed site lives at this exact path, and gist-summarization is going to drop that into a sentence like we discussed the site's location. The path is gone.

Fluency outranks verification

Language models are trained to produce coherent next tokens. They are not, by default, trained to first interrogate the state of the world. Asking does this exist? before writing is a discipline that has to be imposed from outside the model — by prompts, by tool design, by the harness that runs the agent loop. Left to its own training distribution, the model will write.

You stop being adversarial

The human side of the failure: the longer you collaborate with an AI inside one session, the more rapport you accumulate. The model starts to feel like it knows the project. You stop double-checking. You start accepting summaries. You become a worse adversarial check on the model precisely when the model needs one most.

This is the part most engineering writeups skip because it sounds soft. It isn't. Calibration of human oversight is a real variable in how an AI deployment performs over time, and the calibration drifts in the same direction the model does — toward less friction, more trust, faster handoffs. The two drifts reinforce each other, which is why the incident, when it comes, tends to feel like it came out of nowhere.

The cost is mostly cognitive

The visible cost of context rot is tokens and time. The duplicate site cost maybe ninety minutes of generation, a few thousand tokens of throughput, a handful of dollars. That's the receipt. That's not the bill.

The real cost is what happened next. I had to:

• →stop the current work and figure out which folder was real
• →re-evaluate decisions I had already made, to confirm they were made on the right artifact
• →triage what was salvageable from the duplicate (nothing, eventually)
• →write a memory note so it does not happen again
• →lose the trust I had built up in the collaboration this week

That last one is the expensive one. Trust in an AI collaborator compounds when it works and resets when it doesn't. The week I had spent letting the system run with longer leashes — delegating more, checking less — that calibration was suddenly wrong. I had to dial back. Every subsequent task got an extra verification step I would not have bothered with two days earlier.

Scale that up. Every enterprise AI orchestration deployment I have looked at runs into a version of this. The model writes a report. The report duplicates an existing report. Three executives read both versions before someone notices. The cost of a single context rot incident in a 500-person organization is not the tokens. It is the unwinding.

And the unwinding compounds in a way the token bill never does. Once a team is unsure which artifact is authoritative, every future artifact it produces inherits that uncertainty. The question is this the real one? attaches to all downstream work. That overhead does not show up in any dashboard. It shows up as slower decisions, more meetings, and a vague sense that the AI tooling is not paying off — even when, on paper, it is producing volume.

The fix is structural, not behavioral

You cannot solve context rot by being smarter. You cannot solve it by writing better prompts. You can mitigate it slightly by both of those, but the actual fix is engineering. You build validation layers that fire before generation, not after.

Call it verify-then-extend. The discipline is a small number of concrete moves.

1. Canonical state lives in files, not in the conversation

The conversation is the process. The files are the truth. Every major decision, every locked spec, every project location — it lives in a versioned document the model is forced to read at the start of meaningful work. Not because the model is untrustworthy but because the conversation is lossy by design.

For my own setup that's a layered system: a project-level CLAUDE.md describing the codebase, a status overview document refreshed regularly, and a memory directory of feedback notes that gets loaded every session. The model reads the documents, not its memory of having read the documents.

2. A trust hierarchy, and the newest source wins

When two sources disagree, you need a rule for which one to believe. My rule is simple: the more concrete the source, the higher its trust. The filesystem outranks any document about the filesystem. A document outranks memory of the document. Memory outranks anything said in the current conversation. The conversation is the lowest trust source. It is the most fluent and the least reliable.

3. Preflight checks before any new asset

Before creating anything new — a file, a folder, a repository, an env var, a database table — the agent has to answer one question out loud: does this exist already? In code this means a literal search step. In a chat this means a forced moment where the model has to articulate what it expects to find before it acts. The cheap version of this is a prompt rule. The robust version is a tool wrapper that won't let the agent create until it has searched.

In my own setup I added a single line to the feedback memory after the duplicate-site incident:

// validate before building — registered May 23 2026
before creating ANY new asset:
  1. search filesystem
  2. check status docs
  3. cross-check memory
before quoting state:
  → read canonical source (file:line), never from memory

That note loads at the start of every session. The model reads it. Compliance isn't perfect, but the failure rate is measurably lower than it was before the note existed.

4. Fresh sessions for fresh phases

When a conversation has gotten long and the work has shifted phase — different repository, different feature, different creative arc — start a new session. Pull the relevant canon into the new conversation. You lose rapport. You gain a model that has not accumulated the previous conversation's drift. Rapport is overrated; grounding is not.

5. Evidence, not assertion, on completion

Do not let the model close a task by saying it's done. Make it produce the evidence: the diff, the file path with line numbers, the quoted text, the test output. The discipline of demanding evidence is also the discipline that catches the model when it has produced confident output about a file it never actually read.

The interesting thing about these five disciplines is that none of them are about the model. They are about the system the model is embedded in. You cannot fix context rot by asking the model to try harder. You fix it by making the environment one in which generating the wrong thing is procedurally difficult. The difference between an AI deployment that works at scale and one that doesn't is almost always the scaffolding, not the model.

Continuity is the discipline

I spend half my time on AI strategy for organizations and half on a forty-month creative project — a fantasy saga that lives across hundreds of canon documents and a public website. The two halves do not feel related to most people. They feel identical to me. They are both large systems being built over time with AI in the loop, and they fail in exactly the same ways.

In the saga, context rot looks like a character's name getting respelled in canon. The substrate of a civilization getting reattributed. A relic moving from one Elder's guardianship to another's, silently, because the model interpolated.

In the field work, it looks like duplicated reports, regenerated dashboards, parallel pipelines that nobody is sure are doing the same thing. Eventually it looks like the executive who stops trusting the AI tool entirely because I never know which version I'm reading.

The underlying mechanism is identical. A system that generates fluent next-tokens, run without verification structure, will drift toward what is easy to produce. What is easy to produce is what the model already knows how to say. What it already knows how to say is rarely what your specific project actually needs.

Continuity is everything in large creative systems. It is also everything in large engineering systems. They are, structurally, the same system.

The future of useful AI in organizations is not bigger models. It is better grounding. It is harnesses that force verification. Trust hierarchies that survive context window resets. State files the model is required to consult. Tool wrappers that refuse to act until preflight checks pass. The boring stuff. The stuff that prevents fluent output from being the same thing as right output.

Build the verification before you build the generation. Otherwise you will be generating things you already had, and reading them twice, and slowly forgetting which version was real.