Pillar guide

Context rot: what it is and how to fight it with harness changes

How to fight context rot with harness changes, treated as an engineering job rather than a tips list: prompt, tool, skill, memory, and eval fixes — found in and verified against your production traces. And how to prevent context rot in AI agents with the same loop — detect context loss in traces, fix the harness component responsible, verify against replayed traffic — rather than with another vector database. Plus what the Chroma research measured across 18 models and how to fix the lost-in-the-middle effect in a production agent.

Book a demo Read the glossary

TL;DR

What is context rot in AI?

Context rot in AI (LLM context rot) is the degradation of a large language model's performance as its input context grows longer: recall, reasoning, and instruction-following decline unevenly as tokens accumulate, even on simple tasks — measured across 18 models in Chroma's 2025 report. In a production agent it surfaces as forgotten turns and dropped constraints. Prevention lives in the agent harness, not the model: compaction, memory, trimmed tool outputs, and evals on mid-context misses, each verified against production traces.

The phrase comes from a July 2025 technical report by the Chroma research team — Context Rot: How Increasing Input Tokens Impacts LLM Performance — that put a name on something practitioners had been reporting informally for a while: models that ace million-token retrieval benchmarks still get measurably worse at simple tasks as the input grows. Anthropic's engineering guidance adopted the term soon after, defining it as the observation that as the number of tokens in the context window increases, the model's ability to accurately recall information from that context decreases.

This guide answers how to fight context rot with harness changes — component by component, in the agent you run in production — then how to prevent it in general and specifically in AI agents, what the research actually measured, how context rot relates to the older lost-in-the-middle position effect, and how it differs from context loss, the behavioral failure you observe in a production trace. The short version of the fight: you cannot patch the model, and a bigger context window does not fix it. Every mitigation that works is a change to the agent harness — what goes into the window, what gets compacted, what moves to memory or retrieval, which tool outputs are worth their tokens. That makes context rot a harness engineering problem, and this page treats it as one.

One note on the advice landscape before the details. Most of what ranks for these questions comes from vendors of one ingredient — a vector database, a session-memory store, a RAG or session-management layer. Those components can implement part of a fix, but installing one is not prevention, because nothing about adding retrieval or a memory tier confirms the agent stopped dropping mid-context constraints. Prevention is a loop: find where long contexts degrade behavior in your production traces, change the harness component responsible, verify the change against real traffic. That loop is harness engineering, and it is the frame this page uses throughout.

Updated 2026-09-07

By component

How to fight context rot with harness changes

To fight context rot with harness changes, engineer the harness — the prompts, tools, skills, evals, and memory assembled around the model — one component at a time; this is not a model upgrade or an infrastructure purchase. One disambiguation up front, because this query collides with a company name: the harness here is the agent harness around an LLM, not Harness (harness.io), the CI/CD platform — software delivery pipelines have nothing to do with model context windows. The five harness-component changes that fight rot:

1.Prompt changes. Rewrite the compaction prompt so summaries keep decisions and constraints; move load-bearing instructions to the start or end of the window; delete stacked instructions left over from completed workflow phases.
2.Tool changes. Reshape tool schemas so results come back trimmed to the fields the agent uses, grounded with an identifier to re-fetch the full payload; cap the free text a single tool call can dump into the window.
3.Skill changes. Encode long workflows as phases that end in a compacted handoff, so the skill itself prevents transcript pile-up instead of relying on the model to cope with it.
4.Memory changes. Move durable facts into an explicit memory store written and retrieved deliberately; stop using the transcript as the database.
5.Eval changes. Turn traces where the agent dropped mid-context information into regression evals that every future harness change has to pass.

Much of what ranks for this question is session discipline: plan before executing, execute stepwise, reset the session when it degrades, checkpoint the output against the original task. That advice works — for the operator applying it, one session at a time. Harness changes are what make the same discipline structural: a skill's phase boundaries enforce the plan, compaction enforces the reset without losing decisions and constraints, trimmed tool schemas keep the noise from accumulating in the first place, and regression evals are checkpoints that run on every change instead of when someone remembers. Fighting context rot with harness changes, component by component — prompt, tool, skill, eval, and memory changes derived from production traces and verified against replayed traffic — is the version of that discipline that survives the operator leaving the room.

What makes these harness changes rather than tweaks is where they come from and how they are checked: each one is derived from production traces that exhibit the failure, and each is verified against replayed traffic before it ships. That trace-to-verified-fix loop is what Moda automates — a harness engineering platform rather than an observability dashboard: production traces in, verified harness improvements out.

SourcesAgent harness (Moda glossary) · Context rot (Moda glossary)

Symptom triage

Why does my agent forget earlier turns

Why does my agent forget earlier turns? Because something between the user's words and the model's attention lost them — and the fix is a harness-component change, whichever of the five culprits below produced it. Each one yields the identical symptom: the agent forgets earlier turns, re-asks for information it was already given, or drops a constraint set at the start of the session. The five culprits and their harness fixes:

1.Context rot. The window has grown long enough that the model can no longer reliably use what is in it — the earlier turn is still present but effectively invisible, especially mid-context. The harness fix: compaction and trimmed tool outputs that keep the window short.
2.Truncation or overflow. The earlier turn was literally evicted when the session exceeded the window, or a harness bug sliced the history. The harness fix: repair the context-assembly logic and compact before the limit, not after it.
3.Over-aggressive compaction. A summarization step condensed the earlier turns and destroyed the specific fact or constraint in the process. The harness fix: rewrite the compaction prompt to preserve decisions and constraints, gated by regression evals.
4.Failed memory. The fact was supposed to persist in a memory store and either was never written or was not retrieved on the turn that needed it. The harness fix: explicit memory writes, plus retrieval checked on the turns that need the fact.
5.Session boundaries. State was silently dropped when the conversation crossed a session, sub-agent, or hand-off boundary. The harness fix: a compacted handoff that carries decisions and constraints across the boundary.

Which culprit you have determines the fix, and the symptom alone cannot tell you — the trace can. Was the turn still in the request payload the model actually received? Was it summarized away? Was the memory read ever issued? Guessing is expensive: the classic mistake is buying a bigger context window — a fix only for truncation — when the real failure is a bad compaction prompt, which the bigger window makes worse. This triage is the detect-attribute-verify loop Moda runs on production traces: forgotten-turn behavior is flagged as context loss, attributed to the specific harness component responsible, and the resulting fix verified against replayed traffic before it ships.

SourceContext loss (Moda glossary)

The symptom

Context loss in AI agents

Context loss in AI agents is the observable failure where an agent stops acting on information it was already given — and the fix is a harness-component change, not a bigger window. It is the trace-level symptom — distinct from context rot, the model-level mechanism that is only one of its causes — and in multi-turn agent sessions it is among the most common behavioral failures, because every turn, tool call, and tool result makes the window a harder place to keep a fact alive. In a live session it looks like:

Re-asking. The agent requests information the user supplied turns ago — the most visible form, and the one users report as "it forgot".
Contradiction. An answer or action conflicts with something established earlier in the same session.
Dropped constraints. The agent follows the most recent instruction while silently violating a standing rule set early in the conversation.
Redundant work. Tool calls repeat earlier calls because the earlier result is no longer usable context, burning tokens and latency on answers the session already had.

The harness fixes map onto the causes: compact completed turns so history stays usable, trim tool outputs so the window is not flooded with dead tokens, move durable facts into an explicit memory store, and isolate deep sub-tasks in sub-agents with clean windows. Which fix applies is a per-detection question — which is why Moda treats context loss as a first-class behavioral failure: detected across every production trace, attributed to the harness component responsible (a compaction prompt, a tool schema, a memory store), and each candidate fix verified against replayed traffic before it ships.

SourceContext loss (Moda glossary)

Prevention

How to prevent context rot

To prevent context rot, control what enters the context window — a harness job, not a purchase. You cannot fix it inside the model, a bigger window only delays it (Chroma's measurements show degradation beginning well before advertised limits), and a vector database alone cannot confirm the failure went away. The six techniques that work, in any LLM application:

Curate what enters the window. Retrieval that pulls the relevant slice just-in-time beats preloading entire corpora; on LongMemEval, every model tested scored higher on a ~300-token focused input than on the full ~113k-token history containing the same information. Keep lightweight pointers — file paths, queries, record identifiers — in context and fetch the content on demand.
Compact instead of appending. Summarize completed phases and older turns — keeping decisions, constraints, and open questions, dropping raw transcript — and trigger compaction at a set token threshold, before the window is under pressure rather than after behavior has already degraded. The compaction prompt is a harness artifact: tune it on real long traces for recall first, then precision.
Trim what tools and retrieval return. Verbose payloads are dead tokens at scale — in typical agent turns, tool outputs are most of the window. Pass along the fields that will actually be used, with an identifier to re-fetch the full result on demand, and replace stale tool outputs that have already been consumed with a compact placeholder naming the tool and how to re-fetch.
Move durable facts to explicit memory. State that must survive across turns and sessions belongs in a memory store written and retrieved deliberately, not in an ever-growing transcript — and retrieval has to be checked on the turns that need the fact, because a store that is never read reproduces the failure it was installed to prevent.
Position deliberately. Load-bearing instructions go at the start or end of the window, never buried mid-prompt — the lost-in-the-middle result applied as a design rule — and long sessions should restate governing constraints near the end of the window rather than trusting the model to reach back to turn three.
Reset at task boundaries. One task, one working context: when a workflow phase completes, carry a compacted handoff forward — decisions, constraints, open questions — and drop the rest, instead of letting a debugging session become a refactor with forty turns of stale history behind it.

Notice what is not on that list: a bigger context window, a different vector database, a larger session store. Infrastructure can implement these techniques, but none of it tells you whether your system stopped losing mid-context information — that is an empirical question about your own traffic. Every item on the list is a change to the agent harness — a prompt, tool, skill, or memory edit — which is why prevention belongs to harness engineering, and why answering the did-it-work question is where prevention stops being generic advice and becomes the production loop covered next.

SourcesHong, Troynikov & Huber, Context Rot: How Increasing Input Tokens Impacts LLM Performance (Chroma, 2025) · What is an agent harness (Moda) · Harness engineering (Moda)

Prevention in production

How to prevent context rot in AI agents

To prevent context rot in AI agents, run a loop over the agent harness, driven by production traces: detect context loss, change the prompt, tool, skill, eval, or memory component responsible, and verify against replayed traffic. It is not a one-time prompt cleanup and not a vector database or session-store purchase. Agents are the worst case for context rot — every turn, tool call, and tool result accumulates in the window, so an agent manufactures its own rot by construction. The loop, step by step:

Start from production traces, not intuition. Detect the symptom — context loss: the agent re-asks for information the user already gave, contradicts an earlier turn, silently drops a mid-context constraint — across the whole trace population, and rank where degradation actually bites: which intents, which tools, which session lengths concentrate the failures.
Attribute each failure to a harness component. A dropped constraint can be rot in an overlong window, an over-aggressive compaction prompt, a memory store that failed to retrieve, or a tool result that buried the answer — one symptom, four different fixes, and the trace is what tells them apart.
Fix the component, not the vibe. Tune the compaction prompt on real long traces and trigger it at a token threshold before the window is under pressure; reshape tool schemas so results come back trimmed and grounded in the fields the agent uses, with consumed outputs masked behind a placeholder that names the tool and how to re-fetch; move durable state into explicit memory; isolate deep sub-tasks in sub-agents with clean windows that return condensed results.
Build evals on mid-context misses. Turn real traces where the agent lost mid-context information into regression evals, so every future prompt, retrieval, or compaction change is tested against the exact failure being prevented.
Verify against replayed traffic before shipping. A prevention technique never checked against your own traces is a guess; replaying the candidate change over real sessions is what turns it into a verified harness improvement.

Every step in that loop edits the agent harness — prompts, tools, skills, evals, memory — which is why preventing context rot belongs to harness engineering rather than to model choice or database choice. The contrast with the ingredient guides that rank for this question is the loop's two ends: a vector database can implement the retrieval step and a memory store the memory tier, but detection (which sessions are actually degrading) and verification (did the change stop the mid-context misses) run on your own production traffic, and no ingredient ships them. That loop is what Moda runs as a product for teams with an agent already in production: context loss detected across every trace, each detection attributed to the harness component responsible, each candidate fix verified against replayed production traffic before it ships.

SourcesWhat is an agent harness (Moda) · Harness engineering (Moda) · How to write agent evals from production failures (Moda)

The short answer

How to prevent context rot in AI agents: 7 steps

You prevent context rot in AI agents by engineering the harness around the model — controlling what enters the context window, and verifying every change against production traces — not by buying a bigger window or another database. The seven steps:

1.Retrieve just-in-time. Pull the relevant slice into the window when the task needs it instead of preloading whole corpora or full histories.
2.Compact instead of appending. Summarize completed turns and phases into decisions and constraints, drop the raw transcript, and trigger at a token threshold before the window is under pressure.
3.Trim tool outputs. Return only the fields the agent will use, with an identifier to re-fetch the full payload on demand — and mask consumed outputs behind a compact placeholder.
4.Move durable facts to explicit memory. State that must survive turns and sessions belongs in a memory store, not an ever-growing transcript.
5.Position instructions at the edges. Load-bearing constraints go at the start or end of the window, never buried mid-context.
6.Isolate deep sub-tasks in sub-agents. Give heavy work a clean context window that returns a condensed result — not its full working transcript — to the parent.
7.Verify against production traces. Detect where long contexts degrade behavior in real traffic, fix the harness component responsible, build regression evals on the mid-context misses, and replay traffic before the fix ships.

One caution before the detail: a vector database, session store, or RAG layer can implement steps 1 through 4, but none of them is prevention by itself — the ingredients section below explains why, and step 7 is the part ingredient-first guides leave out.

Ingredients vs. prevention

Do vector databases or memory stores prevent context rot

No — not on their own. Much of the guidance that ranks for preventing context rot comes from vendors of a single component: a vector database, a session-memory store, a RAG or context-management framework. The recipe those guides teach — chunk, embed, store, retrieve on demand — is a competent implementation of exactly one technique, just-in-time retrieval, and it happens to be the technique that requires their product. Each component can implement part of a fix, and none of them is the fix, because a component cannot tell you whether your agent actually stopped losing mid-context information.

A vector database implements just-in-time retrieval — the ingredient that keeps whole corpora out of the window. It does not detect that a forty-turn session silently dropped the constraint from turn three, and it does nothing about the compaction prompt, tool schemas, or instruction position that caused the drop.
A session or memory store implements the explicit-memory tier. A store that is never written, or that fails to retrieve on the turn that needed the fact, produces the same context loss it was installed to prevent.
A RAG pipeline can cause rot as easily as prevent it: stacking retrieved documents into the prompt in arrival order grows the window and pushes the answer into the low-attention middle.

What ingredient-first advice leaves out is verification. Prevention is a loop — detect where long contexts degrade behavior in your production traces, change the harness component responsible (a prompt, tool, skill, eval, or memory edit), verify the change against replayed traffic — and a loop is not something you install. Prevention is harness changes verified from production traces, not another store. That is the difference between buying infrastructure and doing harness engineering, and it is the loop Moda runs for teams with an agent already in production.

SourcesWhat is an agent harness (Moda) · Harness engineering (Moda)

Harness component

Compaction in agent harness design

Compaction in an agent harness is the step that summarizes older turns and tool outputs into the decisions and constraints that still matter, so the context window carries distilled state instead of raw transcript. It is the single highest-leverage defense against context rot in long-running agents — and it is a harness component like any other: implemented in a prompt, owned by the team, and capable of failing in ways that look exactly like the problem it prevents.

What it does. Replaces completed workflow phases, stale tool results, and older conversational turns with a compact summary that preserves decisions, constraints, and open questions — and drops everything else.
Where it runs. At phase boundaries in a skill, at a token threshold in the orchestrator, or before a sub-agent handoff — the anatomy of the agent harness determines the trigger points.
Why it prevents context rot. Shorter, structured context keeps load-bearing facts out of the low-attention middle and inside the length regime where the Chroma measurements show models stay reliable.
How it fails. An over-aggressive compaction prompt summarizes away the one constraint that mattered, producing context loss with the model working perfectly — which is why the compaction prompt needs its own regression evals built from real traces.
How to verify it. Replay production sessions against a candidate compaction prompt and confirm context-loss detections fall without new failures — compaction changes are harness changes, verified like any other.

Compaction is where the difference between assembling an agent harness and engineering one shows up most clearly. Any framework can call a summarizer at a token threshold; knowing whether your compaction step is quietly destroying constraints in production requires traces, detection, attribution, and replay verification — the loop Moda runs for teams with an agent already in production.

SourcesCompaction (Moda glossary) · What is an agent harness (Moda)

The research

What the Chroma context rot research measured

The Chroma report is the source to read first, and its design is why the term stuck. Instead of letting task difficulty grow along with input length — the flaw in most long-context benchmarks — it holds each task fixed and deliberately simple, varies only the number of input tokens, and measures what happens across 18 models, including GPT-4.1, Claude 4, Gemini 2.5, and Qwen3.

Performance degrades as input length grows, even on tasks the same models handle near-perfectly at short lengths — and the degradation is non-uniform rather than a smooth decay.
The lower the semantic similarity between the question and the answer buried in the context, the earlier performance falls off as length increases.
Distractors — text topically related to the answer but not quite it — hurt more as the context gets longer, and model families fail differently: GPT models showed the highest rates of confident hallucination, while Claude models tended to abstain under ambiguity.
The surrounding text itself matters: across all 18 models, performance was consistently better on shuffled haystacks than on logically coherent ones — evidence that models are sensitive to the structure of the context, not just its length.
On LongMemEval, a conversational question-answering benchmark, every model scored significantly higher on a ~300-token focused input than on the full ~113k-token chat history containing the same relevant information.

The report's conclusion is not that long context is useless. It is that whether relevant information is present in the context is not all that matters — how that information is presented matters more, which the authors name as the case for context engineering. Real workloads are far messier than the report's deliberately minimal tasks, so the measured effect is a floor, not a ceiling.

SourceHong, Troynikov & Huber, Context Rot: How Increasing Input Tokens Impacts LLM Performance (Chroma, 2025)

In production

LLM context rot in production agents

Benchmarks understate the problem for agents, whose sessions compound input length in ways no fixed-length test captures: a session that starts sharp is operating on a very different context by turn forty.

The mechanics compound quietly. Tool outputs are the usual bulk — one verbose API response or file dump can be tens of thousands of tokens, most of which is never referenced again. Multi-step workflows stack instructions from earlier phases that no longer apply. Retrieved documents pile on top of conversation history. Nothing errors: every call in the trace returns 200 and latency looks normal. The degradation shows up only in behavior — the agent re-asks for information the user already gave, contradicts an earlier turn, or follows the most recent instruction while quietly dropping the constraint from turn three.

Anthropic's guidance on context engineering explains why this is architectural rather than incidental: every token attends to every other token, so the attention budget is stretched across a number of pairwise relationships that grows quadratically with input length. Context is a finite resource with diminishing marginal returns, and agents — which Anthropic describes as generating the very data that fills their own windows — deplete it turn after turn by construction.

SourceAnthropic, Effective context engineering for AI agents

Position effects

Lost in the middle: why LLMs miss mid-context information

Two years before the Chroma report, Liu et al.'s Lost in the Middle: How Language Models Use Long Contexts documented the position half of the problem: models are best at using information at the very beginning or very end of the input, and measurably worse when the same information sits in the middle.

The paper tested multi-document question answering and key-value retrieval and found a U-shaped performance curve — a primacy bias toward the start of the context and a recency bias toward the end — with performance degrading significantly when models must access relevant information in the middle of long inputs. The effect held even for models explicitly built for extended context.

Context rot and lost-in-the-middle are two halves of the same engineering reality: how much you put in the window matters, and where it sits matters. Both break the naive mental model of a context window as random-access memory that merely has a capacity. The practical consequences follow directly — burying load-bearing instructions in the middle of a long system prompt is a design bug, and appending forever instead of compacting guarantees that important early turns migrate into the low-attention middle.

Liu et al. (arXiv:2307.03172) is the primary source for the lost in the middle LLM effect — read it for the measurements. What production teams do after the paper is harness work: audit where load-bearing constraints actually sit in the requests their agent assembles (the trace shows the exact payload the model received), adopt edge positioning and mid-context compaction as harness defaults rather than one-off prompt fixes, and gate every prompt or retrieval change with regression evals built from traces where mid-context information was missed. That turns a benchmark finding into verified harness changes — and it is the loop the rest of this page describes. This page is the practical companion: what the U-curve does to an AI agent in production, and the harness changes that fix it, covered next.

SourceLiu et al., Lost in the Middle: How Language Models Use Long Contexts (arXiv:2307.03172)

Position fixes

How to fix lost in the middle in AI agents

To fix lost in the middle in AI agents, change the harness, not the model: put load-bearing constraints at the edges of the window, compact the middle, and verify every change against replayed production traces. Liu et al. measured the U-shaped position curve even in models explicitly extended for long context, so switching models does not remove it — every lever that works is a harness change, not a retrieval-stack purchase. The five levers:

1.Reposition load-bearing content. Constraints and task-critical facts belong at the start or end of the window — a prompt change with a concrete layout rule: output contracts and standing constraints at the top of the system prompt, reference material in the middle, the live task restated at the end. Audit the assembled request from production traces — where the harness actually places things is a fact of the payload, not of the prompt template.
2.Re-rank what retrieval returns. When multiple documents come back, order them so the most relevant sit at the edges of the context instead of in arrival order — the U-curve makes position a free accuracy lever that most RAG defaults waste. This is a tool-level harness change: reorder inside the retrieval tool before results enter the window, and cap how many documents come back at all, because every marginal chunk pushes something else toward the low-attention middle.
3.Restate before use. For long sessions, have the harness re-surface the governing constraints near the end of the window — a standing reminder block a skill maintains — rather than trusting the model to reach back to turn three. Scope the block to constraints that govern output (format contracts, safety rules, standing scope limits) so it stays cheap in tokens and never becomes mid-context bulk itself.
4.Compact the middle. Summarize older mid-context turns so the window stays short enough that nothing sits deep in the low-attention zone — triggered at a token threshold before the window is under pressure, keeping decisions, constraints, and open questions. Compaction that preserves those shrinks the middle itself instead of just relocating it.
5.Eval on position-sensitive cases. Turn production traces where mid-context information was missed into regression evals, and verify each prompt, tool, or skill change against replayed traffic — position bugs regress silently otherwise, and the eval gate is what separates a harness playbook from a tips list.

Most of what ranks for this question is either an essay explaining the U-curve or mid-context tips for a RAG pipeline. Both stop where an agent's problem starts: an agent assembles its window fresh every turn from prompts, tool results, retrieved documents, and history, so the fix has to be installed in the components that do the assembling — and proven on the agent's own traffic, not on a benchmark harness.

Those are prompt, tool, skill, eval, and memory changes — the agent harness, which is why the fix belongs to harness engineering rather than to RAG tips or model selection. Moda closes the loop in production: mid-context misses are detected as context loss across production traces, each miss is attributed to the harness component that put the information where the model could not use it, and each candidate fix is verified against replayed traffic before it ships. That is not observability (watching the U-curve happen) and not an LLM router (every model tested shows the curve); it is turning traces into verified harness fixes that reduce lost-in-the-middle failures.

SourcesLiu et al., Lost in the Middle: How Language Models Use Long Contexts (arXiv:2307.03172) · What is an agent harness (Moda) · Harness engineering (Moda) · How to write agent evals from production failures (Moda) · Compaction (Moda glossary)

Disambiguation

Context rot vs. context loss

The two terms get used interchangeably and should not be. Context rot is a model-level mechanism: degraded ability to use what is in the window as the window grows. Context loss is the trace-level failure you actually observe: the agent forgets earlier turns, contradicts itself, or asks the user to re-supply information it was already given.

The distinction matters because context loss has more than one cause, and context rot is only one of them. An over-aggressive compaction step that summarizes away a key constraint, a memory store that fails to retrieve the relevant fact, a truncation bug in the harness, or a session boundary that silently drops state will all produce identical symptoms with the model working perfectly. Treat every context loss as context rot and you will reach for a bigger window when the actual fix is a better compaction prompt — and the bigger window will make the rot worse.

That is why detection and attribution have to be separate steps. Moda detects context loss as a first-class behavioral failure across production traces — one of six categories in its behavioral failure taxonomy, alongside tool misuse, reasoning loops, goal drift, hallucinations, and agent laziness — and attributes each detection to the harness component responsible before anything gets changed.

SourceContext loss (Moda glossary)

Detection

How to detect context rot in agent traces

Context rot never throws an error: every call in the trace returns 200, latency looks normal, and the token count just grows. Detecting context rot in agent traces means reading behavior, not infrastructure metrics.

Detect the behavioral signature. Flag traces where the agent re-asks for information the user already supplied, contradicts an earlier turn, or violates a constraint that is verifiably present earlier in the context.
Correlate with window growth. Rot's fingerprint is degradation that tracks input length: failures concentrate in long sessions, late turns, and information sitting mid-context, while short sessions on the same task stay clean.
Confirm the information was actually in the window. Check the request payload the model received: rot means the fact was present and unused; if it never made it in, the failure is upstream — truncation, compaction, or memory — and needs a different fix.
Detect across the population, not spot checks. Sampling a few sessions misses a failure that concentrates in the long tail; run detection over every production trace so you can see which intents, tools, and session shapes accumulate failures.
Turn confirmed detections into evals. Each verified mid-context miss becomes a regression eval — the instrument that later tells you whether a harness change actually reduced rot or just moved it.

This is the detection half of the loop this page keeps returning to, and it is what Moda runs continuously: context loss is one of six behavioral failure categories detected across every production trace ingested over OpenTelemetry, each detection is attributed to the harness component responsible, and each fix is verified against replayed traffic. Detection you have to remember to run is detection that quietly stops happening; on a harness engineering platform it runs on the whole population by default.

SourceBehavioral failure (Moda glossary)

The fix

Context rot is a harness failure

A failure you cannot fix in the model, whose symptoms appear in production behavior, and whose every remedy is a prompt, tool, skill, eval, or memory change belongs to the harness. Treating context rot as a model limitation to wait out — or as a metric to watch — leaves the one layer you control untouched.

Watching a context-length metric tells you the window grew; it does not tell you which traces degraded, which harness component is responsible, or whether your fix worked. The loop that does: detect context loss across the production population, attribute each detection to the component responsible — this compaction prompt, this verbose tool schema, this memory store that failed to retrieve — ship the change, and verify it against replayed traffic before it counts as fixed.

That loop is what Moda runs as a product, built for teams with an agent already in production: production traces in over OpenTelemetry, context loss and the other behavioral failures detected across every trace, each finding attributed to the specific harness component responsible, and each candidate improvement verified against replays of real traffic before it ships. The demo is the fastest way to see it against your own traffic.

SourceModa docs, Harness overview

Frequently asked

Questions

What is context rot in AI?

Context rot is the degradation of a large language model's performance as its input context grows longer. Even on simple tasks, recall and reasoning decline as tokens accumulate — non-uniformly, and differently across model families. The term was popularized by Chroma's 2025 technical report, which measured the effect across 18 models, and adopted by Anthropic's engineering guidance, which defines it as the model's decreasing ability to accurately recall information from the context as the context grows. It happens because transformer attention relates every token to every other token, so a growing window stretches the model's attention budget thin. In an AI agent it surfaces as forgotten turns, dropped constraints, and re-asked questions — and the fix lives in the agent harness, in what enters the window, not in a bigger model or window.

How to fight context rot with harness changes?

You fight context rot with harness changes by engineering the harness component by component. Prompt changes: better compaction prompts and load-bearing instructions positioned at the start or end of the window. Tool changes: results trimmed to the fields the agent uses, grounded with an identifier to re-fetch the rest. Skill changes: workflows encoded as phases that compact at each handoff. Memory changes: durable facts in an explicit store instead of the transcript. Eval changes: regression evals built from real mid-context misses. The harness is the agent harness — everything assembled around the model, unrelated to Harness the CI/CD platform — and what turns these from tweaks into engineering, rather than defeating context rot by anecdote, is deriving each change from production traces and verifying it against replayed traffic before it ships.

Why does my agent forget earlier turns?

Your agent forgets earlier turns because of one of five causes, and every fix is a harness-component change: context rot (the turn is still in the window but the model can no longer reliably use it), truncation (the turn was evicted when the session outgrew the window), over-aggressive compaction (a summary destroyed the specific fact), failed memory (never written or not retrieved), or a session boundary that silently dropped state. The trace tells you which one: check whether the turn was in the request payload the model actually received. The fix differs by cause — a bigger window only helps truncation, and it makes rot worse.

What is context loss in AI agents?

Context loss in AI agents is the observable failure where an agent stops acting on information it was already given. In multi-turn sessions it shows up as re-asking for known information, contradicting an earlier turn, silently dropping a standing constraint, or repeating tool calls whose results the session already had. Context rot is one cause; over-aggressive compaction, failed memory retrieval, truncation, and session boundaries are the others. The harness fixes: compact completed turns, trim tool outputs, move durable facts to an explicit memory store, and isolate deep sub-tasks in sub-agents — with each detection attributed to the responsible component and each fix verified against replayed production traffic.

What is LLM context rot?

LLM context rot is the same phenomenon stated at the model level: as a large language model's input context grows, its ability to accurately recall and reason over what is in that context degrades — unevenly, and on tasks it handles perfectly at short lengths. It applies to any LLM application, not just agents: chatbots with long histories, RAG pipelines that stack retrieved documents into the prompt, and summarizers fed whole corpora all hit it. Chroma's 2025 report measured it across 18 models and none was immune, so model choice is never the fix. Prevention is controlling what enters the window — the harness layer engineered around the model.

Is AI context rot the same as LLM context rot?

Yes. AI context rot, LLM context rot, and context rot in LLMs all name one phenomenon — the degradation of a large language model's ability to use its input as the context grows — coined as context rot by Chroma's 2025 technical report and used that way throughout this page. The synonyms matter only for search; the engineering is identical: the effect lives in the model, prevention lives in the harness, and detection means reading behavior across production traces rather than watching infrastructure metrics.

What is the context rot paper?

The context rot paper is Context Rot: How Increasing Input Tokens Impacts LLM Performance, a July 2025 technical report from the Chroma research team (Hong, Troynikov & Huber) — often searched as the Chroma context rot report or just context rot Chroma. It coined the term and made the effect measurable: task difficulty held fixed, only input length varied, 18 models tested, with needle-question similarity, distractors, and haystack structure all shifting the outcome. The original report is linked as the source under the research section of this page; this page covers what the paper measured and what to do about it in a production agent harness.

What is compaction in an agent harness?

Compaction is the harness step that summarizes older turns and tool outputs into the decisions and constraints that still matter, replacing raw transcript with distilled state as a session grows. It is the primary defense against context rot in long-running agents — and a harness artifact in its own right: the compaction prompt can silently destroy the one constraint that mattered, so it needs regression evals built from real traces and replay verification before changes ship. Compaction in agent harness design is covered in depth in the section above.

What causes context rot in LLMs?

The architecture. Transformer attention relates every token to every other token, so the attention budget is stretched across a number of pairwise relationships that grows quadratically with input length, and models see far less long-sequence data in training than short-sequence data. The measurable consequences: lower needle-question similarity fails earlier as length grows, distractors hurt more at length, and even the structure of the surrounding text changes outcomes.

Is context rot the same as lost in the middle?

They are related but distinct findings. Lost in the middle (Liu et al., 2023) is a position effect: models use information at the beginning or end of the context better than the same information in the middle, producing a U-shaped performance curve. Context rot (Chroma, 2025) is a length effect: performance degrades as total input grows, wherever the information sits. In practice both apply at once — a long context degrades overall, and its middle degrades most.

What is lost in the middle in LLMs?

Lost in the middle is the measured tendency of large language models to use information at the beginning or end of their input context far better than the same information placed in the middle. Liu et al. (arXiv:2307.03172) documented it on multi-document question answering and key-value retrieval: performance follows a U-shaped curve — a primacy bias toward the start and a recency bias toward the end — and degrades significantly when the relevant information sits mid-context. The effect held even for models explicitly built for extended context, which is why it is treated as an architectural property to engineer around, not a bug a bigger window fixes.

How do I fix lost in the middle in AI agents?

You fix lost in the middle in AI agents with harness changes that control where information sits when the model reads it: put load-bearing constraints at the start or end of the window, re-rank retrieved documents so the most relevant land at the edges rather than in arrival order, have a skill restate governing constraints near the end of long sessions, and compact older mid-context turns — at a token threshold, keeping decisions and constraints — so nothing sits deep in the low-attention zone. Then build regression evals from production traces where mid-context information was missed, and verify each change against replayed traffic. Every one of those is a prompt, tool, skill, eval, or memory edit — switching models does not remove the U-curve, since every model tested shows it.

Is context rot the same as context loss?

No. Context loss is the behavioral failure you observe in a trace — the agent forgets earlier turns, contradicts itself, or re-asks for known information. Context rot is one mechanism that produces it. Context loss can equally be caused by over-aggressive compaction, failed memory retrieval, or truncation bugs in the harness, with the model performing perfectly. Diagnosing which one you have determines the fix, which is why detection and attribution are separate steps.

How to prevent context rot?

You prevent context rot by controlling what enters the context window, not by buying a bigger one. The techniques that work: retrieve the relevant slice just-in-time instead of preloading everything, compact completed turns and phases into summaries at a set token threshold, trim verbose tool outputs to the fields the agent uses, move durable facts into an explicit memory store, isolate deep sub-tasks in sub-agents with clean windows, and place load-bearing instructions at the start or end of the window. Every one of those is a harness change — a prompt, tool, skill, or memory edit — and each should be verified against evals built from real traffic.

How to prevent context rot in AI agents?

You prevent context rot in AI agents by running a loop over the agent harness, driven by production traces — not by installing a vector database or session store. Agents fill their own windows (every turn, tool call, and tool result accumulates), so: detect context loss — re-asks, contradictions, dropped mid-context constraints — across real traces; attribute each detection to the harness component responsible, whether a compaction prompt, a verbose tool schema, or a memory store that failed to retrieve; fix that component; build regression evals on the mid-context misses you found; and verify the fix against replayed traffic before it ships. A prevention technique you never verify against your own traffic is a guess.

How do you detect context rot in agent traces?

Look for the behavioral signature, not an error code: re-asks, contradictions, and dropped constraints whose source information is verifiably present earlier in the context, concentrating in long sessions and mid-context positions. Confirm the fact was in the request payload the model actually received — that separates rot from truncation or compaction loss. Run detection across the whole trace population rather than spot-checking, and convert confirmed misses into regression evals. Moda detects this as context loss, a first-class behavioral failure, across every production trace.

Do vector databases or memory stores prevent context rot?

Not by themselves. Retrieval infrastructure and session-memory stores can implement parts of a fix — retrieval keeps whole corpora out of the window, a memory tier moves durable state out of the transcript — but installing one confirms nothing about whether the agent stopped losing mid-context information. Prevention is controlling what enters the window and verifying, against your own production traces, that the failure went away. That verification step is what component-first guides leave out; it is harness engineering, not infrastructure selection.

Does a bigger context window fix context rot?

No. The Chroma results show degradation begins well before advertised limits and is non-uniform — the marginal token is not free just because the window officially has room for it. A bigger window raises the ceiling on how much rot you can accumulate; it does not change the mechanism. Teams that handle long-running sessions well spend their effort on what enters the window, not on how big it is.

How does Moda help with context rot?

Moda is a harness engineering platform for teams with an agent already in production. It detects context loss — the production symptom of context rot — across every trace as part of a six-category behavioral failure taxonomy, attributes each detection to the harness component responsible (a compaction prompt, a verbose tool schema, a memory store that failed to retrieve), and verifies each candidate fix against replayed production traffic before it ships. Production traces in, verified harness improvements out.

Keep reading

What is an agent harness (pillar)→Harness engineering (pillar)→Agent evals (pillar)→Context loss (glossary)→Behavioral failure (glossary)→Agent harness (glossary)→Moda vs Raindrop→Moda vs Letta→

For AI agentsThis page as Markdown · llms.txt · Agent skills index · Claude Code skill

Find where context rot is costing your agent.

Moda is a harness engineering platform: it detects context loss across production traces, attributes each failure to the prompt, tool, skill, eval, or memory component responsible, and verifies the candidate fix against replayed traffic before it ships.

Book a demo