Six failure modes, six fixes

The clearest reason to learn the six-tuple is that it lets you name what's wrong when an agent misbehaves. Almost every recurring failure mode in long-running coding agents falls into one of six bins.

The simulator, with components disabled

The cleanest way to feel each failure mode is to disable the component it corresponds to. Below is a small harness simulator running two real-shaped tasks. Try disabling C, S, L, or V in turn and see what fails. (E and T are required by definition; you can't run without them.)

Harness simulator

Pick a goal, pick which components are enabled, and step the harness through. Disable a component to see what it actually does.

step 1 / 8

Goal

Fix the failing test in auth_service_test.pyBuild a small TODO app and verify it works

A short, well-scoped task. Three turns, one tool call each.

Components

EexecutionTtoolCcontextSstateLlifecycleVevaluation

E and T are required by definition. Click C, S, L, or V to disable.

Context window

1,200 / 12,000 tok

Persisted state (S)

(nothing yet)

Trace

1. T1observe✓ load context

   Read CLAUDE.md and recent file cache.

playstep1×reset

A few patterns the simulator makes visible:

• Disable C on the longer task and the context bar overflows partway through. The harness aborts.
• Disable S and the task technically completes its current feature, but no progress file gets written. The next session would start blind.
• Disable L on Bash and the audit/permission step skips. The call still runs (that's the point). Without L, side effects are unmonitored.
• Disable V and the trace stops capturing structured trajectory entries. Steps still execute; you just can't reproduce them or score them later.

The principal six (survey)

Failure modes specific to long-running coding agents

The four mapped by Anthropic in the long-running-agents post (source: 2511.Effective harnesses for long-running agents.md) are special cases of S and C:

1. Declaring victory too early. A later session sees prior progress and concludes the project is done. Fix: an exhaustive feature_list.json with passes: false flags. The agent can only flip a flag after end-to-end verification.

2. Leaving the environment buggy or undocumented. Fix: initial git commit, progress file, and init.sh written by the initializer; coding agent always commits and updates progress at the end of a session.

3. Marking features done prematurely. Fix: end-to-end browser-based self-verification before flipping the flag. Unit tests aren't enough; the model needs to see the feature work.

4. Wasting time figuring out how to run the app. Fix: init.sh produced once by the initializer, run at the start of every later session.

All four are S-component failures dressed in coding-agent clothes: state that should have survived didn't, or wasn't read when it should have been.

Failure modes from the meta-loop

Once you start automating harness improvements (lesson 12), two new failure modes show up that the original six don't fully capture:

Regression blindness. AHE's measurements (source: 2604.auto-evolution-coding-agents.pdf): the Evolve Agent's fix-precision is 5× chance, but its regression-precision is only 2× chance. It can name what an edit will help. It cannot reliably name what it will break.

The fix is decision observability: a versioned change manifest where every edit predicts both fixes and regressions, and the next round verifies or falsifies the predictions. Bad edits get rolled back at file granularity.

Verifier–benchmark misalignment (NLAH paper, source: 2603.natural-language-harnesses.pdf). A verifier may approve a locally plausible repair that misses the benchmark evaluator's acceptance criterion. Adding more structure can move acceptance signals away from benchmark acceptance.

The fix is to align verifier criteria to the final acceptance object (path-addressable artifacts, explicit object checks) rather than to screen plausibility.

Failure modes from multi-agent and approval state

Two more failure modes show up once a harness involves multiple agents or stores human approvals as state. The Code-as-Harness survey names them as open problems with concrete harness implications. (source: 2605.code-as-harness.pdf, §5.2.4 and §5.2.5)

Stale-state conflicts in shared work. A planner reads the repo at SHA a83c, a tester checks out SHA c12e, a memory entry preserves an obsolete invariant, a reviewer adds a constraint that never reaches the planner. Git detects textual conflicts; it does not detect that a later edit invalidated a prior plan, test, or assumption. The fix is transactional shared program state: each action declares its read set, write set, assumptions, verifier obligations, and conflict policy, so the harness can re-verify or escalate when a dependency moves under it. (source: 2605.code-as-harness.pdf, §5.2.4)

Approval drift. The harness asked the user "may I run the migration?", got a yes, and now treats that as standing permission for a slightly different command, on a different branch, an hour later. Symptoms: approval laundering, evidence mismatch, scope creep, untracked denials. The fix lives in L (lesson 6): make approvals first-class state with action, evidence, scope, expiration, and a consumption log, so later calls can be checked against the original grant rather than reusing a stale yes. (source: 2605.code-as-harness.pdf, §5.2.5)

The discipline this gives you

When something goes wrong with an agent, the question to ask first is which component is at fault. Because the failure-mode space is small, the answer is usually obvious within thirty seconds of reading the trace:

• "The agent re-did work it already did last session" → S
• "The agent kept calling a tool that doesn't exist" → T
• "The agent stopped early for no clear reason" → likely C (context anxiety) or E (a missing recovery arc)
• "I can't reproduce what the agent did yesterday" → V
• "The agent shelled out and ran something I didn't want" → L

Each of those points to a different fix. None of them are model fixes.

Quick check

An agent successfully implements a feature, but the next session re-implements the same feature from scratch. Which component is failing?