What is an agent harness, and how does a coding harness work?

A model asked to fix a bug can request a file, propose an edit, or ask to run a test. Something has to execute those requests, return the results, and decide whether another model call should follow. That surrounding software is an agent harness.

This article explains coding harness architecture through Pi 1.0. We use it to show how a harness handles tool execution and user redirection, then examine request context and partial failure. The final checklist gathers tests you can adapt to your own harness.

What does an agent harness do?

An agent harness is the software around a model that manages its execution. It prepares the model's inputs and dispatches tool requests, then feeds the results back. The model selects actions within the tools and constraints the application exposes. The harness carries out the surrounding control flow.

Harnesses also support email, research, and other applications. Anthropic describes the harness behind Claude Code as supporting these uses. An email agent might search messages and fetch attachments; a coding agent might read a repository and execute tests. Those are different toolsets and tasks, and can share a runtime.

How much the model controls varies. Anthropic distinguishes workflows with predefined code paths from agents with model-directed tool use. We focus on agents' feedback loop. A request produces an action whose result informs the next request. Execution continues until a stopping condition or human intervention.

How does a coding harness work?

Coding gives that loop files to inspect and edit, commands to execute, and test results to use as feedback. A failing test can lead to another edit and another run. Passing tests provide evidence about the checked behavior; human review still matters for broader requirements.

The diagram groups the responsibilities in a coding harness. They can share one process or be split across services.

This pseudocode shows the main flow. It groups the responsibilities above; Pi's implementation handles streaming and control signals in more detail.

while the run is active:
    prepare request context from the current state
    get a model response
    record the response

    if the response requests tools:
        check completeness, arguments, and permissions
        execute allowed calls and collect results or errors
        record those results for the next request
    else:
        deliver the response to the user

    apply queued input and the stopping rule

The agent loop coordinates each model response, tool call, result, and subsequent request. A response can also be an answer to the user. The application needs rules for continuing, waiting for input, and stopping. It also needs a limit for work that keeps iterating.

Request context contains the instructions, selected messages, and tool declarations for one model call. Session records retain the conversation or other state. Events expose progress. A stored transcript can contain information omitted from the current request. Compaction, retrieval, and filtering are ways to manage that selection.

Tool execution and policy checks and dispatches requested operations. Tools act in an execution environment or external service, such as a local shell, container, or issue tracker. Which operations are allowed depends on the implemented controls and the permissions of that destination. Input and control carries the initial task and later steering or cancellation signals.

A short task can use in-memory messages. Persistent sessions, compaction, approvals, sandboxing, subagents, and script composition are options to add when the application needs them.

Long-running work adds another requirement: the next request must contain enough information to continue. In Anthropic's long-running web-app experiment, compaction alone did not ensure reliable continuation. The solution also included progress artifacts and verification. Its agents committed progress to Git and used that history to revert bad changes and recover working code. Saved state and summarized context need to be tested against the work an agent must resume.

What can Pi teach us about those choices?

The case study uses Pi v1.0.0 at commit a13d35a. Two comparisons use Codex 0.162.1 at commit 092d3ac. Our earlier Pi guide covers setup and local models.

For the examples below, a Pi assistant turn consists of a response and its tool results. One user-requested run can contain several turns.

How we tested the examples

The runnable companion and instructions use Pi's packages pinned to 1.0.0. The agent-loop tests supply replies we wrote in advance. No real AI model is called. Fake tools record their effects in lists held in memory. They do not edit files or contact a live service. The script tests use the real QuickJS sandbox with injected fake tools.

The companion contains an eight-case suite and a shorter version of the partial-failure test that prints receipts. The table names the cases used below. Persistence, session projection, compaction, and extension-hook routing were inspected in source. The tests do not verify disk-crash recovery, live MCP, operating-system isolation, or Codex execution. They also do not measure how often a real model retries a failed script. Cancellation uses cooperating tools.

Agent loop and tool execution

A tool request needs a readiness check before execution. Checking the argument schema catches malformed values, but you also need to check whether the model finished producing the request. A provider can reach its output-token limit halfway through a string. Repairing the JSON can leave a value that parses successfully but describes only part of the intended action.

Treat completeness, argument validation, and permission as separate gates. Pi checks completeness first. Its truncation guard rejects every call in a response stopped by the output-token limit. Its execution pipeline then validates arguments and runs hooks before execution and after results return.

The guard's reason is documented in the loop's comment: repaired arguments can parse and validate while remaining incomplete. The truncation test supplied the schema-valid string "hel" directly, bypassing the partial-JSON parser. A complete response executed it once; a response marked truncated executed it zero times. Rejected calls became error results for the next model request.

Tools can finish in a different order from the calls that requested them. Sequential execution makes call order explicit. Parallel execution lets independent operations finish in a different order, so decide which order the next request receives. Pi's parallel mode emits completion events as tools finish but keeps result messages in the assistant's call order. The parallel-ordering test completed the second tool first and confirmed that distinction. Pi also emits start/end events for rejected calls, so event names alone do not count completed actions.

Policy hooks can block a call before it runs; isolation limits what a running call can reach. Pi's security documentation says extensions run with the host account's permissions and remain outside isolation that covers only built-in tools. Codex's orchestrator applies configured approval policy and selects a sandbox, whose restrictions depend on the tool and supported environment.

The stopping rule also belongs in the loop. At this pin, the inspected Pi core loop has no built-in numeric turn cap. It continues for tool work or queued input. A terminating tool batch stops tool-driven continuation, but queued input can still continue the run. An optional finish-turn hook can end it. A builder can add a budget or completion check there. For a coding task, define which test command and checks must pass before accepting completion. Set a stopping point for repeated unsuccessful edits so the agent can ask for user input.

Input and control

Decide when a new instruction takes effect. Suppose the user says, "Stop editing and explain what you found." The harness could apply that instruction after the current tool finishes or after the whole batch. It could also defer the message until the run stops. For cancellation, define which components receive the signal and how they respond. Account for actions that have already completed.

Pi chooses a batch boundary for steering. Steering is polled after the current assistant response's tool calls finish, unless finishTurn ends the run. Follow-up input waits until the agent would otherwise stop. A second prompt during an active run is rejected; callers must queue input or wait.

The steering test queued a message during the first tool.

First tool starts.
User queues a steering message.
First tool finishes.
Second tool runs and finishes.
Steering message enters the conversation.
Next model request includes the steering message.

A redirect button built around that contract should tell the user that pending calls will finish first.

For cancellation, Pi signals an AbortController and passes the signal to tools. The cancellation test left one completed effect intact and skipped the next sequential call.

Remote control adds the possibility that the active run changes before a message arrives. You can require the client to identify its target run or treat the message as session input. Codex's steering handler requires expectedTurnId and rejects a mismatch. Pi's RPC steering command has no target turn ID; its optional id correlates request and response.

Request context and session records

Reducing context can change the saved history or produce a separate view for each request. Replacing old messages can keep the representation simple. Retaining the originals gives you more material for debugging. In either design, capture the actual request so you can establish what the model saw.

Pi retains session entries and projects them into request context. Context edits can omit or replace content in the model's view while preserving the stored entry. Compaction combines a summary, retained earlier entries, and later messages. Its parent-linked records let the request follow the active branch rather than every entry in the file.

Choose what must survive context reduction. Tool results need enough surrounding information to explain which calls produced them. Pi's cut selection avoids a standalone tool result as a cut point and can summarize an earlier part of a split turn. It also rejects summaries stopped at the output-token limit.

A completion event can mean that a tool returned, that a message was recorded, or that a required durable write finished. Define which acknowledgment your application needs. Pi dispatches completed-message events before appending the messages to its session record. The JSONL writer we inspected uses synchronous writes without an fsync call.

Tool composition and partial failure

A multi-step operation can leave completed work behind when it fails. If the first action succeeds and the second fails, rerunning the whole operation can repeat the first effect. You could expose each action as a separate model call, or let a script compose several calls and return selected output. Either way, record enough information to identify completed work before repeating it. For a coding example, suppose a script edits two files, then runs a test. If the first edit succeeds and the second fails, inspect the workspace before retrying. The changed file is part of the recovery state, alongside the script's output and saved receipts.

Pi illustrates the script approach with Codemode. The model writes JavaScript that calls tools in a QuickJS sandbox. Intermediate results can stay inside the script. The output-filtering test fetched 100 fake records and returned only {"open":50}. That reduces what enters request context, but it also groups several actions behind one script result.

Pi's store() lets scripts save JSON values for later Codemode calls. Its adapter commits new store writes only if the script succeeds. The successful-store test confirms that a completed script returns its store writes. Completed tool actions are not undone after failure. The short partial-failure test and run instructions use the real sandbox and a fake service:

const receipt = await tools.record({ value: "created" });
text({ completed: "record", receipt });
store("receipt", receipt);
throw new Error("later failure");

The short version runs this twice. Both runs complete the fake action and print its receipt. Each then calls store() and throws. Neither returns store writes. Partial output retains the receipts.

Pi does not initiate the rerun. A model receiving the failed output may choose another script.

Print a receipt immediately after each completed action. An issue ID, for example, gives the model something to fetch before creating another issue. Output limits or a model ignoring that receipt can still lead to repetition. Where the service supports idempotency keys, reuse a stable key for the same action. Otherwise keep an action journal and reconcile it before retrying. We did not test those approaches.

Also inspect returned error fields. An operation can resolve with an error field without throwing. Your receipt must report confirmed completion.

What should you test in your own harness?

Copy the companion code as a starting point for your own tests. Use predefined model replies and observable tools to check these boundaries:

• Stop a response at the token limit during a tool request whose arguments still validate. Instrument the tool body to check whether it runs.
• Reverse two tools' completion order. Compare progress events with the result order in the next request.
• Steer during a slow tool batch, then cancel in a separate run. Check pending calls and the next request. Send a remote message intended for a run that already ended.
• Keep the model requesting tools. Check your run budget and completion rule, including the required test command for a coding task.
• Emit a completion event and inspect whether the required record has been saved yet. Check the ordering of events and persistence acknowledgments.
• Reduce context after a tool result and compare saved history with the actual outgoing request. Repeat with a failed summary and inspect what remains available.
• Complete an action, fail the next step, and rerun explicitly. Check which effects repeat and which receipts survive. For file edits, compare the workspace with its checkpoint before deciding what to repeat.

Test recovery after process or disk failure separately. Idempotency at a real service also needs its own tests. Our Pi Durable section covers a separate experimental framework for work that must survive restarts. Start with the failure your application would find hardest to reconcile. Define what your harness must preserve before implementing its retry behavior.