AI Agents17 min read

Code Mode for AI Agents: Move the Work Out of the Context Window

Code Mode for AI Agents: Move the Work Out of the Context Window
Instead of making an LLM mediate every tool operation, code mode lets agents write small programs to execute mechanical work in a sandbox. Learn how Arize Phoenix, Cloudflare, and Anthropic MCP move execution out of the context window.

An agent asked to collect invoice attachments should not need to read every attachment into its conversation, explain each download, and reason again before moving to the next message. Much of that task is ordinary software: search, iterate, filter, save, and report.

That observation is the starting point for code mode. Instead of making a large language model (LLM) mediate every operation, let it write a small program that performs the mechanical work. Give that program controlled access to useful capabilities, and return the evidence the model actually needs.

The result is not an agent without tools. It is an agent that uses tools through code.

Arize AI’s experience with Phoenix provides a concrete case study: shell-based composition, sandboxed Python, SQL over traces, browser-side application control, and evaluation across different agent interfaces. The architectural lesson is broader than any one product: use the model for judgment, and executable code for work that does not require another judgment. [1][2]

💡

Use the language model for judgment and ambiguity resolution, and executable code in a sandbox for work that does not require another judgment—keeping mechanical iteration and bulk payloads out of the prompt.

Why the conventional agent loop becomes expensive

A conventional tool-calling agent receives tool descriptions, selects an operation, generates structured arguments, and waits while its harness executes the request. The harness is the surrounding software that manages model calls, tool execution, context, and the agent’s lifecycle. The result returns to the model, which decides what happens next. [2]

This works well for a few bounded operations. Problems emerge when the agent must coordinate a large tool catalog, process substantial data, or work across many steps:

  • Definitions consume context. The model may receive descriptions for capabilities irrelevant to the current task.
  • Intermediate results become conversation material. Pages of records and large documents enter the prompt even when the model only needs a count or a few matching fields.
  • Mechanical steps become model round trips. Pagination, copying values, and applying known filters repeatedly interrupt execution.
  • Long trajectories require context management. The harness may need to summarize or compact earlier material, potentially losing details needed later. [2][3]

A useful analogy is a supervisor who insists on inspecting every item moving along a conveyor belt. Inspection matters at decision points. It is wasteful when the next step is already determined.

There are important qualifications. Direct tool calling does not inherently require one model turn per operation: some systems support parallel calls, dynamic discovery, filtered outputs, and purpose-built batch tools. Code mode should be compared against a competent baseline, not an unnecessarily inefficient implementation. Anthropic’s advanced tool-use work explicitly separates tool search from programmatic tool calling, showing that these are complementary mechanisms rather than a single indivisible feature. [4]

What code mode actually changes

The Context Bottleneck: Conventional Tool Calling vs. Code Mode Sandbox Execution

In code mode, the model writes an executable plan against a capability interface. The runtime executes that plan, which can call functions, retain intermediate values, branch, loop, and return a shaped result. Cloudflare describes the same pattern as code that composes tools rather than requesting every operation separately. [5]

A typical arrangement has four responsibilities:

  1. Discovery: Find relevant capabilities and inspect their contracts.
  2. Composition: Generate code using those capabilities.
  3. Execution: Run the code inside a controlled environment.
  4. Return: Send a bounded result back to the model for interpretation or the next decision. [3][5]

The model does not execute the program itself. The harness or a connected service does. Nor does code mode require exactly two outer tools: search and execute are a common design, not a defining requirement. Some implementations provide tool definitions up front; others expose discovery inside the runtime. [5]

The Model Context Protocol (MCP) supplies a standardized connection between agents and external tools or data. Code mode changes how those capabilities are consumed; it does not replace MCP. An MCP server can expose code execution, or a harness can wrap MCP tools as callable functions in its own environment. [3][6]

A worked example: collecting invoices

Consider an illustrative request: collect invoice attachments from a mailbox for a specified period and place them in a folder.

A code-mode agent could discover the mailbox and file capabilities, then generate a program that:

  1. Searches the permitted mailbox scope.
  2. Iterates through result pages.
  3. Selects matching attachments using an explicit rule.
  4. Downloads them through approved interfaces.
  5. Records saved files and failures.
  6. Returns a concise manifest rather than every message body and attachment.

Worked Example: The Invoice Collection Pipeline Data-Flow and Intermediate Payload Pruning

The model may still need to resolve an ambiguous definition of “invoice.” But once the selection rule is established, pagination and downloading do not require repeated language-model reasoning.

This also explains the broader intuition about coding agents: a user can request a temporary piece of software without writing it personally. That does not mean every task becomes solvable. Available interfaces, permissions, execution limits, and the model’s ability to generate correct code still define the boundary. [2][3]

Four benefits, with four different mechanisms

1. Progressive disclosure reduces irrelevant tool context

The agent can load only the definitions needed for the task. Discovery might use search, typed documentation, or files representing callable functions. [3][5]

But discovery is not free. A poorly designed catalog can replace a large initial prompt with repeated searches and exploratory calls. Function names, input types, return shapes, examples, and structured errors matter because the model must learn enough to write correct code. [2][4]

There is also no guarantee that every code-mode surface uses selective discovery. Phoenix’s documented browser implementation returns its entire browser-action catalog after a search, with the query affecting ranking rather than filtering. That is a deliberate design choice: hiding currently unavailable actions made the agent incorrectly conclude that those actions did not exist. [7]

2. Composition moves control flow into software

Code supports sequential dependencies, bounded parallel work, fan-out, filtering, and conditional execution. Several operations can happen within one execution request without another model call between them. [3][5]

This is related to, but distinct from, composing subagents. A loop over application programming interface (API) calls coordinates operations; a workflow over subagents coordinates additional reasoning systems. Both can use code as an orchestration language, but their execution costs and failure modes differ. [2]

The limiting principle is simple: batch calculations, not unresolved judgments. If the next step requires the model to interpret an unexpected result, the program should return relevant evidence and let the agent reason again. Phoenix’s browser implementation explicitly makes this distinction. [7]

3. Result shaping keeps bulk data out of the prompt

Intermediate values can remain in the execution environment. Code can project fields, count matches, join results, or select examples before returning them. [3]

This is not merely shorter output. It avoids using the model as a transport layer for data that software can pass directly between operations.

Still, returning too little can hide mistakes. A useful result contract should preserve enough evidence to verify the answer: the selection criteria, scope, count, relevant identifiers, failures, and whether anything was truncated. This is an architectural recommendation, not a universal code-mode feature.

4. Generated helpers expand what the agent can compose

An agent with read and update functions can write a transformation that was never exposed as a dedicated tool. One example is removing personally identifiable information (PII) from records before returning a summary. Arize’s written explanation uses the same example to illustrate task-specific helper generation. [2]

The important correction is that a traditional agent is not categorically incapable of redaction. It could have a redaction tool or another execution capability. Code mode makes it possible to construct the missing transformation from available primitives.

And generated redaction is not automatically reliable. A pattern matching one kind of number is not a complete PII detector. Sensitive workloads need tested transformation logic and checks for missed fields, false positives, and accidental disclosure.

⚠️

Data-Flow vs. Privacy Guarantee: Keeping data out of model context is a data-flow property, not a privacy guarantee. Returned values, console output, exceptions, execution traces, and stored artifacts all need appropriate handling. [3]

Phoenix shows that code mode has several execution surfaces

Arize’s talk describes multiple ways of applying the pattern inside Phoenix, its open-source AI observability and evaluation platform. They solve different problems and should not be treated as interchangeable sandboxes. [1]

Arize Phoenix: The Spectrum of Code Mode Execution Surfaces Across Shell, Python, SQL, and Web Workers

Bash: compose existing command-line capabilities

The talk describes starting with Bash, combining Linux utilities such as grep and sed with Phoenix’s command-line interface (CLI), and writing intermediate material to files. This gives the agent a familiar composition environment rather than requiring a dedicated tool for every transformation. [1]

Files can keep large payloads outside the conversation and make later inspection possible. Their usefulness depends on workspace lifetime, access restrictions, and cleanup. A temporary file is not automatically durable agent memory.

Python: expose application operations as functions

Phoenix’s server-side MCP implementation runs generated Python in a restricted Monty sandbox, with an interface shaped around application operations and SQL access. The talk describes exposing Representational State Transfer (REST) endpoints for create, read, update, and delete (CRUD) operations through the coding environment so the agent can compose trace retrieval and dataset manipulation. [1][7][8]

The distinction matters: a restricted interpreter is not necessarily a full Python environment with arbitrary packages, filesystem access, and networking. The available capability interface determines what generated code can actually do.

SQL: move aggregation to the database

Structured Query Language (SQL) expresses queries over relational data.

A trace records an execution; a span represents an individual operation within it. Questions about LLM-span latency, error counts, or trends often require aggregation rather than retrieval of individual objects. [8][9]

For example, the 95th percentile (p95) latency describes a distribution across a selected population of spans. It is not an attribute each individual span possesses. The query must define the relevant project, time range, span kind, and grouping before calculating the percentile.

SQL lets the database perform filtering, joins, and aggregation without paging all underlying records through the model. Code mode can then compose schema inspection and query execution while returning only the needed result. These are two separate gains: query pushdown reduces data movement; code composition reduces model-mediated orchestration. [8]

Phoenix’s documented SQL surface includes describeSqlSchema and executeSql. Its published implementation parses and validates queries, restricts accessible telemetry tables and columns, rebuilds statements, applies database-level checks, and bounds runtime and result size. The built-in MCP server is documented for Phoenix 19.0.0 and later, with SQL tools available from 20.2.0. [8]

Browser-side code: operate the application’s state

Phoenix also exposes user interface (UI) operations through a generated JavaScript interface. PXI, its built-in agent, generates a script that runs in a Web Worker, with calls routed back to registered handlers in the application’s main thread. [7]

This is not screenshot-driven computer use. It is programmatic access to application actions and state. It can change a time range, navigate, configure playground comparisons, or operate datasets through the same handlers the interface uses. Browser-local state matters here: a backend API alone cannot necessarily represent an unsaved comparison or immediately synchronize the visible interface. [7]

The documented design validates each action, checks availability and session state, and requires approval for scripts that mutate state. Workers are created per script, terminated afterward, and return ephemeral results subject to output limits. [7]

PXI Browser Agent Session and Trace Observability Dashboard: Web Worker Execution with Human Approval Gate and SQL Pushdown Spans

A Web Worker is not, by itself, a complete security boundary. Arize’s implementation discussion explicitly acknowledges limitations, including the possibility that a script’s human-readable description misrepresents its actual behavior. Approval and isolation need to be assessed against the real execution path, not inferred from the word “sandbox.” [7]

The complexity moves into the runtime

Code mode removes some orchestration from the model loop. It does not remove the need to control and observe execution.

Resources, networking, secrets, and permissions

Generated programs need CPU, memory, execution-time, and filesystem limits. Hosted environments also need lifecycle management and recovery from startup failures. Network access deserves a separate decision: unrestricted outbound connectivity creates paths for unintended data movement and exposure to untrusted content. [1][2]

Prefer capability-mediated access where appropriate. Cloudflare’s documented executor, for example, blocks external network calls by default and keeps connector credentials outside generated code. That is one implementation’s contract, not a property shared by every sandbox. [10]

The talk also raises the possibility of making additional model calls from inside a sandbox. Treat that as a design option, not a verified dependency on a particular library: the transcript’s library reference is unclear. Such calls introduce another credential boundary and another place where data can leave the environment.

Permissions must hold across the whole composition. A sequence of individually permitted operations can still produce an unintended combined effect. The runtime should enforce access at capability boundaries rather than trusting the model’s plan or assuming that initial approval covers every future action. [2][7]

Debugging now includes program semantics

A valid tool schema is no longer enough. The program may use the wrong field, misunderstand pagination, update the wrong records, or fail after completing some writes. [1][2]

Capture the generated code, the capability calls it made, and the final returned value. Preserve structured errors and relevant execution metadata while applying redaction and retention rules. Otherwise, context efficiency becomes an observability blind spot. Arize’s code-mode guidance specifically recommends retaining these three records. [11]

Nested code mode can duplicate boundaries

A coding harness may already have an execution environment while an MCP server provides another. Results can then cross multiple boundaries, be serialized into JSON, and be parsed again in the outer environment. This adds debugging, artifact-transfer, and lifecycle complexity. [1][2]

Two layers are not inherently wrong. A server-side boundary may protect data or enforce service-specific controls. The question is whether each layer owns a necessary responsibility, or whether it merely duplicates discovery and execution already available elsewhere.

Open problems are implementation-dependent

Durability requires more than a long script

A plain sandbox does not provide automatic recovery from an arbitrary failed line. Retrying a program after partial writes can repeat side effects. Long-running tasks need explicit decisions about checkpoints, idempotency, execution logs, and compensation. [1]

However, “code mode cannot resume” is too broad as a current claim. As of October 28, 2026, Cloudflare documents an experimental durable runtime that can pause for approval and replay previously applied connector results. It also documents important limits: stale running executions are not resumed through approval, replay must preserve call order and arguments, and rollback is compensation rather than database transaction isolation. [10]

Durable approval replay is therefore not the same as universal crash recovery. Nor is parallel execution always compatible with replay: Cloudflare warns that concurrent connector calls can arrive in different orders across passes. [10]

Numerical correctness is not a language stereotype

The talk questions JavaScript for mathematical workloads. The more precise concern is numeric representation and algorithm choice. JavaScript’s ordinary Number type uses IEEE 754 double-precision binary floating point, with finite precision and a bounded safe-integer range. [12]

That does not make JavaScript inherently unsuitable for mathematics, and changing languages does not automatically solve precision problems. Choose numeric types, libraries, and tests that match the task. Separately, deterministic replay requires controlling changing inputs, randomness, clocks, and external state. [10][12]

Binary artifacts need a separate return path

Code mode can manipulate images and other binary data, but returning an entire artifact as serialized text can recreate the context problem it was intended to solve. The talk identifies artifact handoff as an unresolved integration concern. [1]

A sensible architectural pattern is to store the artifact outside the conversation and return an authorized reference plus metadata. A signed URL is one possible mechanism, not a universal feature. Its access scope, expiry, storage lifetime, and consumer support must be designed explicitly. Cloudflare likewise recommends external storage and small references for oversized durable values. [10]

Model portability must be measured

The talk expresses an ambition to support different models while warning that smaller models may struggle with generated-code execution. That is a workload-specific concern, not proof that only frontier models can use code mode. [1]

Evaluate the model, instructions, discovery interface, and runtime together. A cheaper model that requires more repair attempts may erase the benefit; a well-scoped interface may make a smaller model effective. The benchmark, not the model’s category, should decide.

Evaluate outcomes and trajectories together

A successful demonstration does not establish reliability. Agents can take different paths through the same task, and infrastructure failures can be confused with reasoning failures.

Arize uses Harbor to run realistic tasks in fresh environments and Phoenix to inspect results and trajectories. Its published setup runs the real PXI endpoint against a Phoenix instance seeded with traces, rather than relying solely on mocked single-turn tool tests. The same task verifiers can evaluate PXI, MCP-connected coding agents, and CLI-based configurations. [9]

A useful evaluation loop is:

  1. Define the task, data, permissions, and expected outcome.
  2. Reset the environment for each trial.
  3. Run both the baseline and candidate repeatedly.
  4. Verify actual state changes or outputs, not merely the agent’s claim of success.
  5. Inspect trajectories for unnecessary discovery, excessive turns, or incorrect intermediate work.
  6. Change one relevant variable and rerun the same tasks.
  7. Keep separate tasks for checking whether the improvement generalizes. [9][13]

Rigorous Agent Evaluation: Harbor Controlled Trials Flywheel and Empirical Benchmarking Matrix

The talk describes network-disabled trials. That should not be generalized into “Harbor always disables networking.” Harbor supports public, no-network, and allowlist policies, with phase-specific configuration and provider-dependent support. [1][14]

Likewise, the talk’s binary reward is a choice of verifier. Harbor accepts numeric rewards and labeled metrics; its documentation describes a single reward as usually 1 or 0, not necessarily so. A reward-based evaluation setup also does not establish that model weights were trained with reinforcement learning. [15]

Pass@k measures opportunity, not consistency

The talk mentions pass@k. In the conventional code-evaluation definition, this estimates the probability that at least one of k sampled attempts succeeds. It is not the probability that every attempt succeeds, nor necessarily the success rate users experience on their first try. OpenAI’s HumanEval implementation provides the standard estimator. [16]

Report single-attempt success alongside any multi-attempt metric, and include retry behavior, infrastructure errors, latency, and resource usage. Repetition helps reveal variability; it does not eliminate it.

A passing answer can still hide a poor trajectory

Arize’s published benchmarking work shows why traces matter even when answers pass. One agent followed an error-analysis skill into a much longer investigation than the question required. Another published experiment reduced unnecessary skill loads through a tool-description change while preserving benchmark correctness. [9][13]

This is the practical meaning of evaluation-driven hill climbing: inspect where the agent wastes effort, propose a focused change, and test whether it improves behavior without breaking outcomes. Introducing a new capability can initially make performance worse if instructions encourage the wrong workflow.

Verifier quality matters just as much. In Phoenix’s SQL benchmark, a scoring model accepted an incorrect span count that manual review exposed. Passing the judge is not equivalent to being correct. [8]

What Arize’s reported results establish

In his AI Engineer talk, Mikyo King reports that the move to code mode reduced token usage by 43% and cost by 31%, while latency remained approximately unchanged despite sandbox overhead. The recording’s published description confirms the token and cost figures. These are Arize’s reported experimental results, not independently reproduced measurements or universal savings. [1]

The talk does not provide enough detail to establish the exact model mix, task weighting, variance, or whether “cost” includes all execution infrastructure. It therefore supports a directional conclusion: code mode improved efficiency in the evaluated configurations without an obvious latency penalty. It does not support a blanket prediction for other agents.

A separate Phoenix SQL benchmark compared eight telemetry questions, each run three times, using retrieval-only tools versus SQL with code mode. Arize reports average costs of about $3.97 and $0.23 per question respectively, approximately a seventeenfold difference. Those results must not be merged with the talk’s 31% figure: the workloads and architectural changes differ. [8]

The SQL comparison also changes both retrieval strategy and orchestration. It demonstrates the value of the combined design, not the isolated causal contribution of sandboxed code execution.

Where code mode earns its place

Code mode is a strong candidate when the task involves large intermediate payloads, repeated pagination, joins, conditional operations, or many mechanical steps between genuine decisions. Direct tools remain attractive for small, bounded tasks with clear arguments and useful output contracts. Cloudflare’s guidance makes the same distinction, and Arize explicitly cautions that some agents do not need the additional complexity. [2][5]

A practical first experiment is not “give the agent unrestricted code execution.” It is to select one representative workload, expose a narrow capability interface, constrain the runtime, and compare verified outcomes against a strong direct-tool baseline.

The central insight is about division of labor. A language model should not be the place where every record is counted, every page is traversed, and every byte is copied. Let software perform those operations. Bring the model back when interpretation, uncertainty, or a new decision requires it.

Code mode is valuable when it turns repeated reasoning into bounded execution without sacrificing the evidence needed to trust the result.

Sources

  1. AI Engineer, Stop Giving Your Agent Tools. Give It a Sandbox : Mikyo King, Arize AI, published October 27, 2026. Primary recording underlying the supplied material; performance figures are speaker-reported.
  2. Mikyo King, Arize AI, Code mode: Why your agent should code, September 2026.
  3. Anthropic, Code execution with MCP: Building more efficient agents, November 4, 2025.
  4. Anthropic, Introducing advanced tool use, November 24, 2025.
  5. Cloudflare, Code Mode overview. Documentation identifies the implementation as experimental.
  6. Cloudflare, Code Mode: give agents an entire API in 1,000 tokens, February 20, 2026.
  7. Anthony Powell, Arize AI, How we built UI Code Mode into Arize Phoenix, September 2026.
  8. Arize AI, Phoenix MCP SQL tools: query traces in code mode, August 2026.
  9. Elizabeth Hutton, Arize AI, How we benchmark AI agents and tools with Harbor and Arize Phoenix, October 2026.
  10. Cloudflare, How Code Mode works, updated October 12, 2026. Includes executor isolation, durable approval replay, rollback limitations, and result limits.
  11. Arize AI, What Is Code Mode?.
  12. MDN, JavaScript Number.
  13. Arize AI, How to test agent tool descriptions with Harbor and Phoenix, October 2026.
  14. Harbor, Network policies.
  15. Harbor, Verifier documentation.
  16. OpenAI, HumanEval evaluation implementation.

Discussion

Loading...