Context engineering is the practice of selecting, ordering, and maintaining the information an AI agent receives for each model call. It includes the system instructions, user request, conversation state, tool definitions, retrieved files, memory, examples, tool results, and summaries—not just the wording of one prompt.

Prompt engineering asks, "How should I express this instruction?" Context engineering asks, "What is the smallest trustworthy set of information this agent needs now, where should it come from, and what should be removed before the next step?"

That broader question matters because agents accumulate state. A coding agent may inspect 40 files, call 15 tools, encounter conflicting docs, and continue after a compacted history. More tokens do not automatically mean better decisions. Irrelevant, stale, duplicated, or malicious context can make an answer worse while increasing latency and cost.

Context engineering versus prompt engineering

Prompt engineering remains part of the job. Clear goals, constraints, examples, and output formats still matter. The difference is scope and timing.

Prompt engineering focuses on instruction design

Typical questions are:

  • Is the system instruction clear?
  • Does the request define success?
  • Would one canonical example resolve ambiguity?
  • Is the expected output shape explicit?

See prompt engineering for code for that layer.

Context engineering manages the complete inference state

It also asks:

  • Which tools should be advertised now?
  • Which repository files are relevant to this change?
  • Which memories are current, authorized, and task-specific?
  • Should old tool output be cleared or summarized?
  • What must survive compaction?
  • Which content is stable enough to cache?
  • What evidence shows that retrieval helped?

Prompt engineering can be a one-time edit. Context selection happens repeatedly as the agent observes results and chooses the next action.

The anatomy of agent context

A useful inventory is:

  1. System and policy instructions: identity, safety, permissions, global constraints.
  2. Task instructions: the user's current goal, acceptance criteria, and boundaries.
  3. Examples: a small set of representative input-output pairs.
  4. Tool definitions: names, descriptions, input schemas, and sometimes output schemas.
  5. Conversation history: user and assistant messages, plans, decisions, and corrections.
  6. Retrieved knowledge: files, database rows, search results, docs, and tickets.
  7. Tool results: command output, API responses, errors, screenshots, and diffs.
  8. Memory: durable user preferences, project facts, prior decisions, and working notes.
  9. Compaction summaries: compressed state from earlier turns.

Each item competes for a finite context window and for the model's attention. The engineering objective is not to fill the window. It is to maximize useful signal per token while preserving critical evidence.

Context windows are limits, not targets

A context window is the maximum sequence a model can process for a request, usually measured in tokens. Providers count and price tokens differently, and supported limits vary by model and API. Read the current model documentation rather than hardcoding one number into architecture.

Three constraints matter:

  • Hard capacity: inputs plus generated output must fit the API's rules.
  • Attention quality: relevant facts can become harder to use as surrounding noise grows.
  • Operational cost: larger inputs can increase billing, time to first token, network transfer, and cache sensitivity.

Keeping 30,000 irrelevant tokens because a model supports 200,000 is not safety. It is unreviewed state.

Reserve output capacity before sending input. An agent that consumes nearly the whole window with history may have too little room to explain a plan, emit valid JSON, or report a failure.

Selection is the core operation

For every candidate context item, ask:

  1. Does it change a likely decision?
  2. Is it authoritative for this task?
  3. Is it current?
  4. Is it safe and authorized to expose to this model or tool?
  5. Can it be fetched later if needed?
  6. Can a smaller representation preserve the needed evidence?

Use three outcomes:

  • Include now: required for the next decision.
  • Index or reference: keep an identifier and load on demand.
  • Exclude: irrelevant, stale, duplicated, unsafe, or unjustified.

This prevents the common "attach the whole repository" pattern. Start with a map, search results, interfaces, direct callers, and the failing test. Load implementations when the evidence points there.

A practical context budget

Do not begin with a universal percentage. Begin with task risk and model limits, then assign explicit envelopes. For a hypothetical 100,000-token usable input budget:

System and safety policy       6,000
Task and acceptance criteria   4,000
Core tool definitions          8,000
Current working set           35,000
Recent turns and decisions    15,000
Retrieved memory               7,000
Compaction summary            10,000
Contingency                   15,000

These are planning numbers, not provider recommendations. Measure actual token use. A read-only Q&A agent may need few tools and more documents; an operations agent may need strict policy and less raw history.

The contingency is important. Tools return unexpected errors, files are larger than their metadata suggests, and a correction may require new evidence.

Build a context budget workflow

Consider an agent asked to fix an authorization bug in a large TypeScript service.

Step 1: state the decision and success conditions

The immediate decision is not "understand the repository." It is:

> Identify why a user can access another tenant's invoice, patch the server-side authorization path without changing the API contract, and prove cross-tenant access is denied.

This statement selects relevant context: invoice routes, authorization middleware, tenant identity, data query filters, and tests. Styling and unrelated billing UI are out.

Step 2: load a map before full files

Collect:

  • directory and package map;
  • symbol or text search for invoice endpoints;
  • relevant type definitions;
  • current test names;
  • recent error and request trace.

File paths and short match snippets are cheap navigation context. Read full files only after the map narrows the working set.

Step 3: establish authority

Rank sources:

  1. current user instruction and enforced policy;
  2. current code and tests;
  3. schema or API contract;
  4. project documentation;
  5. retrieved tickets and old chat history;
  6. external advice.

An old ticket saying "admins can see all invoices" must not override current server code or an explicit tenant-isolation policy without verification.

Step 4: maintain an evidence ledger

Keep a compact note:

Goal: deny cross-tenant invoice reads.
Observed: GET route passes invoiceId only to repository.
Root-cause hypothesis: query omits tenantId.
Authority: auth middleware exposes session.tenantId.
Must preserve: response schema and same-tenant admin access.
Unresolved: background export caller behavior.
Tests needed: own tenant 200; other tenant 404/403; unauthenticated 401.

This note is more valuable after ten tool calls than replaying every search result.

Step 5: prune after each milestone

After root cause is confirmed, remove duplicate search output and failed hypotheses. Keep the decisive code, changed diff, test expectations, and unresolved caller.

Step 6: compact before pressure, not after failure

When history approaches the operating threshold, create a structured summary that preserves:

  • user goal and explicit constraints;
  • decisions and their evidence;
  • files changed and why;
  • tests run and exact failures;
  • unresolved risks and next action;
  • permissions and actions that remain prohibited.

Do not reduce it to "working on auth bug." That loses the contract.

Tool definitions are context

Every advertised tool name, description, parameter, enum, and schema consumes context or affects selection. A large overlapping tool catalog creates two costs: tokens and decision ambiguity.

Good tool design uses:

  • a distinct verb and domain;
  • a description that says when to use the tool;
  • narrow, typed parameters;
  • safe defaults;
  • bounded output;
  • structured errors;
  • a clear read versus write distinction.

Weak:

{
  "name": "manage_data",
  "description": "Manages data",
  "parameters": {
    "query": { "type": "string" }
  }
}

Better:

{
  "name": "get_invoice_by_id",
  "description": "Read one invoice visible to the authenticated tenant. Use when an invoice ID is known. This tool never updates data.",
  "inputSchema": {
    "type": "object",
    "properties": {
      "invoiceId": {
        "type": "string",
        "description": "Canonical invoice ID, for example inv_123."
      }
    },
    "required": ["invoiceId"],
    "additionalProperties": false
  }
}

Authorization still belongs on the server. A description saying "visible to the tenant" cannot enforce tenant isolation.

Load tools on demand when supported

If a client or provider supports tool search or deferred loading, keep common core tools visible and discover specialized ones when needed. Do not assume this behavior is portable across APIs. Anthropic, for example, documents deferred tool loading for its Tool Search feature; another provider may expose a different mechanism or none.

For protocol-level tools, the MCP specification requires server input validation, access controls, rate limiting, and sanitized output. It advises clients to validate results, use timeouts, log usage, and keep a human able to deny sensitive operations.

Retrieval: fetch evidence just in time

Retrieval can be lexical search, embeddings, graph lookup, SQL, file browsing, or API calls. The right method is the simplest one that reliably finds the needed evidence.

Use up-front retrieval for small, stable, high-probability context: a project policy, the target function, and its tests. Use just-in-time retrieval for large or conditional detail: an error catalog, one customer record, or an API page needed only after a specific failure.

Retrieval needs metadata

Store and return:

  • source identifier or URL;
  • title or path;
  • updated time;
  • authority or owner;
  • access classification;
  • concise matching excerpt;
  • retrieval reason or score.

A chunk without provenance is hard to trust, refresh, or cite.

Avoid top-k cargo culting

"Always retrieve five chunks" has no connection to task difficulty. Some questions need one exact schema. Others need competing policy versions and their dates. Evaluate precision, recall, answer quality, and security exposure together.

Never use retrieval as authorization

Filtering a vector search by tenant is useful, but the underlying data access layer must enforce tenant authorization. Embedding similarity is not an access-control system.

Memory: durable state with retrieval rules

Memory is context stored outside the current model call and retrieved later. Separate at least three classes:

  • working memory: current task notes, hypotheses, and next steps;
  • project memory: architecture decisions, commands, conventions, and known constraints;
  • user memory: stable preferences the user has knowingly allowed the system to retain.

Do not turn every conversation detail into permanent memory. Store only facts with a clear future use, provenance, scope, and deletion path.

A memory record can include:

{
  "fact": "Production deployment requires explicit user approval.",
  "scope": "project:billing-service",
  "source": "policy/operations.md",
  "verifiedAt": "2026-09-16",
  "expiresAt": null,
  "sensitivity": "internal",
  "confidence": "authoritative"
}

On retrieval, check scope and freshness. If memory conflicts with the current user or current source of truth, surface the conflict instead of silently choosing old state.

Sensitive memory creates privacy, compliance, and breach risk. Minimize retention, encrypt where appropriate, restrict access, support correction and deletion, and avoid storing credentials.

Compaction without losing the task

Compaction summarizes older context and continues with a smaller active state. It is useful for long-running tasks, but it is lossy.

Preserve:

  • goals, constraints, and acceptance criteria;
  • explicit user corrections;
  • architecture and security decisions;
  • identifiers, file paths, versions, and exact error signatures;
  • completed actions and validation evidence;
  • open questions and next action.

Discard or externalize:

  • repeated acknowledgements;
  • raw tool output already distilled into verified facts;
  • dead-end hypotheses;
  • duplicate file content;
  • obsolete plans;
  • verbose reasoning that produced no decision.

Use structured summaries

## Goal
...

## Constraints and permissions
...

## Confirmed evidence
...

## Decisions
...

## Changed artifacts
...

## Validation
...

## Unresolved and next
...

Then evaluate the summary by resuming the task from it. Can the agent name the prohibited action, changed files, failing test, and next check? If not, the summary compressed the wrong things.

Provider-managed compaction is implementation-specific. Anthropic documents a server-side compaction API and its own trigger behavior. Use such features when they fit, but keep your application's state model and tests independent of one vendor's exact block types.

Context pollution and instruction trust

Context pollution is any accumulated information that reduces the agent's ability to choose correctly. It includes:

  • stale plans that conflict with the current goal;
  • duplicated tool output;
  • broad tool catalogs;
  • unrelated files;
  • retrieved content with hidden instructions;
  • error logs containing secrets;
  • generated summaries treated as primary evidence;
  • memory from the wrong project or user.

Prompt injection is a high-risk form of pollution. A README, issue, web page, tool result, or database row may tell the agent to ignore policy, reveal secrets, or execute commands.

Defenses include:

  • label content by source and trust level;
  • separate instructions from untrusted data;
  • enforce permissions outside the model;
  • use least-privilege tools and server-side authorization;
  • require confirmation for sensitive writes or disclosure;
  • sanitize and bound tool output;
  • never place secrets in context unless the immediate authorized operation requires them;
  • preserve the user's latest correction over stale memory.

No prompt can turn an unconstrained shell or overprivileged API into a safe security boundary.

Cost, latency, and cache design

Input tokens, output tokens, retrieval, tool calls, and provider-specific cache operations can all affect cost. Measure them per task and per successful outcome, not only per request.

Useful telemetry:

  • input and output tokens by context category;
  • retrieved chunks selected and used;
  • tool definitions advertised versus called;
  • cache read, write, and miss counts where exposed;
  • compaction count and summary size;
  • latency by retrieval, model, and tool;
  • task success and human correction rate;
  • sensitive-data policy violations.

Design stable prefixes for caching

Prompt caches usually reward exact or prefix reuse, but mechanics and prices differ by provider. Place stable content—tool definitions, core system policy, canonical examples—before volatile task data when the API's caching model benefits from that order.

Do not freeze stale policy merely to improve hit rate. Cache correctness comes first. Version the stable prefix, invalidate it when tools or policy change, and compare total cost with and without caching under real traffic.

Anthropic documents a tools → system → messages cache-prefix order for its API. That is a vendor-specific implementation detail, not a universal context-engineering law.

Evaluate context engineering changes

An agent can sound better while becoming less reliable. Build an evaluation set from real tasks and failures.

Run ablations

Compare:

  • baseline prompt only;
  • prompt plus selected files;
  • prompt plus broad retrieval;
  • narrow versus full tool catalog;
  • raw history versus compacted history;
  • memory enabled versus disabled;
  • cached and uncached paths for output equivalence.

If adding context does not improve task success, remove it.

Score more than the final answer

Measure:

  • task correctness;
  • evidence and citation accuracy;
  • tool selection;
  • number of unnecessary tool calls;
  • authorization and privacy compliance;
  • recovery from tool errors;
  • context tokens and total cost;
  • latency;
  • behavior after compaction;
  • resistance to injected instructions.

Include adversarial and stale cases

Test a retrieved document with an instruction to exfiltrate data, an old memory that contradicts current policy, two similarly named tools, a huge irrelevant file, and a compaction summary missing one constraint. These cases reveal context architecture failures that a happy-path benchmark misses.

Use deterministic checks where possible and human review for nuanced quality. Keep provider and model versions in results so changes are explainable.

Mistakes that waste context

Attaching everything up front

This hides important evidence and increases cost. Start with a map and fetch detail on demand.

Treating a larger window as memory

The window is temporary request state. Durable memory needs storage, provenance, retrieval, correction, and deletion.

Keeping all tool results forever

Distill results into evidence and retain a pointer to raw output if it may be needed later.

Writing overlapping tool descriptions

If two tools appear interchangeable, selection will be unstable. Narrow their contracts or merge them.

Compacting without an evaluation

A short summary can silently drop the only security constraint. Resume from summaries in tests and check critical fact recall.

Optimizing token count alone

The cheapest failed run is still waste. Optimize successful-task cost, including retries and human correction.

Trusting retrieved text as instruction

Treat retrieved content as evidence unless its source is an authorized instruction source. Enforce this with tool and policy boundaries, not a reminder alone.

When context engineering is not the first fix

Do not build a vector database, memory service, or compaction layer for a single-turn classification call that already fits in a small prompt. Fix a vague task definition before tuning retrieval. Fix server authorization before changing a tool description. Replace a nondeterministic model step with ordinary code when the rule is exact.

The simplest working design is often:

  1. clear task contract;
  2. small core prompt;
  3. a few distinct tools;
  4. targeted file or record retrieval;
  5. a short working note;
  6. regression tests.

Add memory and compaction when task duration proves they are needed.

The AI hub contains the surrounding agent and API topics. For client-specific coding-agent workflows, see the Cursor AI guide. For API tradeoffs that affect window limits, tool calling, and cache behavior, compare OpenAI and Anthropic APIs.

FAQ

Is context engineering just RAG?

No. Retrieval is one input. Context engineering also covers instructions, tool definitions, history, memory, compaction, ordering, caching, trust, and evaluation.

Does more context improve accuracy?

Sometimes, when the added information is relevant and trustworthy. Irrelevant or conflicting context can reduce accuracy. Test additions against a baseline.

What should be compacted first?

Repeated or already-distilled tool results are often safer early candidates than goals, constraints, user corrections, or unresolved evidence.

How much context should tools consume?

As little as needed to make selection and input construction reliable. There is no universal token quota. Track which advertised tools are called and defer specialized definitions where the client supports it.

Should memory be automatic?

Only with explicit scope, retention, correction, privacy, and deletion rules. Automatic storage of every interaction creates noise and risk.

Is prompt caching portable?

The idea is broad; APIs differ in prefix order, minimum size, lifetime, pricing, and controls. Implement against current provider docs and retain an uncached correctness path.

How do I know retrieval helped?

Run an ablation with retrieval disabled, then compare task success, evidence quality, tool calls, tokens, cost, and latency on the same cases.

Can compaction extend a task forever?

No. Repeated summaries can accumulate errors and lose detail. Keep durable artifacts and checkpoints outside the model context, verify summaries, and restart from authoritative state when needed.

Context review checklist

  • [ ] Current goal and success conditions are explicit.
  • [ ] The source hierarchy is known.
  • [ ] Input and output capacity are budgeted.
  • [ ] Only relevant tools are advertised.
  • [ ] Retrieved data has provenance, scope, and freshness.
  • [ ] Memory is scoped, authorized, correctable, and removable.
  • [ ] Raw tool output is bounded and pruned after use.
  • [ ] Compaction preserves constraints, decisions, evidence, and next steps.
  • [ ] Stable cached prefixes are versioned and invalidated safely.
  • [ ] Sensitive data and injected instructions are contained.
  • [ ] Evaluations include ablations, stale data, failures, and adversarial cases.
  • [ ] Cost is measured per successful task.

Continue learning

Sources

The sources describe principles plus specific Anthropic and MCP implementations. Token limits, cache behavior, compaction APIs, tool-loading features, and prices change by model and provider; verify current documentation before setting production budgets.

ai-agentscontext-engineeringprompt-engineeringagent-memoryllm-evaluation