An AI agent has no persistent memory between calls. Every inference starts from a blank slate: task instructions, conversation history, retrieved documents, tool definitions: all of it must fit inside a fixed token budget. Context engineering is the practice of deciding how to use that budget.
What context engineering covers
The context window is the maximum number of tokens a model can process in a single call. The window sets the budget; context engineering is the discipline of spending it.
Those spending decisions fall into four categories:
Retrieval. Documents, records, or memory fragments stored outside the session must be selected and ranked before each call. How many to include, and where to position them within the window, are context engineering decisions.
History compression. As a conversation or task accumulates, prior turns eventually exceed what the window can hold. Earlier content must be summarised or dropped to make room. Deciding what to keep verbatim and what to discard without losing the thread of the task is among the harder problems in the discipline.
Tool representation. Each tool schema a model can invoke consumes tokens. Exposing the full set of available tools on every call is often wasteful; deciding which tools are relevant to the current step, and how verbosely to describe them, directly affects how much of the budget remains for task-relevant content.
Memory injection. Facts, preferences, and prior conclusions stored across sessions must be retrieved and inserted at each call; this is what agent memory systems hold. How many fragments to surface, how to format them, and where to place them shapes how reliably the model applies them.
Mei et al., synthesising over 1,400 research papers, define context engineering as “the systematic optimization of information payloads for LLMs,” decomposing it into retrieval and generation, context processing, and context management, with downstream applications in RAG, memory systems, tool-integrated reasoning, and multi-agent coordination (“A Survey of Context Engineering for Large Language Models,” arXiv:2507.13334, July 2025).
How it differs from prompt engineering
Prompt engineering addresses phrasing: how to word an instruction so the model interprets it correctly. Context engineering addresses the full payload: what information is present alongside that instruction.
A well-phrased prompt inside a poorly assembled context still fails. If the relevant document was not retrieved, or if system instructions were trimmed to make room for excess history, the model cannot compensate through better instruction-following. The prompt is one piece of the context; context engineering governs all the pieces together.
The two are related but separable, a distinction laid out in context engineering vs prompt engineering. The same prompt can perform very differently depending on what else is in the window.
Why agents push context engineering to the centre
In a single-turn query, context decisions are contained: attach relevant documents, write clear instructions, run the call. In an agent operating across dozens of model calls, those decisions compound.
Each call must carry forward what the agent needs from prior steps without carrying forward everything. Re-injecting the full prior conversation on each turn quickly exhausts the budget. Prior reasoning must be compressed; new retrieved content ranked against what is already present. Tool schemas must be pruned to the subset relevant for this step.
A poor context decision early in a run degrades every step that follows. The agent is not acting on a complete picture of its own prior work; it is acting on whatever survived the compression.
Placement matters, not just inclusion
Here is what most introductions to context engineering leave out: what enters the window is not the only decision. Where it appears within the window also matters.
Research has found that models attend more reliably to content positioned near the beginning or end of the input than to content buried in the middle, even in models built explicitly for long contexts (Liu et al., “Lost in the Middle: How Language Models Use Long Contexts,” arXiv:2307.03172, 2023). The effect held across multiple model families and multiple context lengths.
This makes context engineering a sequencing problem as well as a selection problem. Placing high-priority instructions and the most relevant retrieved content at the edges of the context, not buried mid-window, is a decision with measurable effects on output quality. Including the right content is necessary; positioning it well is the part of the discipline that gets skipped.
Questions, answered
How is context engineering different from prompt engineering?
Prompt engineering focuses on the phrasing and structure of instructions. Context engineering governs the full information payload: what to retrieve, how to compress prior history, which tool schemas to expose, and what memory fragments to inject. The prompt is one component of that payload.
What decisions does context engineering involve?
Which documents or memory fragments to retrieve and rank, how to summarise conversation history when it approaches the token budget, which tool schemas to include on a given call, and where within the window to position high-priority content.
Is context engineering only relevant for long conversations?
No. Even a single-turn query involves context decisions: which background documents to attach, whether to include retrieved examples, which instructions to surface in the system prompt. These are context engineering decisions even without prior conversation to manage.
Why does context engineering matter more in agentic systems?
An agent runs across many model calls. Each call must decide what to carry forward from prior steps, what to compress, and what new information to inject. A poor context decision early in a run degrades every step that follows.
Brief is a team of AI associates you direct in plain language. Opening to a small group at a time.
Request access