What Is Context Engineering? The Successor to Prompt Engineering
Context engineering is the practice of curating everything an LLM sees at inference: instructions, retrieved documents, tool definitions, memory, examples, history, and output format. Popularized in 2025 by Andrej Karpathy, Tobi Lutke, and Anthropic's engineering team, it matters most for agents that accumulate context over many turns. Prompt engineering is now one subset of it.
Why The Term Changed
For a couple of years, prompt engineering was the headline skill of the LLM era. You learned to phrase requests, add few-shot examples, and coax better answers out of a single text box. That framing worked when the main interface was a chat window and the main task was a one-off completion. For the basics that still apply, see our explainer on prompt engineering.
The framing broke down as people started building industrial-strength LLM applications and, especially, agents. In an agent, the model is not answering once. It is looping: reading a task, calling a tool, reading the result, updating a plan, calling another tool, and so on across dozens or hundreds of steps. At each step, what the model actually sees is not just your prompt.
It is a whole assembled window of tokens.
In mid-2025, the vocabulary caught up. Shopify CEO Tobi Lutke and former OpenAI researcher Andrej Karpathy both publicly endorsed context engineering over prompt engineering. As Karpathy put it, people associate prompts with the short task descriptions you type day to day, whereas context engineering is the delicate art and science of filling the context window with just the right information for the next step.
He is fond of an operating-system analogy: the LLM is the CPU, the context window is the RAM, and context engineering is the OS deciding what to load into memory. Anthropic's engineering team formalized the idea in a September 2025 post, Effective Context Engineering for AI Agents, defining it as the set of strategies for curating and maintaining the optimal set of tokens during inference.
This builds directly on ideas we cover in our prompt engineering guide. Prompt engineering did not disappear; it became one layer of a larger stack.
What Actually Lives In The Context Window
The context window is every token the model reads before it generates its next token. In a modern agent, that assembly typically includes:
- System instructions — the role, rules, and constraints (the procedural memory of the agent).
- Tool definitions — the schemas of the functions or MCP servers the model can call.
- Retrieved documents — chunks pulled in via RAG from a vector store or search index (semantic memory).
- Memory — durable facts about the user, project, or prior sessions (episodic memory).
- Few-shot examples — demonstrations of the desired format or reasoning.
- Conversation and action history — prior turns, tool calls, and their outputs (short-term memory).
- Output format — the schema or template the answer must fit.
Prompt engineering mostly touches the first and fifth of those. Context engineering owns all seven, plus the harder question of how they are ordered, budgeted, and refreshed.
The Core Principle: Finite Tokens, Diminishing Returns
The reason this is engineering and not just writing is that context is a scarce resource. Windows are large in 2026, but they are not free and they are not uniformly effective. Two facts drive the whole discipline.
First, tokens cost money and latency. A bloated window is slow and expensive on every call.
Second, and more subtly, more context is not automatically better. Anthropic's team describe context rot: as the number of tokens in the window grows, the model's ability to accurately recall and use any given fact tends to decline. This shows up in needle-in-a-haystack style testing, where a relevant detail buried in a long context gets missed even though it is technically present.
So even before you hit a hard token limit, you can be getting less value out of each token.
The conclusion follows directly: the goal is not to stuff the window, it is to find the smallest set of high-signal tokens that reliably produces the outcome you want. Everything below is a technique for doing that.
Core Techniques
Retrieval (just-in-time RAG)
Rather than pasting a whole knowledge base into the prompt, you index it and retrieve only the chunks relevant to the current step. This is the classic RAG pattern, and it is the workhorse of context engineering. The refinement in 2026 is just-in-time retrieval: the agent fetches context at the moment it is needed, often by calling a search tool itself, rather than front-loading everything. If your retrieval returns junk, see our guide to debugging irrelevant RAG results.
Compaction and summarization
When a conversation or action log grows long, you compact it. That can mean summarizing older turns into a short recap, dropping stale tool outputs, or clearing tools that are no longer relevant. The Claude developer docs describe context-management moves like tool result clearing and compaction explicitly. Done well, compaction keeps the useful signal (the plan, the key findings) while shedding the noise (verbose intermediate outputs).
Structured note-taking and memory
Instead of keeping everything in the window, the agent writes durable notes to an external store: a scratchpad file, a database, or a dedicated memory system. On the next step it reads back only what it needs. This is how long-horizon agents survive tasks that far exceed a single window. We compare the leading tools in AI memory systems compared: Mem0, Letta, Zep, LangMem.
Sub-agent isolation
Rather than one agent with a giant, messy context, you split work across specialists, each with its own clean window. A research sub-agent gathers sources and returns a short synthesis; the main agent never sees the raw clutter. Isolation prevents one task's context from poisoning another's.
Just-in-time context loading
The unifying idea across the above: load context when it is needed, at the granularity it is needed, and evict it when it is not. Tool definitions, retrieved docs, and memory should enter the window on demand rather than sitting there permanently. This keeps the effective context small even when the total available knowledge is huge.
Common Failure Modes
Context engineering also names the ways context goes wrong. Four failure modes recur across the agent-building community:
- Context poisoning — a hallucination or error enters the context (say, a wrong fact in an early tool result) and then gets referenced repeatedly, compounding the mistake.
- Distraction — the window fills with so much marginally relevant material that the model loses focus on the actual task; a form of context rot.
- Context clash — two pieces of context contradict each other (an old instruction versus a new one), and the model cannot reliably resolve which to follow.
- Context overflow — the assembly simply exceeds the window (or the effective useful portion of it), and important content gets truncated or ignored.
Many real agent bugs reduce to one of these. We dig into the diagnosis in why your AI agent loses context and how to fix it.
How Is Context Engineering Different From Prompt Engineering?
| Dimension | Prompt Engineering | Context Engineering |
|---|---|---|
| --- | --- | --- |
| Scope | The wording of one instruction | Everything the model sees at inference |
| Primary artifact | A prompt string | The assembled context window |
| Era | Chat and single completions | Agents, tools, long-horizon tasks |
| Key levers | Phrasing, few-shot examples, format | Retrieval, memory, compaction, ordering, budget |
| Main risk | Ambiguous or weak instruction | Context rot, poisoning, overflow, clash |
| Relationship | A subset of context engineering | The superset discipline |
| Analogy | Writing a good question | Managing the RAM the model reasons over |
The table makes the relationship clear: prompt engineering is not obsolete, it is nested inside context engineering. You still need a crisp system prompt. You just also need to manage the six other things sharing the window with it.
Why It Matters More In The Agent Era
The reason context engineering became the headline skill of 2026 is agentic AI. A chatbot answering one question rarely stresses the context window. An agent that plans a multi-day task, calls twenty tools, reads a dozen documents, and remembers what it learned yesterday stresses it constantly.
Three trends amplified this. Long-horizon agents run for many turns, so context accumulates and must be actively managed. Tool use and MCP mean the window is now full of machine-generated content (schemas, JSON results, error traces), not just human text. And prompt caching changed the economics: because a stable, well-ordered context prefix can be cached, structuring your context well now saves real money, as we explain in prompt caching and when it saves money.
Good context engineering is cheaper to run, not just more accurate.
How To Start Practicing It
You do not need a research lab to apply context engineering. A practical starting checklist:
- Audit your window. Log the full context sent on a representative call and read it. You will usually find dead weight.
- Order for caching. Put stable content (system instructions, tool definitions) first, dynamic content last, so the stable prefix can be cached.
- Retrieve, do not paste. Move large knowledge into a retrieval layer and pull only what each step needs.
- Compact aggressively. Summarize old turns and clear stale tool outputs before they rot.
- Externalize memory. Write durable facts to a store and read them back on demand rather than carrying everything inline.
- Isolate specialists. Give sub-agents their own clean contexts for focused subtasks.
The frameworks help here: both LangChain and LlamaIndex ship context and memory abstractions, and Anthropic's cookbook documents concrete compaction and tool-clearing patterns.
Further Reading And Primary Sources
To go deeper, start with the primary material: Anthropic's Effective Context Engineering for AI Agents and its companion on effective harnesses for long-running agents; the Anthropic engineering blog index; Karpathy's original tweet endorsing the term; the LangChain documentation; and the LlamaIndex documentation. The community-maintained Prompt Engineering Guide also now carries a dedicated context engineering section.
Conclusion
Context engineering is not a rebrand of prompt engineering; it is the larger discipline that prompt engineering now lives inside. As soon as your LLM stops answering single questions and starts acting as an agent, the binding constraint becomes what configuration of the finite context window produces reliable behavior. Master retrieval, compaction, memory, isolation, and just-in-time loading, watch for poisoning, distraction, clash, and overflow, and treat the context window as the scarce, high-value resource it is.
In 2026, that is the skill that separates demos from production agents.
This is an editorial synthesis of primary vendor and standards documentation and community reports; see our [methodology](/methodology). Verify current details with each vendor.
Sources
The primary sources this article cites, checked in November 2026:
- Effective Context Engineering for AI Agents (anthropic.com)
- effective harnesses for long-running agents (anthropic.com)
- Anthropic engineering blog (anthropic.com)
- tweet endorsing the term (x.com)
- LangChain documentation (python.langchain.com)
- Prompt Engineering Guide (promptingguide.ai)
- Memory (LlamaIndex documentation) (developers.llamaindex.ai)
Key Takeaways
- Context engineering designs the full context window an LLM sees at inference, not just the wording of one prompt.
- Andrej Karpathy, Shopify CEO Tobi Lutke, and Anthropic's engineering team popularized the term in mid-to-late 2025.
- It matters most for agents that run many turns with tools, RAG, and MCP, where context accumulates and degrades.
- Core techniques are retrieval, compaction, structured memory, sub-agent isolation, and just-in-time context loading.
- Common failure modes are context poisoning, distraction, clash, and overflow, plus context rot as windows grow longer.
- The goal is the smallest set of high-signal tokens that reliably produces the desired behavior.
Frequently Asked Questions
What is context engineering in simple terms?
It is the practice of deciding what an AI model reads before it answers. Instead of tweaking one prompt, you assemble the whole context window: instructions, retrieved facts, tool definitions, memory, examples, and history. The aim is to give the model exactly the information it needs, and nothing that distracts it.
How is context engineering different from prompt engineering?
Prompt engineering covers the wording of the instruction you type. Context engineering covers everything the model sees at inference, of which the prompt is one part, plus how it is ordered, budgeted, and refreshed. In 2026 prompt engineering is treated as a subset: you still need a crisp system prompt, but you manage everything else too.
Who coined the term context engineering?
No single person owns it. It settled into common use in mid-2025 when Shopify CEO Tobi Lutke and Andrej Karpathy publicly endorsed it over prompt engineering. Anthropic's engineering team formalized the discipline in a September 2025 post, Effective Context Engineering for AI Agents, and agent builders adopted it broadly into 2026.
Why does context engineering matter more for AI agents?
Single-shot chat prompts are short-lived. Agents run over many turns, call tools, read documents, and accumulate history, so their window fills with machine-generated content and can degrade. Deciding what to keep, summarize, or drop separates an agent that stays on task from one that loses the plot after a few dozen steps.
What is context rot?
Context rot is the decline in a model's ability to accurately recall and use information as the number of tokens in its window grows. It shows up in needle-in-a-haystack testing, where a relevant detail buried in a long context gets missed. More context is not automatically better, even before the hard token limit.
About the Author
Aisha Patel
AI Editorial Desk
AI Editorial Desk · Web3AIBlog
Aisha Patel is a pen name for our AI editorial desk. Posts under this byline are written and reviewed by our team of contributors with backgrounds in machine learning, large language models, AI infrastructure, and applied research. The desk covers frontier model releases, agent architectures, retrieval-augmented generation, on-device inference, and the engineering tradeoffs that matter when shipping AI in production. Every technical claim is verified against primary sources before publication.