What Is Context Engineering? The 2026 Successor to Prompt Engineering

Quick Answer

Context engineering is the practice of designing everything an LLM sees at inference time, not just the prompt: system instructions, retrieved documents, tool definitions, memory, few-shot examples, conversation history, and output format. Popularized in 2025 by Andrej Karpathy, Tobi Lutke, and Anthropic, it reframes the job from wording one prompt to engineering the whole context window. Prompt engineering is now a subset.

TL;DR

Context engineering is the 2026 successor to prompt engineering. Where prompt engineering asked how do I word this instruction, context engineering asks what configuration of the entire context window is most likely to produce the behavior I want. That window includes system instructions, retrieved knowledge (RAG), tool definitions, memory, examples, and history, all fighting for a finite token budget. The discipline matters most for agents that run over many turns, and its core moves are retrieval, compaction, memory, isolation, and just-in-time loading.

Why The Term Changed

For a couple of years, prompt engineering was the headline skill of the LLM era. You learned to phrase requests, add few-shot examples, and coax better answers out of a single text box. That framing worked when the main interface was a chat window and the main task was a one-off completion.

The framing broke down as people started building industrial-strength LLM applications and, especially, agents. In an agent, the model is not answering once. It is looping: reading a task, calling a tool, reading the result, updating a plan, calling another tool, and so on across dozens or hundreds of steps. At each step, what the model actually sees is not just your prompt. It is a whole assembled window of tokens.

In mid-2025, the vocabulary caught up. Shopify CEO Tobi Lutke and former OpenAI researcher Andrej Karpathy both publicly endorsed context engineering over prompt engineering. As Karpathy put it, people associate prompts with the short task descriptions you type day to day, whereas context engineering is the delicate art and science of filling the context window with just the right information for the next step. He is fond of an operating-system analogy: the LLM is the CPU, the context window is the RAM, and context engineering is the OS deciding what to load into memory. Anthropic's engineering team formalized the idea in a September 2025 post, Effective Context Engineering for AI Agents, defining it as the set of strategies for curating and maintaining the optimal set of tokens during inference.

This builds directly on ideas we cover in our prompt engineering guide. Prompt engineering did not disappear; it became one layer of a larger stack.

What Actually Lives In The Context Window

The context window is every token the model reads before it generates its next token. In a modern agent, that assembly typically includes:

• System instructions — the role, rules, and constraints (the procedural memory of the agent).
• Tool definitions — the schemas of the functions or MCP servers the model can call.
• Retrieved documents — chunks pulled in via RAG from a vector store or search index (semantic memory).
• Memory — durable facts about the user, project, or prior sessions (episodic memory).
• Few-shot examples — demonstrations of the desired format or reasoning.
• Conversation and action history — prior turns, tool calls, and their outputs (short-term memory).
• Output format — the schema or template the answer must fit.

Prompt engineering mostly touches the first and fifth of those. Context engineering owns all seven, plus the harder question of how they are ordered, budgeted, and refreshed.

The Core Principle: Finite Tokens, Diminishing Returns

The reason this is engineering and not just writing is that context is a scarce resource. Windows are large in 2026, but they are not free and they are not uniformly effective. Two facts drive the whole discipline.

First, tokens cost money and latency. A bloated window is slow and expensive on every call.

Second, and more subtly, more context is not automatically better. Anthropic's team describe context rot: as the number of tokens in the window grows, the model's ability to accurately recall and use any given fact tends to decline. This shows up in needle-in-a-haystack style testing, where a relevant detail buried in a long context gets missed even though it is technically present. So even before you hit a hard token limit, you can be getting less value out of each token.

The conclusion follows directly: the goal is not to stuff the window, it is to find the smallest set of high-signal tokens that reliably produces the outcome you want. Everything below is a technique for doing that.

Core Techniques

Retrieval (just-in-time RAG)

Rather than pasting a whole knowledge base into the prompt, you index it and retrieve only the chunks relevant to the current step. This is the classic RAG pattern, and it is the workhorse of context engineering. The refinement in 2026 is just-in-time retrieval: the agent fetches context at the moment it is needed, often by calling a search tool itself, rather than front-loading everything. If your retrieval returns junk, see our guide to debugging irrelevant RAG results.

Compaction and summarization

When a conversation or action log grows long, you compact it. That can mean summarizing older turns into a short recap, dropping stale tool outputs, or clearing tools that are no longer relevant. The Claude developer docs describe context-management moves like tool result clearing and compaction explicitly. Done well, compaction keeps the useful signal (the plan, the key findings) while shedding the noise (verbose intermediate outputs).

Structured note-taking and memory

Instead of keeping everything in the window, the agent writes durable notes to an external store: a scratchpad file, a database, or a dedicated memory system. On the next step it reads back only what it needs. This is how long-horizon agents survive tasks that far exceed a single window. We compare the leading tools in AI memory systems compared: Mem0, Letta, Zep, LangMem.

Sub-agent isolation

Rather than one agent with a giant, messy context, you split work across specialists, each with its own clean window. A research sub-agent gathers sources and returns a short synthesis; the main agent never sees the raw clutter. Isolation prevents one task's context from poisoning another's.

Just-in-time context loading

The unifying idea across the above: load context when it is needed, at the granularity it is needed, and evict it when it is not. Tool definitions, retrieved docs, and memory should enter the window on demand rather than sitting there permanently. This keeps the effective context small even when the total available knowledge is huge.

Common Failure Modes

Context engineering also names the ways context goes wrong. Four failure modes recur across the agent-building community:

• Context poisoning — a hallucination or error enters the context (say, a wrong fact in an early tool result) and then gets referenced repeatedly, compounding the mistake.
• Distraction — the window fills with so much marginally relevant material that the model loses focus on the actual task; a form of context rot.
• Context clash — two pieces of context contradict each other (an old instruction versus a new one), and the model cannot reliably resolve which to follow.
• Context overflow — the assembly simply exceeds the window (or the effective useful portion of it), and important content gets truncated or ignored.

Many real agent bugs reduce to one of these. We dig into the diagnosis in why your AI agent loses context and how to fix it.

Context Engineering vs Prompt Engineering

The table makes the relationship clear: prompt engineering is not obsolete, it is nested inside context engineering. You still need a crisp system prompt. You just also need to manage the six other things sharing the window with it.

Why It Matters More In The Agent Era

The reason context engineering became the headline skill of 2026 is agentic AI. A chatbot answering one question rarely stresses the context window. An agent that plans a multi-day task, calls twenty tools, reads a dozen documents, and remembers what it learned yesterday stresses it constantly.

Three trends amplified this. Long-horizon agents run for many turns, so context accumulates and must be actively managed. Tool use and MCP mean the window is now full of machine-generated content (schemas, JSON results, error traces), not just human text. And prompt caching changed the economics: because a stable, well-ordered context prefix can be cached, structuring your context well now saves real money, as we explain in prompt caching and when it saves money. Good context engineering is cheaper to run, not just more accurate.

How To Start Practicing It

You do not need a research lab to apply context engineering. A practical starting checklist:

• Audit your window. Log the full context sent on a representative call and read it. You will usually find dead weight.
• Order for caching. Put stable content (system instructions, tool definitions) first, dynamic content last, so the stable prefix can be cached.
• Retrieve, do not paste. Move large knowledge into a retrieval layer and pull only what each step needs.
• Compact aggressively. Summarize old turns and clear stale tool outputs before they rot.
• Externalize memory. Write durable facts to a store and read them back on demand rather than carrying everything inline.
• Isolate specialists. Give sub-agents their own clean contexts for focused subtasks.

The frameworks help here: both LangChain and LlamaIndex ship context and memory abstractions, and Anthropic's cookbook documents concrete compaction and tool-clearing patterns.

Further Reading And Primary Sources

To go deeper, start with the primary material: Anthropic's Effective Context Engineering for AI Agents and its companion on effective harnesses for long-running agents; the Anthropic engineering blog index; Karpathy's original tweet endorsing the term; the LangChain documentation; and the LlamaIndex documentation. The community-maintained Prompt Engineering Guide also now carries a dedicated context engineering section.

Conclusion

Context engineering is not a rebrand of prompt engineering; it is the larger discipline that prompt engineering now lives inside. As soon as your LLM stops answering single questions and starts acting as an agent, the binding constraint becomes what configuration of the finite context window produces reliable behavior. Master retrieval, compaction, memory, isolation, and just-in-time loading, watch for poisoning, distraction, clash, and overflow, and treat the context window as the scarce, high-value resource it is. In 2026, that is the skill that separates demos from production agents.

This is an editorial synthesis of primary vendor and standards documentation and community reports; see our [methodology](/methodology). Verify current details with each vendor.