Skip to content

Context Engineering

Last reviewed: October 2026

What's changed recently

  • "Context engineering" became the standard name in 2025 for assembling, budgeting, and pruning what an agent sees across a run, not just wording one prompt.
  • Long context is not free accuracy. 2025 studies of long inputs ("context rot") found quality drops as input grows, well before the window is full, so relevance and compaction still beat dumping everything in.
  • Compaction and memory moved into platforms. Framework middleware (LangChain 1.0 summarization middleware), managed memory (AgentCore Memory, Cortex Agents threads), and provider-side compaction handle much of what teams used to hand-roll.
  • Progressive disclosure for tools and skills. Load tool schemas or skill instructions only when they're needed, instead of injecting every MCP tool definition up front.
  • Prompt caching shapes assembly order. Put stable content (system prompt, tool definitions, reference docs) first, in a deterministic order, so cached prefixes keep hitting.

Context engineering is the discipline of deciding what goes into the model's context window, and how — the tokens the model actually sees on each call. It's the layer above prompt engineering: prompts are wording; context engineering is assembly and budget management across a whole conversation or agent run.

Why it matters

The context window is finite and every token costs money and latency. Two failure modes:

  • Too little — the model lacks the facts it needs → hallucination.
  • Too much / wrong — irrelevant tokens dilute attention ("lost in the middle"), raise cost, and can even lower accuracy.
flowchart TB
    subgraph CTX[Context window - finite budget]
      SYS[System instructions]
      HIST[Conversation history]
      RET[Retrieved chunks - RAG]
      TOOLS[Tool results / scratchpad]
      Q[Current user query]
    end
    CTX --> LLM[LLM]

What you're assembling

Piece Concern
System prompt Stable instructions, role, guardrails
Conversation history Grows unbounded — must be managed
Retrieved context (RAG) Only the top-k relevant chunks, with budget
Tool outputs / scratchpad Agent intermediate results, often verbose
Current query The actual request

Managing history (memory strategies)

Strategy How Trade-off
Full history Keep everything Simple; blows the budget fast
Sliding window Keep last N turns Cheap; loses old context
Summarization / compaction Summarize old turns into a running summary Preserves gist; summary can drift
Retrieval memory Store turns in a vector DB, retrieve relevant ones Scales; adds retrieval complexity
Structured memory Extract facts to a store (user prefs, entities) Precise; needs schema

Techniques that matter

  • Retrieval budgeting — cap how many chunks/tokens RAG injects; re-rank so the best are first (mitigates "lost in the middle").
  • Compaction — when history nears the limit, summarize older turns and drop raw text; keep recent turns verbatim.
  • Ordering — put the most important context near the top and bottom (models attend to the edges more than the middle).
  • Isolation — separate untrusted retrieved/tool content from instructions (also a prompt-injection defense).
  • Token accounting — measure tokens per component; know your budget.
  • Cache-friendly layout — stable prefix first (system, tools, docs), volatile parts (history, query) last, so provider prompt caching applies.
  • Just-in-time loading — fetch tool schemas, skills, or documents when the task needs them rather than front-loading everything.

Interview deep dive

60-second talking points

  • "Prompt engineering is the wording; context engineering is what's in the window and how it's budgeted."
  • "More context isn't better — relevance and ordering beat volume."
  • "Long conversations need compaction or retrieval memory, not raw history."

Scenario & system-design questions

A long chat agent starts giving worse answers over time. Why?

History grew until it crowded out room for retrieval/instructions, or key facts got pushed into the "lost in the middle" zone. Fix with compaction (summarize old turns), a sliding window for recency, and retrieval memory for older facts; re-rank so the most relevant context is at the edges.

How do you decide what to put in the context window for a RAG call?

System prompt + a budgeted set of top-k re-ranked chunks + the query, with token accounting so you don't exceed the window. Drop or summarize anything low-value; keep citations.

How does context engineering relate to prompt injection?

Retrieved and tool content is untrusted — keep it clearly separated from system instructions and never let it be interpreted as commands. Context assembly is where you enforce that boundary.

Pitfalls interviewers probe

  • Dumping full history/all chunks into every call (cost, dilution).
  • Ignoring "lost in the middle" (bad ordering).
  • No token budget/accounting.
  • Treating retrieved content as trusted instructions.
  • Summaries that silently lose critical facts.

Rapid-fire

Q A
Context vs prompt engineering? What's in the window & budget vs the wording
"Lost in the middle"? Models attend less to middle context → order matters
History strategies? Full, sliding window, summarization, retrieval memory
Compaction? Summarize old turns when nearing the token limit
Why not max context always? Cost, latency, and diluted attention