A model call has no persistent knowledge of your current task beyond the state the application supplies. The model still has pretrained knowledge, but your instructions, private documents, tool results and conversation state must arrive through the current context or connected tools. Context engineering is the work of deciding what to supply for every call.
Prompt writing is one part of it. The larger part is plumbing: retrieving, ranking, trimming and formatting information so the model sees the right facts in a shape it can use.
In a single-turn chat the prompt is most of the context. In an agent, the prompt is a small fixed header on top of a context that changes every step. After a few tool calls, most of the tokens the model reads were not written by you at all.
That changes where quality problems come from:
- The model answers from a stale or irrelevant document that was retrieved.
- A long tool result pushes the actual question out of focus.
- Old turns contradict the current task.
Key takeaway
When an agent gives a bad answer, look at the exact context it received before rewriting the prompt.
This article focuses on four high-volume inputs. A real request can also include tool schemas, multimodal inputs and provider metadata, all of which consume budget.
| Input | Changes | Typical risk |
|---|---|---|
| Instructions | Rarely | Too long, conflicting rules |
| Retrieved documents | Every query | Irrelevant or duplicated chunks |
| Tool results | Every step | Huge raw payloads |
| Conversation history | Every turn | Outdated goals and decisions |
Give each input a token budget instead of concatenating everything and hoping it fits. When an input exceeds its budget, trim that input, not the others.
type Part = { name: string; text: string; budget: number };
const approximateTokens = (text: string) => Math.ceil(text.length / 4);
export const assembleContext = (parts: Part[]) =>
parts
.map(({ name, text, budget }) => {
const limit = budget * 4;
const clipped = text.length > limit ? `${text.slice(0, limit)}\n[truncated]` : text;
return `<${name}>\n${clipped}\n</${name}>`;
})
.join("\n\n");
export const fits = (context: string, window: number, reservedOutput: number) =>
approximateTokens(context) <= window - reservedOutput;The four-characters-per-token estimate is a rough English heuristic. Code and multilingual text can differ substantially, so use the model's tokenizer or provider token-counting API for production limits and reserve space for the answer.
Tool results are the fastest-growing part of an agent's context. Keep raw results only in a controlled trace store when debugging or audit requirements justify it, with redaction, access control and retention limits. Give the model a short, structured summary with only the fields it needs for the next decision.
Replace older conversation turns with a running summary of decisions and open questions. Keep the last few turns verbatim so the model still sees the user's exact wording.
Wrap each input in a clear label, as in the example above. Labels let the instructions refer to "the documents" or "the latest tool result" without ambiguity.
Measure before you tune
Capture a redacted sample of assembled contexts in a restricted evaluation environment and read it. Do not place secrets, personal data or raw tool payloads in ordinary application logs.
- Start with budgets and labels. They are cheap and prevent the worst failures.
- Add summarisation when tool results or history regularly hit their budget.
- Add retrieval ranking when answers cite the wrong documents.
Context engineering is not a single technique. It is the habit of treating the context window as a small, carefully curated workspace rather than a log file.