Skip to content

Search articles, repositories, topics and pages.

Context Engineering

Understanding Context Engineering in AI Agents

Why the context you assemble matters more than the prompt you write, and practical techniques to keep it relevant and within budget.

TIAN NEWS AI

Published Updated 3 min read

Instructions, retrieved documents, tool results and conversation history flowing into a context builder before a model call

What context engineering is

A model call has no persistent knowledge of your current task beyond the state the application supplies. The model still has pretrained knowledge, but your instructions, private documents, tool results and conversation state must arrive through the current context or connected tools. Context engineering is the work of deciding what to supply for every call.

Prompt writing is one part of it. The larger part is plumbing: retrieving, ranking, trimming and formatting information so the model sees the right facts in a shape it can use.

Why it matters more than the prompt

In a single-turn chat the prompt is most of the context. In an agent, the prompt is a small fixed header on top of a context that changes every step. After a few tool calls, most of the tokens the model reads were not written by you at all.

That changes where quality problems come from:

  • The model answers from a stale or irrelevant document that was retrieved.
  • A long tool result pushes the actual question out of focus.
  • Old turns contradict the current task.

Key takeaway

When an agent gives a bad answer, look at the exact context it received before rewriting the prompt.

The four inputs

This article focuses on four high-volume inputs. A real request can also include tool schemas, multimodal inputs and provider metadata, all of which consume budget.

InputChangesTypical risk
InstructionsRarelyToo long, conflicting rules
Retrieved documentsEvery queryIrrelevant or duplicated chunks
Tool resultsEvery stepHuge raw payloads
Conversation historyEvery turnOutdated goals and decisions

Techniques that work

Budget each input

Give each input a token budget instead of concatenating everything and hoping it fits. When an input exceeds its budget, trim that input, not the others.

context-budget.ts, ts
type Part = { name: string; text: string; budget: number };

const approximateTokens = (text: string) => Math.ceil(text.length / 4);

export const assembleContext = (parts: Part[]) =>
  parts
    .map(({ name, text, budget }) => {
      const limit = budget * 4;
      const clipped = text.length > limit ? `${text.slice(0, limit)}\n[truncated]` : text;
      return `<${name}>\n${clipped}\n</${name}>`;
    })
    .join("\n\n");

export const fits = (context: string, window: number, reservedOutput: number) =>
  approximateTokens(context) <= window - reservedOutput;

The four-characters-per-token estimate is a rough English heuristic. Code and multilingual text can differ substantially, so use the model's tokenizer or provider token-counting API for production limits and reserve space for the answer.

Summarise tool results

Tool results are the fastest-growing part of an agent's context. Keep raw results only in a controlled trace store when debugging or audit requirements justify it, with redaction, access control and retention limits. Give the model a short, structured summary with only the fields it needs for the next decision.

Compact old turns

Replace older conversation turns with a running summary of decisions and open questions. Keep the last few turns verbatim so the model still sees the user's exact wording.

Label every section

Wrap each input in a clear label, as in the example above. Labels let the instructions refer to "the documents" or "the latest tool result" without ambiguity.

Measure before you tune

Capture a redacted sample of assembled contexts in a restricted evaluation environment and read it. Do not place secrets, personal data or raw tool payloads in ordinary application logs.

When to use what

  • Start with budgets and labels. They are cheap and prevent the worst failures.
  • Add summarisation when tool results or history regularly hit their budget.
  • Add retrieval ranking when answers cite the wrong documents.

Context engineering is not a single technique. It is the habit of treating the context window as a small, carefully curated workspace rather than a log file.