How to Give AI Better Context: A Plain Guide to Context Engineering

Quick answer: For an AI model, context is everything it sees at once: your instructions, the files, the history, whatever got retrieved. Supplying the right context is the single biggest lever on the quality of the answer, but the research is blunt that more is not better: models get worse as the input grows, even when the extra material is relevant. The skill, now called context engineering, is curating the smallest set of high-signal information the model needs, and talking is one of the cheapest ways to get all of it out.

Ask two people to get the same answer out of the same AI model and you will often get very different results. The usual explanation is that one of them is better at "prompting." That is part of it, but it undersells what is really going on. The bigger difference is usually context: what each person put in front of the model before asking the question.

This is the lever most people never touch, and it is the one that moves the answer the most.

What "context" actually means

For a language model, context is the entire set of information it sees at the moment it generates a reply. Anthropic, in its engineering writeup on the topic, describes context as everything that lands in the model's window during inference: the system instructions, your prompt, the running conversation history, any documents that got retrieved, and the outputs of any tools the model called. It is not just the sentence you typed. It is the whole pile the model is looking at when it answers.

The important part of Anthropic's framing is what kind of resource this is. Context is finite, and it has, in their words, diminishing marginal returns. The model spends a limited "attention budget" across everything in the window, the way you spend limited working memory across everything on your desk. Fill the desk with clutter and the important page gets harder to find, not easier.

Why context is the main lever

Here is the claim worth sitting with: for most real tasks, the model is not the bottleneck. The context is.

Philipp Schmid, who works on AI developer relations at Google DeepMind, put it flatly: most agent failures are no longer model failures, they are context failures. The model was capable of the right answer; it just was not given the pieces it needed to produce it. His definition of the job is getting the right information and tools, in the right format, at the right time, in front of the model.

That reframes a lot of frustration with AI tools. When an assistant confidently does the wrong thing, the instinct is to blame the model or reach for a bigger one. Often the real fix is upstream: the constraint you assumed it knew, the file you did not mention, the example that would have pinned down what "good" looks like. The model filled the gap with a guess because you left a gap.

The failure modes the research documents

If context were just "more is better," this would be an easy problem. It is not, and a run of studies from 2023 through 2025 spells out exactly how it breaks.

Lost in the middle. In the paper of that name (Liu et al., published in the journal TACL, out of Stanford and collaborators), researchers found that a model's ability to use information depends on where that information sits. Performance is highest when the relevant material is near the beginning or the end of the input and drops off sharply when the model has to pull it from the middle, tracing a U-shaped curve. This held even for models built specifically for long inputs, which means enlarging the window does not fix it.

Context rot. A 2025 report from Chroma tested 18 models on tasks as simple as reproducing repeated words, and found performance degraded as the input got longer even on those trivial tasks. Anthropic uses the same phrase: as the number of tokens in the window climbs, the model's ability to accurately recall what is in there goes down. The effect is uneven rather than a clean straight line, but the direction is consistent.

The window is bigger than the usable space. A benchmark called NoLiMa (2025) made one careful change: it removed the literal word matches between the question and the answer, so the model had to actually reason about the connection instead of keyword-spotting. Long-context performance fell off a cliff. At 32,000 tokens, well inside what these models advertise, most of the tested models dropped below half of their short-input accuracy. Those exact numbers are specific to the models and the date, and they will shift as models improve, but the lesson holds: the effective, usable context is far shorter than the number on the box.

Length alone hurts. The most pointed result comes from a 2025 paper (Du, Tian, and colleagues, at EMNLP) showing that even when a model perfectly retrieves every relevant fact, its performance still degrades, by anywhere from about 14 percent to 85 percent, as the input grows longer, while staying inside the claimed window. They even masked out the irrelevant text so the model was forced to look only at what mattered. It still got worse. That rules out distraction as the whole story and isolates raw length itself as a cause.

More context is not better

Put those together and you get the counterpoint that runs against most people's instinct: adding context is not free, and past a point it is actively harmful.

This is the honest center of the whole topic. It is tempting to treat a large context window as a bucket you should fill, dumping in the entire codebase, the whole document, every prior message, on the theory that the model can sort it out. The evidence says it cannot, and that you pay for the excess in accuracy. The goal Anthropic names is the opposite of filling the bucket: finding the smallest possible set of high-signal tokens that get you the outcome you want. Curation beats accumulation. Relevant and complete beats long.

So how do you give an agent good context?

The practical version of all this is less about clever wording and more about deciding what belongs in the window.

Be specific and include what the task actually needs: the exact files in play, the constraints that matter, an example of the output you want. Leave out the rest. That is the same lesson from our piece on whether talking to AI beats typing: what improves the answer is complete, relevant context, not sheer volume.

For anything you repeat, persistent context files carry the load. A file like CLAUDE.md or AGENTS.md is loaded automatically at the start of a session, which turns an otherwise stateless assistant into one that remembers your project's conventions, structure, and patterns. These files merge in layers, from broad user-level defaults down to project and directory specifics, so the most relevant rules win. It is durable memory you write once instead of re-explaining every time.

And rather than pre-loading everything, tools increasingly fetch context on demand. Retrieval pulls in only the passages relevant to the current question, and the Model Context Protocol (MCP) is a standard way for an assistant to reach out to your tools and data when it needs them. Both push in the same direction: bring in the right material at the right moment instead of stuffing it all in up front. (These are newer, and the hard evidence on how much they help in practice is still thin, so treat them as sensible approaches rather than settled numbers.)

All of this now travels under a name. Context engineering, a term proposed by Shopify CEO Tobi Lutke, endorsed by Andrej Karpathy, who called it the art of filling the window with just the right information for the next step, and formalized by Anthropic, is simply the discipline of managing everything the model sees. Prompt engineering, wording the request well, is one piece of it.

Where voice comes in

Here is the friction problem hiding underneath the good advice. Giving a model rich, specific context means saying a lot: the background, the constraint, the file you mean, the edge case that always breaks, the example of what right looks like. Typing all of that feels expensive, so people quietly skip it and send the thin version, which is exactly the under-specified prompt that produces the worse answer.

Talking collapses that cost. When it is easy to get the full picture out of your head, you tend to actually include it, and complete context is the thing the model needed. That is our argument for voice, not a lab result: the win is not raw words per minute, it is that being thorough stops feeling like work.

That is what Rubber Duck is for. You hold a key and talk your whole context into whatever AI tool your cursor is in, and it transcribes on your Mac so the audio never leaves your computer. You are not trying to fill the window. You are trying to get the right things into it, and your mouth is the fastest way to do that.