Building and operating AI5 min read

How Models Fit Into a Product

A practical introduction to model inputs, context, retrieval, variability and the decisions they create for product teams.

On this page
  1. 1.What a model receives and produces
  2. 2.Context has limits
  3. 3.Retrieval supplies information at the time of a task
  4. 4.The same task can produce different results
  5. 5.Choose what to change
  6. 6.Try it: follow the information

A research assistant answers a question about a workplace policy. It sounds certain, gives a source and is wrong. The team needs to find out whether the answer came from an outdated document, an unsuccessful search, a model interpretation or an error in the question itself.

You do not need to train a model to investigate that failure. You do need to know what information the model received and what the surrounding product did with its answer.

What a model receives and produces

A language model processes a sequence of input units called tokens and generates output tokens. Tokens are pieces of text, not necessarily whole words. A product may send instructions, a user's request, examples and retrieved documents as part of the input. Other kinds of models can work with images, audio or other data, sometimes through different representations.

The model generates an output from the information and instructions available to it. A fluent answer can still contain an unsupported claim. Ask where an important claim came from and test the system on tasks that resemble the work users need done.

Context has limits

The context window is the amount of input and output a model can handle in a request, subject to the system's limits. If a team tries to include a long policy library, conversation history and several documents at once, it may exceed those limits. Even when the material fits, an irrelevant or outdated passage can distract from the information the task needs.

A longer window is a capacity, not a guarantee that the system will use every detail correctly. Test the actual documents, questions and relevant information placement. Decide what to include, what to leave out and how a user can inspect or correct the source.

Retrieval supplies information at the time of a task

Rather than put an entire library into every request, a product can search a permitted collection and pass selected passages to the model. This approach is often called retrieval-augmented generation. It can bring current or private information into a task, provided the user is authorised to access it.

Retrieval adds its own failure points. The document may be missing or out of date; search may return the wrong passage; access controls may exclude the right one; or the model may misinterpret a good source. A citation helps a person investigate but does not, by itself, prove the answer is supported by the cited passage. The harness chapter walks through how to inspect these possibilities.

The same task can produce different results

Some model configurations can produce different outputs for the same input. The surrounding system may also change: a source can be updated, a tool can return a different result, or an instruction can be revised. Record the configuration and relevant context when comparing results. Run representative cases more than once when variation matters, and inspect the failures rather than relying on a single convincing demonstration.

Variability is not always undesirable. Different draft options may help a designer explore an interaction. A workflow that updates a customer record needs a much more dependable outcome and independently enforced limits on what it can do.

Choose what to change

When a task performs poorly, start with evidence about the failure. The following choices solve different problems:

  • Change the supplied context when the system lacks the relevant information or receives a stale source. This may involve updating a document, improving retrieval or providing a clearer example.
  • Change the instruction or workflow when the job, expected format or handoff is unclear. Check the new version against cases beyond the example that prompted the change.
  • Compare models when the current system cannot meet the task's quality, latency, cost or other constraints. Compare complete workflows on representative cases, not a model's reputation or a single benchmark.
  • Consider model adaptation when a repeated need cannot be met adequately through the existing model, context and workflow. Fine-tuning changes a model using additional training examples; it requires suitable data and evaluation, and it does not guarantee that current facts or permissions are handled correctly.

Work through these options with the people who build and maintain the system. The evaluation chapter explains how to compare changes without mistaking a successful demo for evidence of general reliability.

Try it: follow the information

Take a task you know, such as finding a policy answer or summarising a report. On paper, write the user's request, the information the system would need, where it would come from and how a person could check the result. Mark one point where the information might be missing or wrong.

With an engineer or another knowledgeable colleague, compare your map with an actual permitted trace, if one is available. If not, use the fictional example in the harness exercise. Identify what you can establish and what remains unknown.

v3.1 · Updated Oct 2026