A support assistant is asked to refund a customer. It must verify the order, check the policy limits, and execute the refund. A language model alone can do none of this reliably. It can write a convincing apology. It cannot confirm the order exists, decide whether the refund is permitted, or move money. The gap between that convincing text and a system trusted with real actions is where most agent projects fail, and it is where the engineering discipline of production AI begins.

The demonstration is the easy part

Adding a large language model to an application is easy. Building an intelligent application that can reliably use data, call tools, make decisions, complete tasks, and operate in production is much harder. The first claim can be verified in an afternoon. The second requires months of engineering, and most teams that skip it discover the difference only after an incident in production. The failure is rarely the model. It is the absence of structure around the model.

Adding an LLM to an application is easy. Building an intelligent application that can reliably use your data, call tools, make decisions, complete tasks, and operate in production is much harder. Noring, 2026

The core distinction is between retrieval and generation. A language model generates text by predicting likely continuations. It does not retrieve facts, and it does not execute actions. When an agent appears to know something, that knowledge either came from the model's training weights, which are static and can be wrong, or from a retrieval step that injected verified context into the prompt. A production system must know which of the two it is relying on at every step. Confusing the two is the most common source of hallucination in deployed agents.

Ground the model before letting it act

The production stack is built in three layers. Retrieval augmented generation grounds answers in business and application data. Ingestion splits documents into coherent chunks, converts them to embeddings, and stores them in a vector database. At query time, a similarity search, usually cosine similarity, returns the few chunks relevant to the request, and only those are placed in the model's context window. The model then generates from a bounded, verifiable source rather than from memory alone. Chunk size and retrieval threshold are not implementation details. They determine whether the context the model receives is coherent and complete, and a poor choice here degrades every downstream answer.

The second layer is tool calling. The model is given a schema describing each available tool, its parameters, and its purpose, and it decides when to invoke one. The application, not the model, executes the function and returns the result. This separation is what turns a chatbot into an agent, and it is why tool schemas must be precise. An ambiguous parameter description will be interpreted creatively, and the model has no way to know it has misread the intent. The third layer, the Model Context Protocol, standardizes how agents connect to external tools and data sources so that the same agent can be extended without rewriting its internals.

None of these layers is optional in production. A model without retrieval will hallucinate facts. A model without bounded tools will suggest actions it cannot verify. A model without a protocol around its integrations will become a bespoke system that cannot be maintained as it grows.

Guardrails come before capability

Responsible design is not a final review stage. It is the first architectural decision, because safeguards added after launch are always weaker than safeguards designed in. A single unsupported claim can propagate through a chain of tool calls and produce a cascade of wrong actions, each one harder to reverse than the last. Five guardrails prevent this, and each maps to a concrete failure mode.

  • Give the model clear instructions about what it may and may not do.
  • Limit which actions are available to it, so a reasoning error cannot reach a dangerous tool.
  • Check permissions before any action is taken, not after.
  • Verify important facts independently of the model's own claim.
  • Keep a human in control of consequential decisions.

The fourth guardrail deserves emphasis. A model that reports a fact should not be trusted merely because it sounds confident. Important facts are checked against the source system independently, whether that is a database query, an API call, or a human review. Equally important is explainability. A deployed agent should make clear what it knows, what it is uncertain about, and what it has not yet done, and it should keep a record that allows any decision to be reconstructed from request to outcome without copying secrets into logs.

Evaluate before you deploy, monitor after

Evaluation and monitoring are different activities, and treating them as the same thing is a costly error. Evaluation happens before release, on a fixed set of representative tasks, and measures whether the agent meets a defined bar. Monitoring happens after release, on live traffic, and detects drift, failures, and new failure modes. Teams that confuse them either ship without a measurable standard or fail to notice when a working system degrades. Structured outputs and deterministic checks are preferred wherever possible, because a machine readable answer can be tested automatically, while free text can only be sampled by hand. A release without an evaluation set is a release whose quality is unknown, and unknown quality in an agent that takes actions is an operational risk.

The system around the model

Production agents are not models with more prompts. They are harnesses: context management, bounded tools, retrieval pipelines, permission layers, evaluation suites, and observability, arranged so that the model is one component among many. The model is the reasoning engine, and the harness determines whether that reasoning is grounded, bounded, and reversible. Teams that start from the harness and add the model last ship reliable systems. Teams that start from the model and add structure later spend their effort recovering from incidents. The distinction is not a matter of scale. It applies to a single internal tool as much as to a customer facing platform.

Further engineering notes are published on the blog, and questions about production agent design can be directed through the contact page. The source treatment of these patterns is available in Building Production AI Agents for the Web (Packt, 2026).