A team builds a customer support assistant. In the demo it answers every question correctly. In production it invents refund policies, contradicts itself across turns, and triples the inference bill during peak hours. The failure is not in the model. It is in the assumption that a large language model, alone, is an intelligent system. Large language models are one component of intelligence, not the whole of it. The parts that decide whether a system survives contact with real users are the parts the model does not provide: persistent memory, a record of what is true, a planner that decides what to do next, bounded control over actions, and evaluation that catches regressions before customers do.

Where models end and systems begin

Large models perform well on a specific set of tasks: open ended generation, summarisation, translation, and code completion, where a single response is judged by a human reader. They begin to break down when a task requires state across turns, consistent reasoning, adherence to fixed business rules, or predictable cost and latency. The context window is a ceiling, not a memory. Information placed in it is forgotten at the next turn unless it is written back, and writing everything back makes every call more expensive. Hallucination is not a bug to be prompted away. It is a property of generative systems, and production systems must assume it will happen. Reasoning consistency fails in the same way: two identical inputs can produce two different outputs, because generation is stochastic. A system that cannot reproduce its own answer cannot be trusted with consequential decisions.

Model capabilities move fast. The patterns that let a system absorb those changes without a rewrite move much more slowly. Maharaj, 2026

Five failure patterns recur in production. The confident hallucinator states false policy as fact. The expensive loop enters a reasoning cycle that consumes tokens without converging. The prompt drift failure changes behaviour when a prompt is edited, with no test to catch it. The business rule leak lets the model override a fixed rule it was only supposed to follow. The silent state corruption writes incorrect information to memory, poisoning every later turn. When a team spends more time debugging prompts than code, when costs grow faster than usage, or when identical inputs produce different outputs, the system needs a redesign, not a larger model.

The frontier agent stack, in seven layers

A reliable intelligent system is built as a stack of seven layers, each with a single responsibility. The model sits inside the reasoning layer and is replaceable.

  • Perception and ingestion convert raw inputs, text, images, API events, into a structured form the rest of the system can reason about.
  • State and the world model track what is true at this moment, separate from what the model believes.
  • Reasoning is where the language model generates candidates, interpretations, and next step proposals.
  • Planning and control decide which action to take, under a bounded policy the model cannot override.
  • Memory persists facts, preferences, and history across conversations and sessions.
  • Hybrid guardrails combine deterministic code for rules that must never break with model judgement for open ended choices.
  • Evaluation and observability measure whether outputs meet a defined bar and detect drift in production.

The model is not absent from this stack. It is confined to the layer where stochastic generation is useful, and every layer above and below it is deterministic enough to test. Structured outputs are the contract between layers. When the model returns JSON with a fixed schema, the rest of the system can validate, route, and store its output without parsing free text. A model that fails to conform to the schema is a failed request, not a silent corruption.

From prompt design to interface design

The most durable shift in the book is the move from prompt design to interface design. A prompt is a fragile artefact: change one sentence and behaviour changes unpredictably. An interface is a stable boundary: a request router decides which model or tool handles a request, a session state object holds the current conversation, a memory interface reads and writes persistent facts, a reasoning loop calls the model until a stopping condition is met, and an evaluation harness scores every output. Each component can be replaced, upgraded, or tested independently because the boundaries are explicit. When the underlying model changes, as it will, the system absorbs the change through its interfaces rather than through a rewrite.

Multi agent coordination follows the same principle. The three role pattern assigns an agent to propose, an agent to verify, and an agent to execute under control policy. No single agent holds all three powers. Hybrid neuro symbolic layers handle compliance: deterministic code enforces the rules that can be written down, while the model handles the open ended judgement that cannot. This is what makes a system explainable. When a failure occurs, the team can point to which layer produced it, instead of concluding that the model is mysterious.

What changes in practice

Three practices separate systems that scale from systems that stall. First, evaluate before release on a fixed set of representative tasks, so that every prompt or model change is measured against a known bar. Second, monitor after release, tracking cost, latency, and failure rate per request, so that drift is detected when it begins, not when customers complain. Third, keep a human in the loop for consequential decisions while the system builds a track record. None of this is exotic. It is ordinary engineering discipline applied to a component that happens to be stochastic. The frontier of AI is not a larger model. It is the system architecture around the model, and that architecture is what determines whether intelligence survives in production.

Further notes on applied AI engineering are published on the blog, and system design questions can be sent through the contact page. The full treatment is available in Beyond LLMs: The Next Frontier of AI by Sahir Maharaj (Packt, 2026).