PRODUCTION ARCHITECTURE · OCTOBER 2026
A capable model still needs a reliable operating system around it.
Build the agent around an explicit task contract, deterministic permissions, durable execution and tests of real outcomes. Choose a current model through your workload evaluation; a historical model leaderboard cannot certify production reliability.
Define the contract before selecting a model
Write the task in terms of the destination system. An agent asked to prepare a refund must identify the customer and order, retrieve the applicable policy, calculate the allowed amount and present an approval request. Its contract should distinguish preparing the refund from issuing it. List required evidence, allowed tools, authority limits, completion conditions and escalation rules.
Specify resource limits as part of behavior: maximum tool calls, elapsed time, retry count and spend per task. Define what happens when the budget runs out. A graceful handoff with preserved evidence is better than a loop that continues calling the same failing tool.
Use current provider catalogs to select candidate models and confirm supported interfaces, modalities and availability status. OpenAI, Anthropic and Google update their catalogs independently. Compare candidates on your tool schemas, language mix and exceptions instead of assuming an old flagship remains the correct deployment choice. OpenAI models; Claude models; Gemini models
Separate reasoning from execution authority
A ReAct-style loop alternates interpretation and tool use; a plan-and-execute design represents a sequence before running it. Both need bounded execution. An explicit plan can simplify approval and diagnosis, but it can also become stale when evidence changes. Revalidate the next action against current state rather than treating the original plan as irrevocable.
Expose narrow tools such as read_order, prepare_return and submit_approved_refund. Avoid a broad arbitrary database or shell tool when a constrained operation will do. The server handling the tool must authenticate the caller, verify the business object and enforce permissions. A model-generated customer ID is an argument to validate, not authority to access that customer.
MCP can standardize integration, but its security guidance documents risks around authorization, token handling and confused-deputy behavior. Identify who trusts each server, who authorizes its actions and how credentials are scoped. Tool compatibility is distinct from trustworthy execution. MCP security guidance
Design memory and recovery deliberately
Separate temporary task state, user preferences and authoritative business records. Task state contains the current case, evidence references, pending approvals and completed actions. A preference such as reply language should not override payment policy. Accounting or order status belongs in the system of record, not a conversation summary.
Store an action ledger with an operation ID, target object, intended mutation, approval reference and observed result. Before retrying after a timeout, inspect whether the mutation already occurred. Use idempotency keys where the destination supports them and your own duplicate protection where it does not.
Durable execution frameworks can preserve progress across failures. LangGraph documents persistence and human oversight, but you must still choose storage, define resumption rules and test side effects. Replaying the reasoning step is not always safe if it replays a charge, refund or outbound message. LangGraph execution overview
Treat outside content as untrusted data
An email, retrieved page or tool response can contain instructions intended to hijack the workflow. Keep external content separate from system policy and never let it add tools, change the recipient or waive an approval. Validate tool arguments after reasoning and enforce sensitive-action policy outside the model.
Use layered defenses: constrained permissions, content handling, action validation, review gates and incident detection. OpenAI documents input, output and tool guardrails; these checks complement server-side authorization rather than replacing it. DeepMind’s agent-security roadmap similarly emphasizes defense in depth and incremental permissions. Guardrail mechanisms; Agent security roadmap
Test attacks that ask the agent to reveal credentials, follow a link, replace bank details or ignore a policy embedded elsewhere. Verify the actual refusal or escalation behavior and destination state. A polite answer is not sufficient if a tool call has already leaked a secret or changed an account.
Release against workflow evidence
Build a versioned evaluation set covering normal work, difficult cases, adversarial input, missing evidence and tool outages. Score correct outcome, harmful action, citation validity, handoff usefulness, latency and cost separately. Track failures by retrieval, reasoning, authorization and execution so the fix addresses the right layer.
Capture trace IDs, model and prompt versions, evidence IDs, tool arguments with sensitive values redacted, approvals, duration and final system state. OpenTelemetry’s GenAI attribute registry includes evolving conventions, so pin the schema your instrumentation uses rather than assuming every field is permanent. OpenTelemetry GenAI attributes
Run offline replay first, then shadow processing, then a bounded live cohort. Compare against the current workflow and maintain a fallback owner. Promote a changed model only after the regression suite passes. Document unresolved limitations instead of treating a passing demo as proof that every customer request is safe.
Compare the choices
| Pattern | Useful for | Watch closely |
|---|---|---|
| Deterministic workflow with model steps | Known process with constrained interpretation | Schema validation and branch coverage. |
| Bounded tool loop | Uncertain sequence within a narrow task | Loops, budgets and unauthorized tool arguments. |
| Plan and execute | Auditable multi-step work and approval | Stale plans and changes between approval and action. |
| Multiple agents | Distinct tasks with clear interfaces | Duplicated authority, coordination and diagnostic complexity. |
Worked example: make the decision concrete
Illustrative incident rehearsal: a refund tool times out after the payment provider accepts the request. The agent cannot infer failure from the timeout. Its recovery step reads the operation record or payment status using the same business identifier, then records the confirmed result. It submits a second refund only if the first definitively did not occur and the original approval still applies.
Add this case to the regression set together with an expired approval and a changed order amount. The expected result is explicit for each variant. This provides much stronger release evidence than a general-purpose benchmark score, and it tests engineering behavior that a more capable model alone cannot supply.
Failure modes to plan for
Common defects include cross-customer memory leakage, retrying side effects, dropping evidence during handoff and treating a summary as the latest source of truth. Keep customer boundaries in storage and authentication, not merely in a prompt.
Another failure is excessive observability: logs may capture secrets or personal data. Redact and restrict trace access while preserving enough identifiers to investigate. If a production incident requires a model rollout to fix a permission defect, review the architecture; the tool service should enforce that boundary independently.
Your useful next step
Write one task contract, implement narrow read-only tools and build a replay set with recovery and injection cases. Add a reversible write behind approval only after you can trace its complete lifecycle and recover from a timeout without duplicating the action.
Questions and answers
Which current model should I deploy?
Select candidates from current official catalogs, then evaluate them on your task, tools, languages, latency and cost. No model choice removes the need for authorization and recovery controls.
Does an agent need long-term memory?
Only store memory that serves a defined purpose with an owner and retention rule. Keep authoritative business facts in the source system and retrieve their current state when acting.
Are guardrails enough to stop prompt injection?
No single guardrail guarantees that. Combine limited tools, deterministic authorization, validated arguments, careful content handling, adversarial tests and monitoring.
When should I add multiple agents?
Add them when distinct responsibilities or context boundaries justify coordination costs. Begin with the simplest architecture that meets the task contract and operating requirements.
Sources and evidence scope
Primary sources checked on 11 October 2026. Product pages establish vendor-described features; they do not establish independent comparative performance. Worked examples are explicitly illustrative. Implementation recommendations are editorial guidance.
- Openai Models — Official catalog documents current model identifiers, capabilities and supported interfaces.
- Claude Models — Official catalog documents current Claude models, identifiers and capabilities.
- Gemini Models — Official catalog documents Gemini models, availability status and capabilities.
- MCP security best practices — Documents authorization and token-handling risks including confused deputy and token passthrough.
- LangGraph overview — Documents durable execution, persistence, streaming and human oversight for agent orchestration.
- Guardrails — Documents input/output and tool guardrails; guardrails do not themselves replace authorization.
- Deepmind — June 18 roadmap supports insider-risk threat modeling, defense in depth, monitoring, prevention/response and incremental permissions.
- Otel — GenAI attributes include Development status and deprecated field names; pin semantic-convention versions.