The Real Architecture Behind Production AI Agents

Most agent diagrams show a clean sequence: context, rules, agents, action. It is elegant and easy to explain. But the moment an agent can create a ticket, comment on a pull request, send an email, or change a business record, the model is no longer the whole system. It is one decision-making component inside a backend that must control identity, state, risk, and failure.

A demo can succeed once while a developer watches. A production system must behave predictably across different users, permissions, tools, partial outages, duplicated events, and long pauses. It must also leave enough evidence for an operator to reconstruct what happened.

The architecture becomes easier to reason about when it is divided by responsibility. Connectors move data and actions across system boundaries. Identity and context decide what the caller may know. Orchestration coordinates work. Policy limits actions. Durable state and observability make the system recoverable and inspectable.

These are logical boundaries, not a requirement for ten separate services. A small team can implement several responsibilities in one application. The important part is that the boundaries remain visible in code, data, and traces.

A connector is not merely an API wrapper. It must handle pagination, webhooks, incremental sync, deletions, rate limits, retries, metadata, and source-specific semantics. It must also distinguish reading data from taking action because those operations rarely deserve the same authority.

• Propagate the caller. Prefer user-scoped authorization when the action represents a user. Use service identities only for explicitly system-owned work.
• Minimise capability. Restrict tokens to the resources and actions required for the current task. OAuth security guidance recommends restricting access-token privilege by resource and action.
• Enforce access before disclosure. Permission checks belong in retrieval and tool execution, before restricted content reaches prompts, traces, or logs.
• Preserve provenance. Keep source identifiers, revisions, owners, and timestamps so the agent and the reader can judge authority and freshness.

The detailed indexing and retrieval mechanics are covered in the RAG 101 visual guide. Here, the key architectural rule is that enterprise context must remain permission-aware from ingestion through action.

Agents do not coordinate by magic. If a product task leads to implementation, testing, and release, some component must decide what happens next, persist the current stage, route the work, and handle a failed or late response.

• The orchestrator owns progress. It decomposes or routes work, records transitions, applies timeouts, and decides whether a failure should retry, compensate, escalate, or stop.
• An agent owns bounded judgment. It interprets context and proposes or performs work through a limited tool set with explicit success criteria.
• Tools own deterministic effects. Tool adapters validate structured input, enforce authorization, use idempotency keys, and return typed results.
• Policy owns permission to proceed. The model may recommend an action, but deterministic controls decide whether the action is allowed and whether approval is required.

Useful work often spans minutes, hours, or days. Tests run, webhooks arrive, people approve, rate limits reset, and external systems recover. Holding all of that inside a request or model conversation makes the workflow difficult to resume and almost impossible to audit.

• Checkpoint after meaningful transitions. Persist the plan, completed steps, tool results, pending approval, and the next valid actions.
• Assume duplicate delivery. Queues and webhooks may deliver an event more than once. Use stable workflow and action identifiers so retries do not duplicate external effects.
• Bound every wait. Tool calls, approval requests, and callbacks need timeouts with an explicit expiry path.
• Separate retry from repair. Retry transient failures with limits and backoff. Route invalid input, denied access, and repeated failures to a corrective or human path.

Rules are best treated as a cross-cutting control plane rather than one prompt or one box between context and the agent. Validate input before planning, enforce permissions during retrieval, constrain tools before execution, and validate output before an external effect.

Risk is not a property of the model alone. It depends on the caller, target system, data sensitivity, blast radius, reversibility, and evidence available for the decision. OWASP describes excessive agency as a risk when an LLM system receives unnecessary functionality, permissions, or autonomy.

Human-in-the-loop should not appear only after the system has failed. It should be designed into the workflow wherever judgment, accountability, or irreversible impact requires a person.

• Give the reviewer context. Show the proposed action, supporting evidence, target, expected effect, and what will happen after approval.
• Make approval specific. Approval for one email or deployment must not become open-ended permission for later actions.
• Support correction. Reviewers should be able to edit, reject, or request more evidence—not merely click approve.
• Handle silence. Define expiry, reminders, reassignment, and cancellation when nobody responds.

The goal is not to make every action slow. Low-risk reads and drafts may proceed automatically. Higher-risk writes should require progressively stronger validation or approval.

Traditional observability asks whether the system is available. Agent systems must also answer whether the work was useful, grounded, authorised, and completed. That requires connecting infrastructure telemetry with workflow and quality signals.

• Trace causality. Correlate the user request, workflow, model calls, retrieval, tool calls, retries, approvals, and external changes under stable identifiers.
• Protect recorded content. Prompts, tool arguments, and results may contain secrets or personal data. Default to metadata and sampled, redacted content rather than indiscriminate logging.
• Measure system quality. Track task completion, tool success, policy denials, escalation, correction, abandonment, latency, and cost by workflow step.
• Evaluate with real tasks. Maintain representative cases and adversarial scenarios. Score the final business outcome as well as intermediate model output.

OpenTelemetry’s semantic conventions provide a common vocabulary for traces, metrics, logs, and events. Its Generative AI conventions are still evolving, so pin the convention version and treat model or tool content as sensitive.

The exact framework matters less than the contracts between planning, policy, state, and effect. The following sketch keeps those responsibilities visible.

def run_workflow(workflow_id: str, request: WorkRequest) -> Result:
    state = state_store.load_or_create(
        workflow_id=workflow_id,
        requested_by=request.principal_id,
        objective=request.objective,
    )

    principal = identity.resolve_and_revalidate(state.requested_by)
    plan = planner.create_or_resume(state, request.context)

    for step in plan.pending_steps():
        action = agent.propose(step=step, context=state.safe_context())
        decision = policy.evaluate(
            principal=principal,
            action=action,
            evidence=state.evidence,
        )

        audit.record(workflow_id, action, decision)

        if decision.requires_approval:
            state.checkpoint(status="awaiting_approval", action=action)
            return approvals.request(workflow_id, action, decision.reason)

        if not decision.allowed:
            return state.finish(status="denied", reason=decision.reason)

        result = tools.execute(
            action,
            principal=principal,
            idempotency_key=f"{workflow_id}:{step.id}",
            timeout=step.timeout,
            retry_policy=step.retry_policy,
        )

        state.checkpoint(
            status="step_completed",
            step=step,
            result=result.redacted(),
        )

    return state.finish(status="completed")

1. Map every external effect. Identify the target, required authority, reversibility, blast radius, and approval rule.
2. Make tool contracts explicit. Validate structured inputs and outputs, distinguish retryable errors, and assign idempotency semantics.
3. Model the workflow states. Include waiting, denied, expired, cancelled, failed, compensated, and completed—not just running and done.
4. Test permissions end to end. Verify connector reads, retrieval, prompts, logs, tool calls, and resumed workflows with multiple roles and revoked access.
5. Exercise failure paths. Simulate timeouts, duplicate events, partial tool success, stale approvals, malformed model output, and unavailable dependencies.
6. Define the human interface. Reviewers need evidence, scope, editable proposals, expiry, and an audit trail.
7. Instrument business outcomes. Connect traces and costs to completion, correction, escalation, and user value.
8. Increase autonomy gradually. Promote workflows only after measured performance and operational review justify a wider action boundary.