Deploying AI coding agents into production requires moving beyond simple prompt-response patterns. Architects must treat the model as a probabilistic reasoning engine rather than deterministic code.
A clean separation between the harness, the model, and the UI is essential for maintaining control. This architecture prevents the model from becoming a black box that is impossible to debug or scale.
In short
- •
Implement a three-layer architecture: a harness for orchestration, a model for reasoning, and a UI for user interaction.
- •
Expect a 10-50x cost multiplier when moving from hardcoded workflows to agentic systems due to reasoning overhead and token consumption.
- •
Instrument every thought, action, and observation using OpenTelemetry to maintain visibility into agent decision-making loops.
The Three-Layer Architecture
The harness acts as the primary orchestrator, managing the agent loop and tool execution. By isolating the harness, developers ensure that the agent's actions remain predictable even when the underlying model's reasoning is probabilistic.
The model serves as the reasoning engine. It should not be burdened with state management or UI concerns. Keeping this layer thin allows for easier model swapping and performance tuning as requirements evolve.
Managing Production Costs
Teams often underestimate the cost of agentic systems. A workflow that executes five hardcoded steps might cost pennies, but an agent reasoning through twenty decisions to complete the same task can increase costs by an order of magnitude.
Each decision step consumes tokens and accumulates context. To prevent budget spikes, prioritize hardening perception and action layers first. Refine reasoning logic only after establishing a baseline for cost and performance.
Observability is the final piece of the puzzle. Without granular telemetry, debugging a failed agent loop is nearly impossible. Send every thought, action, and observation to your observability stack to identify where reasoning goes off track.
Source
AI Agents in Production: The Three-Layer Architecture
https://aienhancedengineer.substack.com/p/ai-agents-in-production-the-three
