Deterministic Guardrails for Probabilistic Agents: Pydantic, Circuit Breakers & Dead-Letter Queues

The Problem with “Prompt-Only” Guardrails

When engineering teams encounter unwanted agent behavior, the default reflex is to add more rules to the system prompt: “You MUST ALWAYS return valid JSON” or “NEVER execute deletions without confirmation.”

This is a category error. System prompts are soft guidance; they are suggestions given to a statistical token predictor. They are not deterministic control structures. Under stress, long context windows, or adversarial inputs, prompt-only constraints decay.

The Three-Tier Guardrail Architecture

In production systems like Wellows, we enforce guardrails across three distinct architectural perimeters:

[User Input]
     ↓
[Tier 1: Ingress Guardrail] ──(Pass/Reject)──→ Fast Token/Injection Filter
     ↓
[Tier 2: Runtime Sandbox]   ──(StateGraph) ──→ Max Iteration & Tool Whitelist
     ↓
[Tier 3: Egress Validator]  ──(Pydantic)   ──→ Strict Schema Contract & Asserts
     ↓
[Safe Output Dispatched]

Tier 1: Ingress Screening & Input Normalization

Before a customer query touches an expensive reasoning model, it passes through lightweight deterministic sanitization:

• Input Token Truncation: Hard ceiling on payload length preventing context window exhaustion attacks.
• Adversarial Pattern Matching: Fast regex and embedding checks screening for jailbreak templates (e.g. “Ignore previous instructions”).
• Intent Router: Directing administrative or read-only queries to deterministic SQL/cache handlers without invoking an agent at all.

Tier 2: Runtime Execution Sandboxes

Within LangGraph, every agent node executes within constrained boundaries:

• Hard Recursion Limits: LangGraph state graphs must be initialized with strict recursion_limit caps (typically 10–15 steps). If an agent fails to reach a terminal state within the budget, it transitions immediately to an error node.
• Scoped Tool Credentials: Agent tool calls execute using scoped service accounts with least-privilege permissions. An agent analyzing a database never possesses write credentials.
• Human-in-the-Loop Escalation: High-consequence actions (e.g. sending a customer email, initiating a wire transfer, or deleting a record) emit a suspended state waiting for explicit human review.

Tier 3: Egress Schema Validation via Pydantic

An agent cannot emit freeform text to the frontend product. Every agent output is parsed against a strict Pydantic model:

• Fields must match exact enum definitions and types.
• Missing required attributes automatically trigger a single repair pass with targeted error feedback.
• If repair fails, the orchestrator emits a structured fallback payload rather than unparsed garbage.

Circuit Breakers and Dead-Letter Replay

When external model APIs experience transient outages or 429 rate limits, naive agent systems either hang or crash. In our architectures, we wrap external calls in circuit breaker state machines:

1. Closed State: Requests execute normally. Latency and error rates are monitored.
2. Open State: If error rates cross 15% over a 60-second window, the circuit trips. Outgoing calls are immediately routed to a secondary provider (e.g. AWS Bedrock Claude fallback) without hammering the failing API.
3. Dead-Letter Queue (DLQ): Unresolvable tasks are serialized into an SQS dead-letter queue with full execution state, allowing engineers to inspect the exact prompt trace and replay without data loss.