AI Agents / SaaS11-week deliverySolutions Architect

Wellows Multi-Agent AI Workflow Architecture

LLM Search Visibility Platform

How I designed the production architecture for a platform that measures whether a brand shows up in AI answers across ChatGPT, Gemini, Perplexity, and Google AI, then closes the gaps it finds through automated content generation and technical page remediation. Prototype to live business system in eleven weeks.

3 production agents80% fewer manual workflow steps<200ms vector retrieval

View the live platform →

System Architecture Blueprint
W
WellowsOrchestration Pipeline & Guardrails
3 agents active
Ingestion GatewayFastAPI NodeLlamaGuard NodeContent SafetyLangGraph RouterState OrchestratorKIVA AgentWriting AssistantOPTA AgentTechnical ExtractorCitation IntelLLM Brand MonitorQdrant IndexVector IngestionOpenAI GPT-4oPrimary LLM NodeAWS Bedrock SonnetRateLimit FallbackLLM Judge / EvaluatorConfidence > 0.85LangSmith / ArizeAsync TracerRedis Ingestion CacheAudited Responses
01Ingress
Ingestion GatewayFastAPI Node
LlamaGuard NodeContent Safety
02Orchestration
LangGraph RouterState Orchestrator
03Agents
KIVA AgentWriting Assistant
OPTA AgentTechnical Extractor
Citation IntelLLM Brand Monitor
04Retrieval & Models
Qdrant IndexVector Ingestion
OpenAI GPT-4oPrimary LLM Node
AWS Bedrock SonnetRateLimit Fallback
05Verification & Ops
LLM Judge / EvaluatorConfidence > 0.85
LangSmith / ArizeAsync Tracer
Redis Ingestion CacheAudited Responses
[SYSTEM DETAILS]: Hover any architecture node to inspect structural details, runtime constraints, and scaling tradeoffs.

The problem

The prototype worked in demos. It was not ready for production.

Wellows had a strong product hypothesis: brands are losing visibility inside LLM ecosystems and do not know it. The prototype proved demand, but the system behind it was still a set of disconnected scripts.

Citation monitoring, technical auditing, and content drafting each ran manually. None shared data, so content was written without the visibility findings that should have informed it. There was no orchestration, cost control, observability, or failure isolation. A rate limit or API timeout could break the pipeline silently.

The work was to turn the demo into a product architecture where measurement and remediation share one spine: agents with clear responsibilities, a shared retrieval layer that grounds both the audit and the content written against it, async processing, typed APIs, and production operating controls.

Operating constraints

The non-negotiable boundaries before writing code.

Prototypes ignore limits. Production architectures are defined by them. These three constraints anchored every architectural choice we made.

Latency SLA

<200ms Vector Retrieval

Brand monitoring queries demanded immediate sub-second dashboard rendering, precluding naive, synchronous multi-LLM re-ranking on the critical user path.

Tenant Cost Governance

Strict Token Spend Caps

Continuous crawling, citation audits, and remediation content at 10K+ articles a month could easily balloon API costs if agent loops ran unbounded without strict token quotas and deterministic cycle limits.

Failure Isolation

Zero Shared Mutable State

A transient timeout or hallucination in the content drafting agent (KIVA) could not be allowed to corrupt memory or interrupt ongoing technical audits (OPTA).

Architecture decisions

The decisions that determined whether the system would survive production.

These were not implementation details. They were the product boundaries that made the platform operable under real traffic, real cost, and real failure modes.

Multi-agent over monolith

Problem

A single AI pipeline for citation monitoring, technical auditing, and content drafting would share failure modes. One model hallucination or rate-limit would block everything.

Decision

Designed three specialized agents with isolated responsibilities, separate retry logic, and a shared retrieval layer. Failure in one agent does not propagate to the rest of the system.

Tradeoff accepted

More orchestration complexity upfront, but operational independence at scale. LangGraph handled the coordination contract.

Vector search as the retrieval spine

Problem

LLM outputs needed grounding in real brand data: keyword corpuses, competitive citations, and crawl artifacts, without re-indexing on every query.

Decision

Built a vector search layer with chunked ingestion pipelines. Agents query the same store while new data is ingested asynchronously.

Tradeoff accepted

Requires careful chunk sizing and embedding consistency. Paid off in sub-200ms retrieval latency at query time.

Remediation grounded in the same retrieval layer as the audit

Problem

Measuring a visibility gap is only half the product. Drafting content to close that gap without the brand corpus and citation findings produced output that contradicted the platform's own analysis.

Decision

Pointed KIVA at the same Qdrant store the monitoring agents query, so every draft is grounded in the crawl artifacts and citation history that identified the gap in the first place.

Tradeoff accepted

Remediation inherits the retrieval layer's freshness constraints, but measurement and fix stay consistent and one ingestion pipeline serves both.

FastAPI for the agent API surface

Problem

Orchestration results needed to be consumed by a frontend product team without coupling them to LangGraph internals.

Decision

Wrapped agent outputs in a clean FastAPI layer with typed response schemas. The frontend receives structured JSON instead of internal orchestration details.

Tradeoff accepted

An extra serialization layer, but essential for team separation. Agent internals can change without breaking the product contract.

AWS-native infrastructure

Problem

Managed AI wrappers abstract away control at the cost of observability and cost predictability, both non-negotiable at production scale.

Decision

Used SQS for async job queuing, ECS for agent containers, and CloudWatch for observability. Kept AI logic in code, not in proprietary platform abstractions.

Tradeoff accepted

More infrastructure ownership, but the team owns failure paths, instrumentation, and cost controls.

System architecture

How the agents connect.

AWS infrastructure boundary
Brand / user requestFastAPI entry point
LangGraph orchestratorRoutes tasks, manages retries, coordinates outputs
KIVAWriting assistant drafting grounded remediation content
OPTATechnical audits and remediation workflows
Citation IntelligenceBrand mention tracking across LLM ecosystems
Shared retrieval layerVector search and real-time ingestion pipelines
SQS

Async job queue

ECS

Agent containers

CloudWatch

Observability

OpenAI / Claude

LLM backends

No single point of failure. Each agent operates independently with isolated retry logic. A timeout in Citation Intelligence does not block KIVA from completing its task.

Failure modes & defenses

Where naive AI systems break — and how this architecture survives.

A resilient AI system is engineered around worst-case execution. These are the primary failure vectors identified during pressure-testing and the deterministic defenses built to neutralize them.

Failure Mode 01

Model Hallucination & Schema Drift

Production Impact

An LLM returning unstructured markdown or omitting required schema keys would crash the downstream frontend dashboard.

Architectural Defense

Strict Pydantic JSON schema contracts enforced on every agent step. If validation fails, LangGraph triggers an automated correction prompt with lowered temperature before routing to human fallback.

Failure Mode 02

Third-Party Rate Limits (HTTP 429)

Production Impact

Concurrent enterprise brand audits overwhelming OpenAI or Perplexity rate limits, causing pipeline abortion and data loss.

Architectural Defense

Decoupled asynchronous worker queues using AWS SQS and Celery with exponential backoff and jitter, combined with automated model failover via AWS Bedrock.

Failure Mode 03

Cascading Multi-Agent Deadlocks

Production Impact

One slow or stuck agent blocking the entire evaluation graph, leaving user requests hanging indefinitely.

Architectural Defense

Independent circuit breakers per agent. If an agent fails after 3 retry cycles, the orchestrator issues a partial result flag, saves checkpoints, and completes the remaining workflow.

Failure Mode 04

Vector Retrieval Context Poisoning

Production Impact

Outdated crawl snippets or redundant competitive brand mentions polluting the LLM context window with high noise.

Architectural Defense

Semantic deduplication, chunked ingestion pipelines with TTL expiration, and cross-encoder re-ranking ensuring only the top-5 verified citations enter the prompt.

Build timeline

Eleven weeks, prototype to production.

Week 1-2

Architecture clarity session. Mapped product goals, user flows, constraints, and the three agent contracts before implementation.

Week 3-4

Built KIVA: the writing assistant drafting remediation content against retrieved brand context, plus the vector ingestion pipeline, OpenAI integration, and production output schema.

Week 5-6

Built OPTA: technical audit agent, crawler integration, remediation output format, and LangGraph orchestration wiring.

Week 7-8

Built Citation Intelligence: multi-LLM monitoring across ChatGPT, Gemini, and Perplexity with sentiment diff logic.

Week 9-10

Production hardening: retry logic, cost controls, observability dashboards, load testing, and FastAPI contract finalization.

Week 11

Handoff: architecture documentation, agent operating playbooks, team onboarding, and zero open critical issues.

Outcomes

What the architecture delivered.

These are production numbers, not decorative metrics. Every outcome connects to an architecture decision made before handoff.

3

Production AI agents shipped

KIVA, OPTA, and Citation Intelligence, each with defined responsibilities and independent failure paths.

80%

Reduction in manual workflow steps

Measured against the pre-agent manual analysis process the client used before the platform.

<200ms

Vector retrieval latency

Achieved through chunked ingestion design and a shared retrieval layer across all three agents.

10K+

Articles generated per month

Automated remediation output, grounded against the shared Qdrant retrieval layer and processed through SQS-backed generation, review, and publishing services.

Work together

Moving AI from prototype to production?

I do not do patch jobs. If the architecture is wrong, I will tell you before another expensive layer gets built on top of it.

Book Architecture CallView all work