Retrieval silently degrades
The model can still produce convincing prose even when the right evidence stops reaching the context window.
Engineering reference / Production RAG
From RAG demo to measurable production architecture.
A public reference implementation showing how I structure retrieval-augmented generation when correctness, regression detection, observability, cost, and provider change all matter—not just whether a demo returns a plausible answer.
FastAPIHybrid retrievalRerankingRecall@5 + MRRCI quality gate
The engineering problem
A useful answer is not enough evidence that a RAG system works.
Prototype RAG stacks often collapse retrieval, ranking, generation, and provider code into one path. That makes failures difficult to diagnose and quality difficult to measure. This reference architecture separates those responsibilities and makes retrieval quality part of the delivery pipeline.
Provider logic leaks everywhere
Embedding or generation changes become expensive when application code depends directly on a specific SDK.
No release-time quality signal
Without labelled queries and thresholds, a code change can reduce retrieval quality while every unit test remains green.
Architecture decisions
Boundaries first. Models second.
Hybrid retrieval
Combine lexical and semantic signals instead of betting retrieval quality on one technique. The boundary stays replaceable so retrieval can evolve without rewriting the API.
Explicit reranking
Keep candidate retrieval separate from final ranking. This makes ranking behavior testable and creates a clean insertion point for stronger rerankers later.
Grounded generation
Return answers with source citations and abstain when retrieved evidence is insufficient rather than treating fluent output as proof of correctness.
Provider boundaries
Retriever, reranker, embedder, and generator are explicit ports. Local deterministic behavior remains the default; OpenAI generation is optional rather than embedded into the architecture.
POST /documents → deterministic chunking → retrieval index POST /query → hybrid retrieval → reranking → grounded generation / abstention → citations evaluation dataset → Recall@5 + MRR → thresholds → CI pass / fail
Production controls
The architecture exposes evidence, not just output.
The public repository is intentionally small enough to understand, but it includes the controls I expect around a production RAG path.
A labelled evaluation dataset measures whether relevant evidence is retrieved and how highly it ranks.
The evaluation runner exits non-zero when configured quality thresholds fail, so retrieval regressions can block a pull request.
Requests expose trace identifiers and usage metadata, with token and cost accounting at the generation boundary.
External generation is wrapped in explicit timeout and bounded retry/backoff policy rather than hidden provider behavior.
Verification
Quality becomes a software delivery concern.
The repository contains a labelled evaluation dataset and a CLI evaluation runner. Recall@5 checks whether relevant evidence appears in the top five results; MRR rewards ranking the first relevant result higher. Configured thresholds turn those measurements into a regression gate that can fail CI.
Production AI architecture
Building RAG that has to survive production?
I work on the layer between a promising AI prototype and an operable system: architecture boundaries, retrieval quality, evaluation, observability, reliability, security, and cost controls.