You're designing the production retrieval system for Carlos Quispe, head of epidemiological data systems at Peru's national epidemiology center, whose epidemiologists lose critical hours searching fifteen years of bulletins, lab reports, and outbreak investigations when an outbreak is moving.
The discipline skills: an indexing pipeline that handles a living corpus -- documents added, updated, deleted, and the preliminary-vs-final version conflict; multi-tenant retrieval where access control is built in before ranking, not bolted on after; cache strategies that trade response speed against information freshness; the RAG-vs-fine-tuning-vs-long-context-vs-hybrid judgment grounded in cost, freshness, and corpus volatility; retrieval-quality evaluation that runs continuously across thousands of queries rather than as a deployment gate; cost-aware re-index decisions; monitoring that monitors itself; and the handover documentation that lets Carlos's team operate the system during an emergency without the person who built it.
The AI-direction lesson: a retrieval system is a living production system, not a search box, and AI builds the search box and stops. It implements retrieval over a static corpus, applies one chunking strategy to every document type, ranks across the whole corpus and filters access after, and writes monitoring without the layer that checks the monitoring is still alive. You design the architecture and direct AI within it; the catches that matter -- the multi-tenant leak, the silently-dead drift job -- are invisible by default and have to be designed out, not tested for after.
Your Role
Retrieval-and-monitoring production architect for the surveillance system. Sub-second response under a 10x dengue-season query spike, patient-identifier isolation, retrieval quality you can watch as a number every day, and a system Carlos's team can run at 2 a.m. without you -- these shape every decision.
The scaffolding thins again. Last time you had a fresh CLAUDE.md skeleton with empty sections to fill and Marcus Webb on call across the serving architecture. This time the senior colleague narrows to a single seam: a data-governance consultant reachable on-demand only for whether your access-control boundary holds, who won't volunteer or hand you answers. The first contact is a sparse email -- symptoms, not a spec. The constraints are not named for you; they surface through discovery with Carlos, who confirms a constraint only if you ask the right question, or through the corpus itself. You author the CLAUDE.md and its path-scoped fragments. AI is supervisory: you name the architecture, AI implements pieces, you verify that composition holds across every boundary.
What's New
Last time you designed the classical-ML production capstone for Shu-Fen Lin at PrecisionTek -- offline/online feature parity, blue-green serving, a four-agent pipeline with explicit boundary schemas. The production-architecture judgment transfers directly. The terrain does not: this is retrieval, not classical ML.
What's genuinely new: chunking is no longer a single decision but one piece of an indexing pipeline that handles a document's whole life. Multi-tenant access control as an architectural constraint built in before ranking. Cache invalidation tuned to how fast the corpus changes. The approach judgment -- RAG vs fine-tuning vs long-context vs hybrid -- made with full production experience behind it. Retrieval-quality SLOs with error budgets tied to outbreak-response timelines. Cost projected at real query volume, not from a dev run. And monitoring-of-monitoring -- the recursive layer that catches a drift job that died silently, which AI will not write on its own.
The hard parts cluster on the new terrain and stay hidden until you look for them. AI's first retrieval code will leak patient identifiers across tenants. Its first chunking will treat OCR-garbled rural reports like clean tabular bulletins. Its first cost estimate will come from the dev run. Its monitoring will have nothing watching it. The discipline is designing the system so these are impossible, not finding them after.
Tools
- Claude Code -- directing; you author the CLAUDE.md and path-scoped fragments for retrieval-specific directories
- A vector store -- you select it (e.g. ChromaDB, Qdrant, or a managed option); the selection is part of the architecture
- An embedding model and an LLM for generation -- locally served and/or API; the connectivity decision is yours, including when NOT to connect a managed service
- A monitoring and metrics stack -- you design it (e.g. scheduled evaluation jobs writing to a metrics store with a dashboard)
- Python, Git/GitHub -- familiar; carried forward
- Your accumulated skills directory -- the skills you authored on prior projects carry forward
No tool is introduced as a setup step. Every tool choice is an architecture decision you make and justify.
Materials
You receive:
- A sparse first-contact email from Carlos -- symptoms, not a brief
- The heterogeneous document corpus -- five document types (bulletins, lab reports, outbreak investigations, clinical guidelines, regional situation reports), deliberately varied quality, version-conflict pairs, and patient-identifier samples in outbreak reports
- A small starter project skeleton -- directory layout plus an empty
CLAUDE.mdfor you to fill - Your own accumulated
skills/directory from prior projects
The architecture, the discovery, the monitoring design, and all documentation are yours to produce.