Diagnosing an Inherited Pipeline

Lucia Tembe runs freight operations analytics for CFM, Mozambique's state railway. Her data engineer resigned two months ago and left no documentation. The daily freight reports still run and have never failed — but the Maputo corridor dashboard says 790,000 tonnes for last month and the corridor manager counted 850,000 in the field. You're inheriting that pipeline to find out why.

The discipline skills: tracing data from source to report through a system you did not build; reconciling source and target to locate where numbers diverge; reading a git history to reconstruct decisions nobody wrote down; auditing a test suite you didn't write; fixing a root cause rather than the symptom and backfilling the history the fix exposes; auditing who can see what. Every skill you have built, applied from the opposite direction — not designing the pipeline, but proving where someone else's went wrong.

The AI-direction lesson: reading code tells you what a pipeline should do; only querying the warehouse tells you what it does. AI reads the inherited code fast and explains it with full confidence — including the part that's broken, which it will rationalize as a deliberate business rule. Treat every account AI gives you as a hypothesis until the data or the git history confirms it. Where they disagree, the data wins.

Your Role

You are the maintenance engineer. By the end you will have a documented understanding of how this pipeline actually works, a proven root cause, a root-cause fix with the corrupted history backfilled, a test suite that would now catch the failure, an access audit, and a handover that breaks the silence the predecessor left.

The scaffolding is gone. There is no brief — the symptom is the brief. No answer key, no defect list, no checklist. The absence of documentation is the condition of the work, not an oversight. You impose the investigation structure entirely from your own judgment.

The AI relationship is investigator-in-charge. You direct AI to map an unfamiliar codebase and trace data flow at speed, while you hold the picture AI cannot and verify every claim against the warehouse itself.

What's New

Last project you evaluated a tool landscape change for PayNow Armenia — you wrote a three-layer durability framework, classified 14 configs against it under a vendor EOL deadline, and shipped a migration with a parity check encoded as a fail-loud hook. Those configs were a classification target you understood by construction.

This time there is nothing to construct. You inherit a running production system you did not design, with no documentation, and a number that does not match reality. Genuinely new: data-reconciliation as the source of truth instead of code-reading; diagnosing under live business pressure where the board reviews the report monthly; distinguishing a root-cause fix from a faster symptom adjustment; designing test coverage for a defect that already exists; and producing the handover infrastructure the predecessor never left.

The hard part is the category of the work: a system that runs every day, passes every test, and is consistently wrong. "Does it run?" was never the question — and you are about to find that this reframes every verification habit you built. The truth is in the data, and you have to go get it yourself.

Tools

No new tools. Every tool here is one you already know, encountered from the maintenance angle.

• dbt (dbt-duckdb adapter) — the inherited transformation layer; you read and audit the models, then fix one.
• DuckDB — the inherited warehouse; you query source and target tables to reconcile.
• Dagster — the inherited orchestrator; you read run history and asset lineage to understand the schedule and run a scoped backfill.
• Git — the inherited repository; you read commit history to reconstruct the predecessor's decisions, then commit the fix and the documentation.
• dbt tests — the inherited quality layer; you audit coverage and add the business-logic test that was missing.
• Claude Code — directing the investigation and reading the unfamiliar codebase; you design the context and decomposition strategy.

Materials

You'll receive:

• The handover document (handover-document.md) — the only written input from Lucia. Symptoms only: the 790,000 versus 850,000 figure, consistently low, runs daily without errors, all tests pass, no documentation, read access arranged. It does not describe how the pipeline works.
• The engagement context file (CLAUDE.md) — the context you use to direct the investigation: client, symptom, the investigation discipline, verification targets, commit convention. This is the engagement's context, not the inherited repo's project memory — that absence is part of the work.
• The inherited pipeline (freight-pipeline.bundle) — a git bundle of the predecessor's repository, exactly as they left it. In Unit 1 you clone it (git clone freight-pipeline.bundle freight-pipeline, then cd freight-pipeline) and work in the clone. Inside: a dbt project, a Dagster orchestration, a DuckDB warehouse with source and target tables, a quality test suite, an RBAC configuration, and the full git history. It runs. It is not documented. There is no CLAUDE.md, AGENTS.md, or README.md at its root — you build the handover infrastructure in the final unit.

There is no investigation log — you start one in Unit 1 and it becomes the spine of the handover.