Faisal Al-Harbi runs the Data and Analytics Division at Al-Jubail Chemical Industries, a three-plant petrochemical complex in Saudi Arabia. His CEO has mandated a "single source of truth" for operational data across the plants. You're designing and building that platform.
The discipline skills: turning a one-paragraph mandate into a defensible architecture; selecting and profiling raw sources across three plants whose systems do not match; designing data product boundaries and a layered transformation architecture; building ingestion across incompatible exports; designing a right-sized quality architecture and the observability that runs alongside it; designing governance controls nobody asked for in those terms; and standing up the AI infrastructure the build runs on. Every column you have climbed across twenty-two projects, exercised together on one platform.
The AI-direction lesson: AI builds each component competently — each extraction, each model, each control is correct on its own. What it cannot do is hold the whole. Given "build the platform," it produces a uniform pipeline that treats every source and every stakeholder the same, builds the happy path, and reports the work as done when it runs — not when the numbers reconcile. It will also reach for streaming when batch is enough, because streaming looks more sophisticated. You decompose the platform into pieces AI can build, specify the per-source and per-consumer constraints up front, and verify that the parts compose into something true. That coherence is yours to carry; AI structurally cannot.
Your Role
You are the platform architect. By the end you will have a written problem statement, a defended architecture, a built and deployed multi-source platform with quality and observability designed in, a governance system, and a handover that lets another engineer operate it.
The scaffolding is gone entirely. There is no brief, no spec, no source inventory, no data dictionary. The mandate is the input. The project memory file starts almost empty — you build it out. Discovering what data exists, what the stakeholders actually need, and what is regulated or confidential is itself the architectural work, not a preamble to it.
The AI relationship is architect-directing-builder. You decompose, specify constraints, and verify composition. AI moves fast on the pieces. The judgment about how the pieces fit, and whether the platform is right rather than merely running, stays with you.
What's New
Last project you inherited Lucia Tembe's running freight pipeline at CFM and diagnosed why it was silently wrong — no brief, no defect list, data-reconciliation as the truth source. That was investigation: the system existed and your job was to understand someone else's decisions.
This inverts it. There is no system to inherit. There is a problem and a client who cannot specify it. You make every architectural decision yourself — sources, schema boundaries, ingestion pattern, transformation architecture, quality strategy, observability, governance, AI infrastructure — defend each one, and build the platform by directing AI. Then you carry it through deployment and at least one iteration cycle, because the project does not end at "it runs."
The hard part is the category of the work. A first version of any real platform reveals problems that were always in the data but invisible, and stakeholders change the requirements after you have shipped. You designed the observability that has to surface the first; you have to judge what to do about the second without rebuilding what works. Holding the coherence of a platform this size, from a sentence, is the whole test.
Tools
No new tools. Every tool here is one you already know, now composed into a single lifecycle you architect.
- dbt — the transformation architecture: staging, intermediate, and mart layers across the five data domains.
- DuckDB or BigQuery — the warehouse. You choose the engine and defend the choice. The source data loads locally with no cloud credentials; BigQuery is available if you can justify it.
- Dagster — orchestration and observability: asset lineage, schedules, and the monitoring that detects the problem the first version reveals.
- A streaming/CDC path — introduced only if the iteration cycle justifies it, when a real-time requirement lands on a batch-designed platform.
- Soda Core or dbt tests — the quality architecture you design and right-size.
- Git — the project repository; commit history is the decision record. Final push and README.
- Claude Code — directing the whole build. You design the context, connectivity, decomposition, and AI infrastructure strategy.
Materials
This is the scaffolding floor. You receive almost nothing authored:
- The raw plant source data (
source-data/) — what the three plants actually hold: production, quality-lab, logistics, and safety extracts. Plants 1 and 2 share a schema. Plant 3 is a different system — different column names, a different file format. There is no data dictionary, no README, no manifest. You discover the shape by profiling it, the way a real platform architect receives data: as messy plant exports. - The engagement context file (
CLAUDE.md) — a near-empty stub: client, the one-line mandate, the tech stack, the commit convention, and an explicit note that the data dictionary, the data product boundaries, the governance rules, and the verification targets are yours to build into this file across the first three units. Its emptiness is deliberate.
The mandate arrives in chat from Faisal. Everything else — the architecture, the dbt project, the Dagster project, the quality config, the handover — is what you build.