Warehouse Architecture: Data Products, Quality, Governance, and Observability as One System.

Ingrid Halvorsen needs the Nordfjord warehouse structured for four consumer groups with conflicting needs — site biologists, the research team, sales and export, and regulatory.

The discipline skills: warehouse architecture as data product boundary decisions across consumers; shared-vs-separate dimension judgments as the highest-consequence schema decisions of the track; transformation engine choice per data product against actual computation shape; quality, governance, and observability designed together as one integrated system; cost forecasting as a board input; knowing what NOT to monitor as part of the architecture.

The AI-direction lesson: AI builds each data product coherently in isolation and silently introduces cross-product inconsistency — the biologist dim_site rolls up by region, the regulatory dim_site rolls up by jurisdiction, both pass their own tests, and the same site_id resolves to different regions across products. The same shape repeats across quality, governance, and observability. Decomposition has to happen at two levels at once — per data product and across data products — and the architectural coherence is what only you can hold.

Your Role

You are the warehouse architect for Nordfjord. You design the data product boundaries, the transformation strategy, the quality architecture, the governance architecture, and the observability architecture as one system. You write the architecture documents each consumer group can sign off on, build representative data products as the proof, run a cross-product reconciliation pass, and hand the integrated warehouse back to Ingrid.

The scaffolding has thinned again. There is no architecture template, no per-data-product checklist, no decomposition template, and no colleague who reviews the architecture. Ingrid hands over the consumer groups and a constraint set; everything else is yours. Your repository carries forward — dbt, BigQuery, Dagster, the streaming and CDC pipelines from last project, the path-scoped CLAUDE.md, your MCP setup, your hooks. You extend it; you do not start over.

The AI relationship sits one register higher again. You decompose the warehouse problem before AI writes any code, supply the architectural constraints each piece must respect, and verify coherence across what four consumer-domain agents produce. No colleague reviews the warehouse this time — JT Thompson covered the ingestion side; the architecture is yours. (Sigrid Aune, an analytics-engineering consultant, is reachable on demand for a narrow sanity check on whether a shared dimension stays consistent across your four data products — she volunteers nothing and reviews nothing.)

What's New

Last project you designed the ingestion architecture for Nordfjord — per-source pattern decisions across 12 sites, the schema registry as a first-class deliverable, per-source-family agents with selective MCP, JT Thompson reviewing at architecture moments.

This project the comparison axis disappears. The ingestion landscape is operational; you consume from the landing zone it built. Genuinely new: four consumer groups with conflicting needs as the central design problem. Per-consumer-domain agents — biologist, research, sales-export, regulatory — replacing per-source-family agents, with connectivity now configured for handoff. Cross-data-product reconciliation tests as the warehouse's coherence proof. A cost forecast Ingrid will use to defend the data platform budget to the board. And no colleague reviewing the work — you are the only architect.

The hard part: cross-product silent failures live at the boundaries no single agent owns. Shared dimensions drift between products; materialization strategies fit each product in isolation but burn the budget across the warehouse; tests pass inside each product while the cross-product reconciliation is unwritten; masking is correct at the consumer layer and a debug-friendly intermediate view leaks the raw vet name; a 180-day backfill fires thousands of false-positive trend alerts; every model gets a freshness sensor and alert fatigue mutes the real failures.

Tools

• dbt, Soda Core, BigQuery, Dagster, Kafka, Debezium, Confluent Schema Registry, DuckDB, PostgreSQL, MinIO, Docker Compose, GitHub Actions, Claude Code, Codex CLI, MCP — all carrying forward. The streaming and CDC pipelines from last project keep feeding the landing zone; you do not redesign them.
• PySpark — re-entering for the one data product (research-team multi-year aggregations across all 12 sites with statistical operations) where dbt + SQL would be expressively limited. The discipline is engine-per-data-product against actual computation shape, not against AI's stylistic defaults.
• BigQuery INFORMATION_SCHEMA.JOBS — the cost-forecasting baseline. The capability is new even though the warehouse is the same.

Materials

You'll receive:

• A project governance file (CLAUDE.md) — Nordfjord warehouse-phase context: client, the four consumer groups, carry-forward state from the ingestion architecture, verification targets, commit convention.
• A consumer-needs prompt sheet (consumer-needs-prompts.md) — starting questions worth asking each consumer group. Hidden constraints surface only on targeted questions.
• A landing-zone seed script (scripts/seed-landing-zone.py) — only used if your existing BigQuery from last project is not available.
• Ingrid's follow-up Slack message — in the chat at project start, in the same #data-architecture channel.

Deliberately not provided: any template for the warehouse, quality, governance, or observability architecture documents. No per-data-product checklist. No integration-test scaffolding. Your existing repository carries forward as the working platform.