learntodriveai.dev/Data Engineering/Schema Contracts and Configuration Tracking
Data Engineering·Project 06·8 units

Schema Contracts and Configuration Tracking

**Track:** Data Engineering

§ Brief

You're building data infrastructure for a wind farm analytics company in Esbjerg, Denmark — designing schemas that track turbine configurations over time, formalizing metric definitions in a semantic layer, and implementing PII masking across every output surface.

The discipline skills: choosing slowly-changing dimension strategies (Type 1 vs Type 2), defining metrics in MetricFlow so every consumer gets the same calculation, enforcing schema contracts in dbt, and classifying and masking PII in maintenance logs.

The AI-direction lesson: context curation. What you include in an AI session — and what you leave out — changes the output. Including the turbine data dictionary plus the SCD design decisions produces categorically better dbt models than a bare prompt. Including everything dilutes AI's attention. The skill is deciding what's relevant to this specific task. And when AI starts contradicting conventions it followed earlier or generates models inconsistent with established patterns, that's context degradation — the signal to consolidate and start a fresh session with the right information loaded. Context management becomes a deliberate practice here, not something that just happens.

Your Role

You're building the infrastructure that tracks turbine configurations over time and standardizes metric calculations across all 14 farms. Governance moves from awareness to enforcement — PII masking verified across marts, staging, docs, and logs. The pipeline spec and CLAUDE.md come as templates for you to populate.

What's New

Last time you built Roberto's quality analysis pipeline with complex transformations — window functions, Jinja macros, Soda Core, CI/CD quality gates, Dagster freshness policies. The transformations were the hard part.

This time the schema design is the hard part. You'll decide how the data warehouse handles change over time — which dimensions preserve their full history and which simply overwrite. AI will have strong opinions about this. You'll need to evaluate them against what Katrine actually needs.

MetricFlow enters as a semantic layer — defining metrics once so every consumer gets the same calculation. A valid formula and a correct formula are not the same thing — the difference shows up when Katrine's clients compare reports.

Governance moves from the question planted in P5 ("who is this data about?") to enforcement. Technician names in maintenance logs are PII. Masking them in the output tables is necessary but not sufficient. PII leaks through surfaces you won't expect.

Tools

  • dbt Core with DuckDB adapter — transformation framework, plus MetricFlow semantic layer and model contracts (new this project)
  • DuckDB — local analytical database
  • Soda Core — quality monitoring
  • Dagster — orchestration with freshness policies
  • GitHub Actions — CI/CD quality gates
  • Claude Code — AI directing tool, with context curation as deliberate practice (new this project)
  • Git / GitHub — version control

Materials

You'll receive:

  • SCADA data — a 210-row sample for orientation and a 14,872-row full dataset covering 20 turbines across 6 farms over 6 months
  • Component change log — 49 records tracking gearbox replacements, blade upgrades, software updates, and generator changes (manually maintained, not always complete)
  • Turbine mapping table — cross-referencing farm-assigned IDs and manufacturer serial numbers across all farms (some entries out of date)
  • Maintenance logs — 295 records with technician names, maintenance types, and durations
  • Pipeline spec template — empty structure for you to fill from Katrine's requirements
  • SCD design template — explains the design decision you'll make about how the schema handles change
  • CLAUDE.md template — project governance file for you to populate
  • MetricFlow guide — how to define metrics in the semantic layer
  • PII classification checklist — framework for identifying and masking personal data