You're building a multi-source quality assessment and metric hierarchy for a sustainable packaging manufacturer outside Cork, Ireland — integrating data from four sources in three working file formats and designing metrics that decompose into diagnostic components.
The discipline skills: format-specific data handling (Parquet, JSON, CSV, and CSV-from-PDF each carry different risks), metric hierarchy design (OEE decomposes into availability, performance, and quality — changing one cascades upward), structuring findings as a professional argument rather than a list, and assessing data freshness as a quality dimension.
The AI-direction lesson: AI makes different errors on different data formats. It flattens nested JSON in ways that lose data. It infers schema types from the first few rows and misses inconsistencies later. It generates metric hierarchies that are mathematically consistent but don't match how the business makes decisions. Cross-model review enters here — directing a second AI to review the first AI's analysis surfaces errors that the original session normalized. A second perspective catches what familiarity hides.
Your Role
You're pulling together four sources delivered in three working file formats. The data arrives as Parquet, JSON, and CSV (with one CSV transcribed from PDF lab reports). Each format — and each provenance — carries different risks. What worked for single-source CSV analysis needs to expand.
The metrics you build are not flat definitions. Production efficiency decomposes into components — availability, speed, quality — where changing one definition cascades to everything above it. You design the hierarchy, document the dependencies, and deliver an argument, not a list.
What's New
Last time, you framed Wei's business questions as testable hypotheses, chose the right statistical test for binary outcome data, and reported confidence intervals instead of point estimates. You specified constraints before computation and crossed from descriptive to inferential statistics.
This time, the data itself is the new challenge. Four formats, each with different risks. Metrics that exist in hierarchies — one number that decomposes into three, where changing one changes the rest. Your findings need to be structured as a professional argument, not a chronological list. And you will use a second AI to review your first AI's analysis — cross-model verification.
The hard part is not any single source. It is connecting four sources with different formats, freshness, and granularity into a coherent analysis, and then communicating that analysis as a structured argument.
Tools
- Python 3.11+ (via Miniconda, "analytics" environment)
- DuckDB
- Polars (new — for scale-appropriate data handling)
- Jupyter Notebook
- pandas
- scipy.stats, statsmodels
- matplotlib / seaborn
- Metabase (via Docker — continuing from previous projects)
- Docker
- Claude Code
- Git / GitHub
Materials
- Production logs — Parquet file from the manufacturing execution system, about 50,000 rows of shift-level production data.
- Sales data — JSON export from the e-commerce API with nested customer objects and line item arrays. About 12,000 orders.
- Procurement records — CSV from finance spreadsheets with monthly supplier costs, lead times, and quality grades.
- Quality results — CSV derived from PDF lab reports with batch-level test results, pass/fail, and moisture readings.
- Data dictionary — describes all four sources, their formats, date ranges, and freshness status.
- Metric hierarchy template — introduces OEE decomposition, leading/lagging indicators, and hierarchy documentation.
- CLAUDE.md — project governance file with client context, work breakdown, and verification targets.