You're evaluating a loyalty program for a pharmacy group in Muscat, Oman — assessing whether the experiment design supports causal claims before computing any statistics, and authoring your first project memory files.
The discipline skills: experiment validity assessment (selection bias, novelty effects, confounding variables), separating what a program caused from what was already true, communicating validity caveats to a board that expects good news, and authoring persistent AI infrastructure (CLAUDE.md and AGENTS.md with metric definitions and data quality rules).
The AI-direction lesson: infrastructure as directing. You author project memory files for the first time — writing metric definitions and data quality rules that load at every AI session start. The before-and-after difference is immediate: AI with your definitions uses them consistently; AI without them reinvents the definitions every session, differently each time. The quality of what you write into project memory determines the quality of every AI session that follows. Directing AI is no longer just about what you type in the moment — it's about what you build for every future moment.
Your Role
You're producing a board-ready analysis of the loyalty program's effectiveness. Before you compute anything, you assess whether the "experiment" is valid. A loyalty program where customers opt in is not a randomized trial. Your job is to figure out what the data can honestly tell the client, present that clearly, and give the board a number they can act on.
You also build your first project memory files. Instead of starting every AI session cold, you write metric definitions and data quality rules once and AI loads them at session start.
What's New
Last time, you ran an A/B test analysis for a booking page, connected AI to a database via MCP, and handled a deliberately ambiguous brief. You computed p-values, confidence intervals, and effect sizes.
This time, you assess validity before computation. The same statistical methods apply, but first you check whether the experiment design supports the conclusions the statistics would suggest. AI will compute a p-value on invalid data without hesitating. Your judgment about whether to trust the experimental setup determines whether the numbers mean anything.
You'll also author your first CLAUDE.md and AGENTS.md files -- persistent metric definitions and data quality rules that load at session start. You've used pre-built governance files before. Now you write your own. The before-and-after difference -- AI with your definitions versus AI making up its own -- is the clearest evidence yet that infrastructure quality determines analytical output quality.
Tools
- Claude Code (with DuckDB MCP active from last project)
- DuckDB (local database, MCP-connected)
- Python (pandas, scipy.stats, matplotlib/seaborn)
- Jupyter notebooks
- Git and GitHub
- CLAUDE.md and AGENTS.md (project memory -- authored for the first time)
Materials
- Transaction data -- 12 months of pharmacy transactions across all eight stores. Six months before the loyalty program launched, six months after.
- Data dictionary -- column definitions, enrollment notes, known data limitations.
- CLAUDE.md skeleton -- a minimal project governance file with the project context and one example metric definition. You'll author the full metric definitions and data quality rules yourself.