You're building the next version of a churn prediction system for the same Lagos-based mobile operator -- this time with a prepaid segment the first model missed and documentation the board can review.
The discipline skills: writing a PRD, designing evaluation criteria before training, choosing metrics that match the business problem, handling class imbalance with encoding and imputation decisions you justify, tracking experiments in MLflow, and evaluating per segment.
The AI-direction lesson: last time, every decision was made for you -- the algorithm, the metric, the threshold. This time you make the decisions, and AI will make them too if you don't. AI defaults to accuracy on imbalanced data even when you know better. It applies LabelEncoder to nominal categories, imposing a false ordinal relationship. It jumps to complex models without establishing a baseline. The new skill is checking AI's choices, not just its code. Self-review enters here: asking AI to "list every transformation applied before the split" produces findings. Asking "does this look right?" produces reassurance.
Your Role
You run the full artifact creation pipeline: requirements document, evaluation design, preprocessing with documented decisions, experiment tracking in MLflow, per-segment evaluation, serving, and board-facing documentation.
You still direct Claude Code through the implementation. What changes is that the decisions are yours. P1 told you which metric to use and what threshold to hit. This time you decide.
What's New
Last time everything was provided: the algorithm, the evaluation criteria, the preprocessing instructions. You directed AI through a defined plan.
This time you make the plan. You choose the metrics. You decide how to handle encoding and imputation based on what the data looks like. You write the PRD. You design the evaluation before training starts.
The hard part is not the model. It's the decisions that come before the model — and being able to explain why you made them.
Tools
- Python / pandas — data loading, profiling, preprocessing
- scikit-learn — preprocessing, training, evaluation
- MLflow — experiment tracking (first systematic use)
- Jupyter — notebook workflow
- FastAPI / uvicorn — model serving
- Claude Code — AI direction
- Git / GitHub — version control
- curl — API testing
Materials
You receive:
- Emeka's follow-up email describing the prepaid gap and board needs
- An updated subscriber dataset (same format, three more months)
- A data dictionary for the updated dataset
- A PRD template to structure your requirements document
- A ticket breakdown for the full project
- A project governance file (CLAUDE.md)
The brief and tickets give you structure. The metrics, preprocessing choices, and evaluation design are yours.