You're investigating a drifting credit scoring model for an SME lending platform in Singapore -- a gradient boosted model that approves working-capital loans in under four hours, and that now has to satisfy a new regulator requirement for plain-language explanations on every decision.
The discipline skills: using SHAP to explain which features actually drive model decisions and interpreting those findings with domain judgment, diagnosing training-serving skew (the same feature computed two different ways in two different code paths), redesigning a feature layer so training and serving share one source of truth, monitoring feature distributions for drift, extending CI/CD eval gates to cover feature parity and SHAP stability, and translating feature importance into borrower-facing plain language.
The AI-direction lesson: this is the deepest form of data leakage you'll meet in classical ML -- and the one your Pipelines can't catch. The Pipeline discipline you've been building since P6 prevents leakage between rows. It does nothing about leakage between code paths. When training computes a feature one way and serving computes it another, the model you evaluated and the model you deployed are two different models wearing the same name. AI doesn't see the problem on its own -- it generates feature code with a single execution path by default, produces SHAP visualisations without interpreting whether the features make domain sense, and monitors predictions without monitoring the feature distributions that warn you before predictions go bad. You drive all three.
Your Role
Production feature engineer and explainability lead on a model that's already live. You diagnose the drift, prove the structural defect, redesign the feature layer, and deliver the explanations the regulator now requires. The model already exists. The infrastructure from P11 is yours to extend. What's new is the terrain: SHAP, training-serving skew, production feature architecture, distribution monitoring.
The scaffolding keeps its full-discovery shape from P11 but removes the familiar-discipline shield. Last time the routing problem was deliberately familiar so the infrastructure work could be dominant. This time the infrastructure is familiar and the discipline is genuinely new.
What's New
Last time you authored project memory for a routing model at Saigon Express and ran the before/after contrast that made infrastructure quality tangible.
This time you carry that infrastructure forward -- your CLAUDE.md and AGENTS.md travel with you -- and extend it with new production rules. The discipline terrain is what's new: SHAP as a diagnostic tool (not just a visualisation), training-serving skew as leakage that Pipelines can't prevent, feature distribution monitoring as early warning, eval gates that check parity and SHAP stability alongside accuracy.
The hard part is judgment. AI generates SHAP plots competently. It does not tell you whether a high-importance feature is a genuine predictor, a regulatory proxy, or a skew signal in disguise -- and getting those wrong has consequences your client's regulator cares about. The drift Mei-Ling reports has three plausible explanations. Only one is right. You work out which.
Tools
- Claude Code -- AI agent, project memory from P11 travels with you (familiar)
- Python / pandas / scikit-learn -- existing model stack (familiar)
- SHAP -- feature importance and attribution analysis (new -- the primer material walks through it)
- MLflow -- experiment tracking, MCP-connected from P10 (familiar)
- FastAPI -- existing serving endpoint (familiar)
- Docker -- existing containerised serving (familiar)
- pytest -- now extended to offline/online feature parity tests (familiar, new use)
- Git / GitHub -- version control and CI eval gates (familiar)
- Jupyter -- SHAP analysis and feature distribution work (familiar)
Materials
You receive:
- The existing credit scoring model -- Python, scikit-learn, FastAPI, Docker, already in production
- Two years of historical loan application data with outcomes
- The current feature computation code -- separate training and serving paths (the skew lives here)
- A SHAP primer covering summary plots, dependence plots, and force plots
- The MAS Section 6.3 extract describing the plain-language explanation requirement
- A sample of denied-application letters from the compliance team's existing manual process
- A feature catalogue listing every feature, its data source, and its computation path
- Mei-Ling's forwarded email chain, including her compliance officer's longer internal message
- Your own CLAUDE.md and AGENTS.md carried forward from P11, ready to extend with production feature rules