learntodriveai.dev/Machine Learning/Project Memory and Evaluation Workflow Encoding
Machine Learning·Project 11·5 units

Project Memory and Evaluation Workflow Encoding

**Track:** Machine Learning

§ Brief

You're building AI development infrastructure for a freight forwarding company in Ho Chi Minh City -- authoring the project memory files (CLAUDE.md and AGENTS.md) that encode 23 hard-won routing rules so every AI session starts with the right constraints, then verifying that the infrastructure actually changes AI's output quality.

The discipline skills: writing project memory entries that produce specific, verifiable AI behavior (not vague "best practices"), authoring dual-format infrastructure (CLAUDE.md for Claude Code, AGENTS.md for cross-platform portability), encoding evaluation workflow standards as persistent rules, and designing context briefs that front-load session intent.

The AI-direction lesson: this is the first time your directing act persists beyond a single session. Every project so far, you've re-stated the same rules: don't preprocess before splitting, compute per-slice metrics (per-class for classification, per-corridor for this project's regression target), log experiments to MLflow. Now those rules become infrastructure. The difference between "follow best practices" and "never apply StandardScaler before train_test_split -- use sklearn.pipeline.Pipeline" is the whole project. The first produces AI's default patterns. The second produces something you can verify. You'll run a before/after contrast -- one session with project memory loaded, one cold start -- and see the difference in output quality immediately. Infrastructure determines outcomes, and you're now the one building it.

Your Role

Not a model this time -- infrastructure for how AI works on the project. You author the governance files that shape every future AI session. No guides, no templates, no structured brief. A voicemail. Everything else comes from conversation and exploration.

What's New

Last time you connected Claude to MLflow via MCP and built cost monitoring for a predictive maintenance system.

This time the focus shifts from what AI does to what AI knows before it starts. You author the infrastructure files that shape every future AI session on a project. The rules that you've been manually enforcing -- never preprocess before splitting, always compute per-slice metrics (per-class for classification, per-corridor for this project's regression target) -- become persistent constraints that AI follows from the first prompt.

The hard part is writing constraints that are specific enough to produce consistent behavior. "Follow best practices" produces AI's default patterns. "Never apply StandardScaler before train_test_split -- use sklearn.pipeline.Pipeline" produces something you can verify. The difference between the two is the whole project.

Tools

  • Claude Code -- AI agent, now authoring project memory for it (new aspect)
  • Python -- scripting, data processing (familiar)
  • scikit-learn -- modeling, evaluation (familiar)
  • MLflow -- experiment tracking, MCP-connected (familiar from P10)
  • Git / GitHub -- version control (familiar)

Materials

You receive:

  • An existing route optimization model (Python, scikit-learn) -- working code with known issues
  • Historical shipment data -- 3,600 records across 18 months
  • Port schedules -- operating hours, congestion windows, vessel schedules for 5 Vietnamese ports
  • Duc's team's routing rules document -- 23 rules in informal language, the raw material for your project memory