learntodriveai.dev/Web Development/What Transfers When the Tool Changes: A Portable Evaluation Framework
Web Development·Project 23·5 units

What Transfers When the Tool Changes: A Portable Evaluation Framework.

Quick-reference identity file. Not student-facing content.

§ Brief

Brunei's Digital Services Division runs the platform 200,000 citizens use for permits, business licensing, and public services. After eighteen months building their AI development setup on one tool, a government IT committee has told them to prove they are not locked into a single vendor — and Haji Aziz, the division director, needs an honest evaluation of what would actually transfer to another tool, what it would cost, and a method his team can reuse the next time the landscape shifts.

What you're practicing: classifying a team's AI infrastructure by how portable it is, moving an MCP server configuration into a tool you have never used, testing skills and hooks for what reimplements cleanly versus what must be rebuilt, assessing migration risk against a live system that cannot go down, and building a tool-agnostic evaluation framework — plus a two-audience presentation for a government committee.

The loop is different here because the thing being evaluated is the AI substrate itself. AI cannot tell you from experience what transfers to a tool it did not configure — it pattern-matches portability claims and is confidently wrong. It reaches for the most complete migration when the answer might be "move only the portable layer" or "stay multi-tool" or "not yet." You hold the cross-tool comparison AI cannot make about itself, and every portability claim is a claim until you run it in the other tool.

Your Role

You are the independent technical evaluator. You produce a recommendation and a reusable method — not a system, and not a migration. The judgment that carries this engagement is restraint: knowing what genuinely needs to move, what stays multi-tool, and when the right answer is "do not migrate this quarter."

This is thinner than P22. There you still had a complete running system to work on and decision-log templates to fill. Here there is no system to fix and no template. The team's current AI infrastructure is the thing you evaluate; the alternative tools' public docs are a research surface, not a guide. You design the evaluation framework, the risk assessment, the validation method, and the committee presentation from nothing.

AI does the legwork on every layer except the comparison judgment — it reads documentation, drafts classifications, proposes migration plans. You decide what is load-bearing, you reject the migrate-everything reflex, and you verify its portability claims by running them, because "it was fine in the old tool" is not evidence about a different one.

What's New

Last time, you inherited an undocumented four-year-old system, recovered its architecture and threat model from running code, and handed it over with a packet a fresh AI session could verify operationally. Everything you verified was something already built.

Now the object of evaluation is the tool the whole track was taught on. You have not assessed AI development tools you have never used, built a framework rather than a one-time answer, run a parallel-tool validation as real evidence, or weighed a tooling change against a platform that serves citizens with zero acceptable downtime.

The hard part is honest: some of what looks portable transfers almost untouched, and some of what looks safe to move is expensive to rebuild. Reading that gap correctly — and resisting AI's pull toward migrating everything — is the work.

Tools

No new tool category. Every tool here is used to evaluate, not to build.

  • VS Code and Claude Code — the team's incumbent AI development tool, and the system under evaluation.
  • Codex CLI — the alternative tool you install and use hands-on for the portability comparison. Introduced in Unit 2.
  • MCP (Model Context Protocol) — the open standard whose portability you test directly by moving a PostgreSQL MCP server configuration between tools.
  • AGENTS.md / CLAUDE.md / SKILL.md — the cross-platform infrastructure formats you classify and attempt to port.
  • Gemini CLI and Cursor — additional reference tools you research from documentation only, to check the framework is general.
  • Git and GitHub — the engagement repo where the framework, the assessment, and the presentation live.

Materials

  • CLAUDE.md — the engagement governance file: the client, the mandate, the no-disruption constraint, the restraint discipline, the verification targets, the commit convention.
  • digital-services-infra/ — the team's eighteen months of AI infrastructure: the project memory files, the government-workflow skills, the compliance hooks, the MCP server configuration, and the AGENTS.md/CLAUDE.md/SKILL.md set. This is what you classify and attempt to port. It connects against a non-production seeded stand-in — never production citizen data.
  • references/ — short references, not a framework: the MCP specification and a cross-tool quick reference, a note on reading unfamiliar tool documentation, and a migration-risk vocabulary note.

The brief is the meeting transcript with Haji Aziz and the mandate. The durability model and the framework are yours to build.