Evaluating Infrastructure Through a Forced Platform Migration

Oyungerel Bat-Erdene runs the analytics division at Mongolia's national tourism agency, and a government mandate is forcing her team off the platform their entire analytical operation runs on — she needs to know what survives the move.

The discipline skills, all of them, run again under one new pressure. You don't build a metric layer — you take a working one and prove whether it transfers intact to a platform that consumes it differently. You don't design dashboards from a brief — you rebuild the ones that matter on a new platform and prove each one returns the same numbers as the original. You don't analyze fresh data — you inventory a working estate, sort every piece of it by how portable it is, and predict its fate before the move tests your prediction. The deliverable is an evaluation, not a migration: a durability verdict per artifact, an equivalence memo, a transition plan that keeps the division running during the switch, and a framework the agency can run again the next time the ground moves.

The AI-direction lesson is the one this track has been building toward from the first infrastructure project. AI rebuilds a dashboard so it looks right and reports success — it rarely checks that the numbers actually match. AI maps a tool change feature by feature ("can the new platform do X? yes") and never asks the real question: should this exist at all, and is it portable because it was designed that way or only by accident? AI proposes connecting the new platform to everything, because connection is capability — it does not weigh a legal constraint that makes some connections forbidden. Portability is not something AI discovers after the fact. It is a property you design in, and the judgment about what survives a forced change is yours, not AI's.

Your Role

You're the analytical-infrastructure evaluator advising a deputy director through a platform change she did not choose and cannot refuse. You're delivering a structured evaluation she can act on — what is lost, what is gained, what survives, and how to switch without leaving her division blind during the transition. She wants frameworks, not opinions.

What's different is that the infrastructure under examination is good. Every project so far asked you to build well, or — last time — to audit someone who hadn't. This one hands you working, well-built infrastructure and asks the inverse: was it built durably enough to survive a move it was never designed for? There is a senior colleague reachable in chat, and he steps in once at the moment that reframes the whole project. The judgment in the evaluation is still yours.

What's New

Last time you inherited an untrustworthy estate from a departed analyst — you ran a six-stage audit, graded your confidence on every finding, and built the durable collaboration layer that workplace never had.

This time the estate works, and nobody has left. The new ground is predicting what survives a forced platform change and then proving it: equivalence verification across a platform boundary, connectivity designed under a hard legal constraint, and a reusable evaluation framework authored from a blank workspace — there is no method to adapt and no skeleton to fill.

The hard part is restraint and judgment, again, but pointed differently. The instinct will be to migrate everything that exists. The more valuable move is deciding what should exist at all — and proving, number by number, that what you rebuilt is genuinely equivalent and not just visually convincing. "Looks right" is the trap this project is built to catch.

Tools

• Established and carried forward: Claude Code, Codex CLI (cross-AI equivalence review of every rebuilt dashboard), DuckDB with the read-only MCP server, dbt as the metric layer under test, Python, DuckDB SQL, Git and GitHub with conventional commits, and the carried-forward skills library.
• Metabase as the legacy platform — read-only via its REST API to extract dashboard definitions and usage logs. Familiar, here as the source you are moving off.
• Apache Superset as the target platform — brought up locally via Docker as a stand-in for the government cloud, so you can rebuild dashboards and prove equivalence without real cloud access. Familiar from earlier work; the unit that needs it walks through bringing it up.
• A new tool-transition-evaluation skill — you author it in the final unit, encoding the evaluation framework so the agency can run it again for the next platform change.

No new tool is introduced. What is new is using familiar tools across a forced transition.

Materials

• The first-contact Slack message — Oyungerel's three terse lines, in the project chat. The discovery is the thread, not the message.
• The working estate under current-estate/ — this runs; it is not a dead inheritance. The 45 Metabase dashboards as exported JSON plus per-card SQL and usage-logs.csv; the dbt project (metric models, tests, docs); the existing MCP connectivity and infrastructure/connectivity.md; the carried-forward CLAUDE.md/AGENTS.md infrastructure; and the analytical database current-estate/data/mtb.duckdb, which carries a tourist_pii schema that, by law, cannot leave Mongolian servers.
• A target environment — a local Apache Superset instance via Docker (docker compose up from superset/), pointed at a local copy of the analytical database.
• Short reference context at reference/ — Mongolia tourism domain context, a factual Superset-vs-Metabase capability comparison (facts only, no method), and the read-only Metabase REST API reference.
• Starter layout — an empty evaluation/ directory with no skeletons, and the carried-forward AI infrastructure under current-estate/ai-infrastructure/ (CLAUDE.md, AGENTS.md, the skills library).

Download the materials zip (https://learntodriveai.dev/materials/analytics/p-22/materials.zip) and unzip it to ~/dev/analytics/p-22.