You're designing a metric governance framework, a data acquisition strategy across six regions with three different collection systems, and a multi-stakeholder analysis serving three donors for Selam Tekle, the Director of Monitoring and Evaluation at an Ethiopian agricultural cooperative federation.
The discipline skills: metric governance as organizational work where definitions are 20% of the framework and ownership, change process, version history, transition plan, training materials, and AI access policy are the other 80%; a data acquisition strategy as a deliverable across heterogeneous regional systems (KoBoToolbox, paper-digitization, custom Access, and a central database) with multi-question fitness trade-offs across 15 indicators; a three-tier verification architecture designed before any analysis begins; a multi-stakeholder analysis that serves USAID, the EU, and DFID from one coherent base without contradicting itself; historical reconciliation across five years of reports under old definitions; and "non-reportable" as a professional category for indicators where the baseline data isn't there.
The AI-direction lesson: AI generates governance frameworks that are structurally complete and organizationally naive — a Metrics Committee with majority vote when the board's political resistance would weaponize it. AI conflates household-level and individual-level income data, estimates rather than flagging an indicator as non-reportable, and writes a 40-page training manual when a two-page card is what regional data officers can use. You design the framework, the acquisition strategy, the verification architecture, and the analytical approach — and you encode the directing discipline as infrastructure with a governance-framework-review skill that catches AI's organizationally-naive defaults per draft section.
Your Role
You're the M&E consultant Selam reached out to after USAID flagged inconsistencies in last quarter's impact report and she could not explain them on the call. The deliverables are an organizational governance framework, a data acquisition strategy, a verification architecture, a multi-stakeholder analysis with three donor-specific reports, historical reconciliation across five years, six regional training cards, a limitation report at federation scale, and a governance-framework-review skill that lives on after the project closes.
The infrastructure floor runs in the background. What's new is the scale of judgment: three surfaces of analytical work reach the top of the track at the same time. You're naming one owner per indicator, deciding what's acquirable from existing collection and what would need new collection, designing verification proportionate to risk before any analysis runs, and serving three donors' competing questions from one base without letting their definitions contradict each other.
What's New
Last time you delivered three audience-driven dashboards across Tableau, Metabase, and Superset for Sophie Whitcombe's boutique hotel group, with the dashboard-design-review skill catching audience-blindness, the cross-platform metric verification spec, the cross-dialect SQL audit, and the maintenance-strategy document.
This time the first contact is a formal memo in donor-reporting register. It names five requested deliverables. The political reality, the regional-system heterogeneity, and the donor-specific conflicting requirements surface through a scheduled 90-minute video-call discovery and focused email exchanges. The audience set scales from within-organization to across-institutions — three donors with competing indicator definitions, six regional offices with three different collection systems, the federation's board with political resistance to standardization, and the regional data officers as a training audience.
The hard part is the moment AI's first-draft governance framework arrives structurally complete — roles, responsibilities, review cadence, approval flow — and you have to recognize it as organizationally naive. The Metrics Committee with majority vote would empower the board's resistance. You catch each failure with the governance-framework-review skill before the framework ships.
Tools
- Established and carried forward: Claude Code, Codex CLI, BigQuery and DuckDB (with their MCP servers), dbt Core, Metabase, Tableau Desktop, Apache Superset (all three used lightly), Python with pandas and scipy, Jupyter, Git and GitHub with
pre-commit, the rootCLAUDE.mdandAGENTS.md, theexperiments/path-scoped rules, theinfrastructure/connectivity.mdartifact, theprofile-datasetanddashboard-design-reviewskills, and the auto-memory from P18 (curated at project start). governance-framework-reviewskill — new; SKILL.md format with a scoped description encoding the organizational-completeness rubric (ownership, change process with executive sponsorship, version history, transition plan, training materials, AI access policy).governance/directory — new path-scoped artifact set; the pre-commit hook regex extends to govern it alongsideexperiments/.- KoBoToolbox, Access, and paper-digitization exports — new data inputs from the three regional collection systems.
Materials
- Selam's formal memorandum with terms-of-reference attachment — the entry artifact. The memo names the problem and the requested deliverables; the political, systemic, and methodological realities surface during discovery.
- Synthetic ECDF dataset in BigQuery — five years of quarterly indicator history across six regions and 35 cooperatives, with the per-region inconsistencies preserved so profiling surfaces them. Tables for federation-level donor submissions, regions, cooperatives, households, individuals, training events, market access, food security, gender disaggregation, youth participation, and the five-year
donor_reportshistory. - Three regional-collection-system samples at
regional-exports/— KoBoToolbox from Oromia, paper-digitization from Amhara, Access ODBC from Tigray. - Carried-forward dbt project at
dbt-ecdf/configured against ECDF's BigQuery dataset with placeholder metric models for the 15 indicators — you author the standardized definitions and the version-history change-log entries. - Starter directories with empty section-header skeletons:
governance/(framework, change-process, ai-access-policy, historical-reconciliation),strategy/(data-acquisition, verification-architecture),analysis/(approach, limitations),donor-reports/{usaid,eu,dfid}/(templates matching each donor's reporting framework),training/(six regional quick-reference cards),skills/governance-framework-review/SKILL.md(empty),experiments/(path-scoped memory carrying forward),infrastructure/connectivity.md(P17-P18 phases preserved, P19 section to be appended), plusdiscovery/,memo/, andemail/for the Selam touchpoints. - Reference primers at
reference/— metric governance as organizational work, data acquisition strategy design, multi-stakeholder analysis design, verification architecture design, donor reporting requirements, historical reconciliation logic, training material design, and the regional collection systems. - Project
CLAUDE.mdandAGENTS.md— context and tech stack populated; the auto-memory curation notes, discovery notes, standardized definitions, framework summary, strategy summaries, donor-template summaries, non-reportable indicators, training-material summaries, and closing presentation notes are placeholders you fill in.
Download the materials zip (https://learntodriveai.dev/materials/analytics/p-19/materials.zip) and unzip it to ~/dev/analytics/p-19.