Evaluating a Tool Landscape Change

Armen Hovhannisyan, CTO at PayNow Armenia, has 14 pipeline configs running on Dagster 0.15.10 and Python 3.9 -- versions the Central Bank of Armenia's IT audit has just flagged as unsupported, with 90 days to remediate. Current Dagster removed the API every one of those configs is written against, and the finance reconciliation pipeline runs daily for the Central Bank and cannot go down.

A note on the scenario: PayNow, Armen and the audit finding with its 90-day window are fictional. The tool facts are real: Dagster 1.0 (August 2022) removed the @solid / @pipeline API PayNow's configs use, 0.15.10 was the last release before it, and Python 3.9 reached end of life on October 31, 2025. The breaking changes in orchestrator-changelog.md are Dagster's own migration notes, quoted with sources.

The discipline skills: a three-layer durability framework written down explicitly — open standards, cross-platform patterns, tool-specific features; a 14-config inventory classified against it; a migrate / stay / switch evaluation memo in Armen's terse register; a staged migration plan that respects the daily-reporting cadence; a parity check on the finance reconciliation cutover encoded as a GitHub Actions hook the merge cannot bypass.

The AI-direction lesson: the durability vocabulary you have accumulated implicitly across the track is a transferable framework — writing it down is what makes your experience portable. AI defaults to "migrate to the new version" with an effort estimate; you hold the three-option discipline and require AI to evaluate each option against the framework first. AI proposes vendor comparison from changelogs; you ground the evaluation in what you have built. AI under-prices the volatile tool-specific layer. AI writes a clean migration plan that ignores the production calendar.

Your Role

You are the architect. Armen sends a Slack message. You reply in his register, open a targeted-discovery thread, read the 14 configs directly, write the three-layer framework as the explicit deliverable, classify every config against it, evaluate the new version and two alternative orchestrators, build the recommendation, stage the migration around the off-cycle window, encode the parity check as a hook, run the cutover, and hand PayNow a portable engagement bundle.

The scaffolding is thin. A Slack message and a 90-day regulator-imposed deadline are the entire brief. No framework template, no inventory template, no migration-plan template, no parity-check template. Everything is your design.

The AI relationship is supervisory-to-autonomous inside a framework you author. AI does not propose the framework, the classification rules, the recommendation, or the parity logic. AI reads each config, drafts the per-config rewrite against the new version's API, and runs the parity query — inside the structures you designed. The cutover hook is the most autonomous expression: the parity check runs in GitHub Actions on the cutover commit, and the merge cannot complete unless row count, total amount, and per-merchant settlement match between the old and new versions. AI cannot override the hook.

What's New

Last project you evaluated an eleven-pipeline portfolio for Marlene Castellanos at the Belize Tourism Board — per-pipeline build-vs-buy, three-year cost forecast, transition planning around the cruise-season reporting calendar, a board-defensible recommendation. The framework-before-execution discipline carries forward.

No colleague reviews the work — Armen is the only client check-in, and his bandwidth is measured in minutes. (Tigran Marutyan, a data-reliability engineer, is reachable on demand for a narrow sanity check on whether the finance-reconciliation cutover is protected by a parity gate the merge cannot bypass; he volunteers nothing and writes no part of the migration.) Genuinely new: a regulator-imposed remediation deadline triggered by end-of-life software (every previous deadline was client-set); the three-layer durability framework written down as an explicit deliverable PayNow inherits; evaluating alternative orchestrators by applying a pattern vocabulary, not a feature-matrix; a staged migration executed end-to-end rather than recommended on paper; a parity check encoded as infrastructure the cutover cannot bypass; Armen's terse Slack register.

The hard part: holding the three-option discipline when Armen says "I'm thinking migrate." Not everything the upgrade breaks is in the changelog. Configs written by a contractor who left six months ago can lean on the orchestrator in ways nobody wrote down, and a migration can leave a pipeline running green while its numbers are wrong. Those dependencies surface only when your questions to Armen target them. The cutover window is overnight; the regulator reads the daily report at 6 AM; a row-count mismatch is a regulator-flag event.

Tools

• Claude Code, Dagster (the current orchestrator, 0.15.10 → current 1.x as the canonical migrate target), dbt, BigQuery, Soda Core, GitHub Actions, Git + GitHub, CLAUDE.md + AGENTS.md as portable engagement memory — all carry-forward.
• Alternative orchestrators evaluated against the framework but not implemented: Airflow (the AI-popularity-default trap) and Prefect (the per-pattern-fit candidate).
• Managed-tool references named in the alternatives evaluation but not implemented: a managed CDC service (Fivetran or Airbyte) as a partial-switch option for the merchant transaction stream; a managed observability layer (Monte Carlo or SYNQ) if Armen's scope-creep beat about better monitoring is accepted.

No new tools. What is distinctive is the use: the three-layer durability framework as the explicit evaluation contract, and a parity check encoded as a GitHub Actions hook with no AI-override path.

Materials

You'll receive:

• A project governance file (CLAUDE.md) — the engagement repository's session context for Claude Code: the engagement briefing, plus an @AGENTS.md import that brings in the shared rules (scope, the three-layer durability vocabulary, the constraint-flow sequence, verification targets, recommendation-register rules, commit convention).
• The cross-tool project memory (AGENTS.md) — the engagement memory written once in the open-standard project-memory file and imported by CLAUDE.md, so the engagement can be re-opened in any AI tool.
• Armen's opening Slack message (armen-slack.md) — channel #urgent-data, 11:23 AM, three short paragraphs. The register you reply in.
• PayNow operational picture (paynow-context.md) — the 40-person Yerevan fintech, the 50,000-transactions-a-day scale, the Central Bank of Armenia reporting commitment, the 14-pipeline portfolio named at the function level, the team and contractor history, the operational constraints.
• The 14 pipeline configs (paynow-configs/) — PayNow's Dagster 0.15.10 files, which you read directly against the framework. Realistic in shape so each layer of the durability framework is legible in the code; they import on 0.15.10, and the finance reconciliation runs locally with stub data. The folder's README says how to run the legacy stack.
• The orchestrator changelog (orchestrator-changelog.md) — Dagster's own migration notes for 0.15.10 → 1.x, excerpted with sources: the legacy-API removals, the schedule and sensor changes, the executor-default change for jobs, the instance-storage migrations and public-API boundary, and the migration path.
• The alternatives reference (orchestrator-alternatives-reference.md) — Airflow and Prefect profiled at evaluation depth, plus a closing summary. No recommendation; the decision rules are yours.

Deliberately not provided: the durability framework, the classification criteria, the migrate / stay / switch decision rules, the migration-staging rules, the parity-check specification, the recommendation register. The BTB engagement workspace from last project does not carry forward — fresh project root for PayNow.