Pre-launch · Training and the free Foundations exam are open now. Levels 1–5 open soon. Get notified on any level page.
20 min read
Sooner or later every production system changes models: a provider retires a version, a cheaper option appears, or a new capability becomes worth having. Moving a prompt to a new model is a migration, not a copy-paste. This module covers three linked skills: choosing the simplest architecture that meets the need, proving a prompt still works after the move, and migrating the many copies of a prompt that tend to accumulate across an organization.
Architecture options run from simple to complex:
Each step up adds cost, latency, failure points, and maintenance. Move up only when the simpler option fails a defined quality bar on your test set. A team that builds an agent for a task a single prompt can handle has created risk with no measured benefit.
Routing sends different requests to different models. Simple, high-volume requests may go to a smaller, cheaper model. Complex or high-risk ones go to a larger one. The routing rule must be based on measured performance per request type, and it needs its own tests: a misrouted complex request is a quality failure.
Models differ in how they weight instructions, format output, handle long context, and refuse. A prompt tuned on one model is an untested hypothesis on another. Before switching:
Popular prompts get copied. Over a year, a single prompt may exist in eight versions across teams, each with small edits. Some edits are drift. Some are legitimate: a branch that must include a regional code, a team that needs a different output field. A migration that replaces everything with one "clean" version will break the legitimate ones.
So inventory first: find every copy, its owner, its consumers, and what it does differently. Separate legitimate variants (documented, needed) from drift (undocumented, accidental). Build a shared core with documented parameters for real variants. Migrate in risk order, test each branch's outputs old versus new, and retire old copies only after their consumers have moved.
Rosa Delgado leads AI enablement at a regional bank's contact centers. A call-summary prompt runs in nine copies across three centers and two business lines. The current model is being retired in 90 days. One team proposes rebuilding the whole thing as an agent on the newest large model "to future-proof it."
Weak approach. Build the agent, copy the most recent prompt version into it, and switch all nine copies on the retirement date.
Better approach. Rosa runs the migration as a measured project.
Before:
Rebuild call summaries as an agent on the new large model. Use the
newest prompt copy for everyone. Switch on retirement date.
After:
1. Architecture: test single prompt on new small model and new large
model against the 300-call regression suite. Small model meets
the quality bar on 94% of call types; fails on complaint calls
with regulatory language.
Decision: single prompt, routed. Small model by default; large
model for calls tagged complaint or regulatory. No agent.
2. Robustness: add 5 paraphrase variants for 40 key cases. Small
model drops the "next action" field when agents speak in
fragments. Targeted repair: move field rules after the
transcript. Full suite rerun: pass.
3. Inventory: 9 copies found, 9 owners. 2 legitimate variants
(mortgage line needs loan_stage; Center C needs Spanish-language
flag). 7 differences are drift.
Shared core v1 + 2 documented parameters.
4. Migration: Center A (week 1), B (week 3), C (week 5). Old vs new
outputs compared on 50 calls per branch. Old copies retired after
logs show zero calls for 14 days.
Why this is stronger. The architecture is the simplest one that passed the tests, with routing justified by measured failures. Robustness was tested on paraphrases, and the repair targeted one cause. The inventory protected two real requirements and removed seven accidental ones. Retirement was verified, not assumed.
Setup. Pick a prompt you use. If you have two different chat models or model sizes available, use both. If you only have one, use that.
Task. Write 6 test inputs, then write 2 paraphrases of each, for 18 total. Run all 18 on each model you have and record whether each output meets your quality bar. Identify any input that fails across paraphrases.
What a good result looks like. You have a pass/fail grid across all 18 inputs. You've named a specific cause for at least one failure rather than just "the model was worse." If you ran two models, you can also say which model is adequate for which input types.