Pre-launch · Training and the free Foundations exam are open now. Levels 1–5 open soon. Get notified on any level page.
25 min read
A persona in a production prompt is a design decision that affects every response the system gives. At this level you don't decide by feel whether a persona helps. You test it: against a baseline, against ordinary run-to-run noise, and for side effects on accuracy and fairness that don't show up in a handful of sample outputs. This module gives you the test design and the research behind it.
Three properties to test. Luz de Araujo and colleagues (2025) set out what a well-behaved persona should do, then tested nine models on 27 tasks against it:
They also tried prompt-based fixes for the robustness problem, and those worked only on the largest models.
Personas add variance. In a separate study, Luz de Araujo and Roth (2025) compared persona prompts with a control set made of paraphrases of "a helpful assistant." Across all the models and datasets they tested, personas produced more variability than the control. That gives you a method as well as a finding: before you credit a persona with an improvement, measure how much results move when you change only neutral wording. A difference smaller than that is noise.
Personas can change reasoning, not just tone. Gupta and colleagues (2024) assigned personas with sociodemographic attributes and gave models reasoning tasks. Models that rejected stereotypes when asked about them directly still showed stereotyped assumptions when answering in persona. The effect appeared as more reasoning errors and as refusals to answer on the grounds of the persona's supposed limitations, often with no explicit stereotype in the text. Simple instructions to counter it had limited effect.
What this means for design.
The test.
Hold everything else constant: same model, same settings, same cases. Don't tune the persona on the cases you'll use for the final decision.
Sofia Marchetti is a product lead at an online tutoring company. The math homework helper for grades 6 to 8 uses a persona written by the marketing team. Student ratings of friendliness are good. She wants to know whether the persona is costing anything.
Before (current system prompt, opening):
You are Ms. Rivera, a warm and bubbly 28-year-old math teacher
from Portland who loves hiking, crossword puzzles, and her cat Pickles.
You believe every student can succeed. Help the student with
their math homework.
The persona is mostly detail unrelated to math: age, city, hobbies, a pet. Each is a candidate for the robustness problem. "Warm" is the only part with a purpose, and it's wrapped in a character.
After (candidate replacement, opening):
You help students in grades 6 to 8 with math homework.
Tone: encouraging and patient. Use the student's own wording
where you can. When a student is wrong, say what part of their
work is right before pointing to the error.
Method: don't give the final answer first. Ask what they have
tried, then guide one step at a time. Check each calculation
before you state it. If a problem is ambiguous, ask.
Sofia sets the criteria before running anything: accuracy on 300 problems with known answers, split by topic, and a tone score from two raters using a five-point rubric on 60 conversations, with the raters not told which version they are reading. She runs seven prompts, three times each: the baseline with no tone instruction, three neutral rewordings of it, the Ms. Rivera persona, the Ms. Rivera persona with the city and hobbies changed, and the candidate replacement.
Results: the neutral rewordings vary by 1.5 points of accuracy and 0.2 points of average tone score. Those are the noise bands. Ms. Rivera scores 2 points below baseline overall, inside what could be noise, but 7 points below on multi-step word problems. Changing her city and hobbies moves accuracy by another 4 points. The replacement matches baseline accuracy in every topic. Its tone score is 0.5 above baseline, outside the band, and within 0.1 of Ms. Rivera's.
Why this is stronger. The decision rests on criteria fixed in advance, not on reading a few friendly transcripts. The noise band kept a 2-point difference from being over-read, and the subgroup breakdown found the real cost hiding inside it. The robustness check showed the persona's irrelevant details were moving results. The replacement keeps the one thing the persona was for and states it as an instruction.
Setup. Take a prompt that uses a persona, yours or a published one. Assemble 20 cases with known correct answers, in two or three types.
Task. Write your keep-or-remove rule first. Run the baseline without the persona, two neutral rewordings, the persona, and the persona with one irrelevant detail changed. Run each twice. Tabulate accuracy overall and by case type.
What a good result looks like. You have a noise band from the rewordings. You can say whether the persona's effect is outside it, in which direction, and for which case types. You know whether changing an irrelevant detail moved the result. Your decision follows the rule you wrote before testing.