Role Prompting: Does "You Are an Expert" Do Anything?

“You are a senior tax accountant with twenty years of experience” is probably the most-copied line in prompt engineering. It has also been studied properly, at scale, and the result is not the one the advice assumes.

The claim

The folk model is that a persona unlocks capability: the model contains an expert somewhere and the role prompt selects it. The weaker version — that a persona sets the voice — is uncontroversial. It is the strong version, that accuracy improves, that got repeated into best-practice lists, and it is the one worth checking.

It is worth being fair about where the advice came from. On base models, and on the earliest instruction-tuned ones, a persona did change output quality — because the model was completing a document, and the opening established what kind of document it was. A prompt beginning “the following is a transcript of an expert consultation” selected a region of text that came with more careful writing attached. Chat post-training replaced that mechanism with an explicit assistant identity, and the technique kept its reputation after losing the thing that made it work.

What the studies found

Zheng et al. (2024) ran the experiment the advice never did: When “A Helpful Assistant” Is Not Really Helpful: Personas in System Prompts Do Not Improve Performances of Large Language Models. They evaluated a large set of personas — 162 of them, spanning occupations and relationships — across several model families on factual question-answering, comparing each against a control with no persona at all.

Two findings matter. Adding a persona did not produce a systematic improvement over the no-persona control. And the best-performing persona varied unpredictably by model and by question set, which is the more damaging result: it means any single reported win — including one you find yourself on twenty examples — is consistent with noise.

The picture is not that personas do nothing at all. Salewski et al. (2023), In-Context Impersonation, found that asking a model to adopt an identity did shift its behaviour in structured ways: assumed age and assumed domain expertise moved performance on their tasks in directions that tracked the persona. Personas are a real conditioning signal. They are simply not a reliable accuracy lever, which is a different claim.

Why the mechanism predicts that

A model computes a distribution over the next token given the context. A persona is context. It shifts that distribution toward continuations that resemble text written in that role — vocabulary, hedging, the density of jargon, the shape of the paragraph.

What it cannot do is add information the weights do not contain. If the model does not know the depreciation rule, no amount of accountancy costume will produce it; the most likely effect is a more confident statement of the same wrong rule, because the persona also conditions the register, and expert register hedges less.

That is the design risk worth naming. A credentialed persona (“you are a licensed physician”) tends to reduce visible uncertainty in the output without changing the underlying reliability — which is precisely the wrong direction for anything a user might act on.

If you want to see the effect rather than argue about it, measure the thing it actually moves. Hold the question fixed, vary only the persona line, and compare output length, reading level, hedging-word frequency and jargon density over a few hundred generations. Those move visibly. Then compare accuracy on the same runs — that is the comparison where the published work found nothing systematic, and where a twenty-example impression cannot tell you otherwise.

The part that does work

Everything useful people attribute to role prompting is recoverable by being specific about the output instead of about the speaker:

• Name the audience, not the author. “Written for a hospital procurement officer who has not used the product before” changes what content is required, which is a real change. “You are a technical writer” changes only the costume.
• Name the genre. “Write this as an incident postmortem” carries structure — timeline, impact, root cause, actions — that the model will supply because the genre has it. This is the cheapest structural instruction available.
• Constrain the register explicitly. “No marketing adjectives, no second person, define every acronym on first use” is checkable and portable. It is what “professional tone” was gesturing at.
• Keep a persona for product consistency. A stable assistant identity in the system prompt is worth having for voice and for refusal scope. Just do not book its accuracy benefit in advance.

One caution about deleting personas wholesale. A persona line is often load-bearing for reasons that have nothing to do with accuracy: it is what keeps the assistant from discussing competitors, what makes three different endpoints sound like one product, and what a support lead points at when the tone is wrong. Removing it to save forty tokens is a product change dressed up as a prompt cleanup.

Testing it on your own task

If you want to know whether a persona helps your workload, the method is short and worth running — the studies above are about general benchmarks and yours may not behave like them.

• Fix an eval set of at least a few hundred cases with known-good answers. Twenty is not enough to see a small effect through sampling noise.
• Run the identical prompt with and without the persona line, temperature at zero, everything else byte-identical.
• Compare with a proportion test rather than by reading outputs. At a baseline accuracy near 80%, detecting a genuine three-point improvement needs roughly 2,800 cases per arm — which is why almost every reported persona win is unfalsifiable.
• Re-run it when you change model. The persona effect is a property of the model, not of the task, and Zheng et al.’s unpredictability finding means it does not transfer.