Persona-Framing for AI Product Behavior
Pick a human metaphor for your AI, then derive its behavioral guardrails from that role
- Difficulty
- Moderate
- Time to result
- ~weeks to results
- Steps
- 5
- Confidence
- 91%
Facing unanswerable ethical questions ('if the AI produces something offensive, who's at fault?'), Salva's team stopped reasoning about AI ethics in the abstract and instead named a persona: Copilot is an AI pair programmer. That single framing generated concrete, arguable answers about acceptable behavior — a human pair programmer who whispered slurs or politics into your ear would be intolerable, so the AI's behavior is constrained the same way. The persona converts an open philosophical debate into a design specification, and it also fixes the product's ceiling: a pair programmer augments, it never replaces.
Origin
Developed by Salva's team at GitHub while wrestling with the legal and ethical surface of shipping Copilot. It borrows the pair-programming practice from Extreme Programming (Kent Beck) and repurposes it as an AI product metaphor.
Core principles
- 01A persona turns 'what should the AI be allowed to do?' into 'what would this human role do?' — a question people can actually answer
- 02The persona defines the ceiling as well as the floor: a pair programmer augments a developer, it never replaces one
- 03Behavior guidelines derive from the use case plus the persona, not from an abstract ethics framework
- 04The persona must be defended in dialogue with the community, not just declared internally
- 05Moderation implementation should evolve — crude blocklists are a starting point, not the destination
How to run it
- 1
Name the human role the AI plays
Choose an existing, well-understood human collaboration role that matches the product's actual interaction pattern. For Copilot, the pair programmer: two developers side by side, one at the keyboard, one talking through ideas and making corrections.
Pro tip Pick a role your users already have intuitions about — the whole leverage comes from importing their existing expectations.
Watch out Do not pick a persona that overclaims (a 'senior architect', an 'autonomous engineer'); it sets an expectation the model cannot meet and invites the replacement anxiety.
- 2
Derive behavior guidelines from the persona plus the use case
Ask what a person in that role doing this specific job would and would not say or do. A pair programmer bringing politics, gender identity, slang, or slander into your ear would destroy your focus — so the AI must not either.
Pro tip Run the 'crazy pair programmer' test on any proposed model output: would a human colleague be fired for saying this?
- 3
State the augmentation boundary explicitly
Declare what the AI is not. Copilot is not a replacement for a developer, and does not replace any other part of the stack — static analysis, unit tests, or existing code-quality measures all stay in place.
Pro tip Say it publicly and repeatedly; the boundary is a product claim, not an internal memo.
Watch out Never ship a design where the AI can generate code with no thinking, reasoning human on the other side of the keyboard.
- 4
Build the filter as a ladder, not a launch requirement
Start with no filter to learn what the model actually produces, add a simple blocklist to remove the most egregious output, then graduate to sentiment/offensiveness models that can read context. Words offensive in one context are legitimate in another — medical software being the canonical case.
Pro tip Look for a parent company or partner with a responsible-AI research capability; GitHub used Azure's Department of Responsible AI to solve a problem it could not have solved alone.
Watch out Blocklists are fraught with peril — you become an editor of language, and developers will (correctly) complain when a legitimate word is blocked.
- 5
Hold the guardrails in public dialogue
Convene a wide cast — legal, privacy and security champions, and above all the developers using it — and keep asking what works, what doesn't, why this is offensive, why that isn't. Scale the product team, not just engineering, to carry the conversation.
Pro tip Treat community skepticism as an asset: 'We owe it to ourselves as a community to be skeptical of any AI.'
Watch out This product surface consumes far more legal and product time than a conventional feature. Budget for it.
In the wild
Salva reframed the abstract ethics question by imagining a pair programmer sitting next to you whispering politics, slurs, or slander into your ear. Nobody would be able to focus on their work. That intuition — not a policy document — produced the guidelines for Copilot's acceptable behavior.
→ The team converted an open-ended ethics debate into concrete developer-experience guidelines they could implement and defend.
Copilot shipped internally with no filter at all, then a simple word blocklist, which developers complained about when legitimate words were blocked or when the blocked term was itself offensive. GitHub then partnered with Azure's Department of Responsible AI to use models that detect patent offensiveness in context.
→ Context-aware moderation replaced crude blocklists, solving cases (like medical software vocabulary) where a word is offensive in one context and entirely reasonable in another.
Common mistakes
Debating AI ethics in the abstract
'If AI produces something offensive, who's at fault?' has no tractable answer. A persona gives you a concrete standard people can reason about and disagree with productively.
Shipping a permanent blocklist
Blocklists make you an editor of language, generate false positives that break real workflows, and cannot handle context-dependent terms. Treat them as a stepping stone.
Letting the persona imply replacement
If the metaphor suggests the AI substitutes for the human, you inherit every anxiety and every liability. Copilot's framing explicitly keeps a reasoning human at the keyboard and every existing quality system in place.
Is it for you?
Best for
Product teams shipping generative-AI features into a professional workflow who need defensible behavioral guardrails and a public story about what the AI is and isn't
Not ideal for
Fully autonomous back-office AI systems with no human in the loop, where a collaboration persona would be misleading
From the transcript
“what we ended up coming to is actually the the framing of Copilot as an AI pair programmer”
“some principles about what is the use case we're trying to solve, what is appropriate, I put this in scare quotes, behavior of the AI…”
“When we first started out, we didn't really have any filter on Copilot whatsoever, in the very very very early days.”
“And so, we introduced a simple block list of words. And these blocklists are always fraught with peril.”
“We do not want Copilot to replace any other part of the stack, whether it is static analysis tools or unit tests or, you know,…”
From the episode
The role of AI in product development
Ryan J. Salva (VP of Product at GitHub, Copilot)