Flow-Preserving AI Assistance Design
Design AI suggestions around the user's flow state: no panel switches, no waiting, ephemeral by default
- Difficulty
- Moderate
- Time to result
- ~months to results
- Steps
- 5
- Confidence
- 90%
Copilot's UX was not obvious — GitHub tried side panels, right-click menus, and other surfaces before landing on inline multi-line autocomplete. The organizing constraint throughout was flow: any interaction that pulls the user's attention out of the cursor, or makes them wait, destroys the value even if the suggestion is correct. That constraint produced measurable design targets (a ~200ms latency sweet spot found by experiment) and visual conventions (gray italicized text signalling ephemerality), plus the recognition that model quality is only half the job — prompt crafting, what you feed the model, is the other half.
Origin
Emerged from GitHub and OpenAI's UX experimentation on Codex between the Arctic Code Vault data handoff and Copilot's technical preview, in partnership with the Visual Studio Code team, who had no extensibility point for multi-line autocomplete at the time.
Core principles
- 01Flow is the metric — the user's job is creating; anything that isn't creating is labor to be eliminated
- 02Attention cost is real even without a window switch: moving to a side panel is itself a distraction
- 03Latency is a product decision, found by experiment, not an engineering leftover
- 04Ephemeral suggestions need a visual grammar that says 'this is not yet yours'
- 05What you feed the model matters as much as which model you use
How to run it
- 1
Name the labor you are removing
Identify the specific non-creative drudgery in the user's workflow — recalling parameter order for an API, remembering syntax, generating dummy data, leaving to check docs, watch a tutorial, or ask on Stack Overflow and wait for an answer. That is typing, not creating.
Pro tip The strongest use cases are the ones where the user currently leaves the tool to get an answer. Every exit is a flow break you can reclaim.
- 2
Prototype multiple surfaces, then kill the ones that break attention
Test the plausible interaction surfaces — right-click side panel with whole-function options, inline completion, and others. Reject any surface that moves the user's eye off the cursor, even if it delivers richer output.
Pro tip The side panel gave whole functions, which is more output — and was still rejected, because the attention cost outweighed the content gain.
Watch out Richer output is not better output if the user has to break flow to consume it.
- 3
Experiment your way to a latency budget
Run experiments to find how many milliseconds you can take before the user feels interrupted rather than assisted. For Copilot, the sweet spot landed around 200ms, before regional network latency.
Pro tip Set the latency budget as a hard product requirement and make model/prompt choices subordinate to it.
Watch out A one-to-three second wait for a suggestion destroys the flow you were trying to protect, no matter how good the suggestion is.
- 4
Give speculative output a visual grammar
Present the AI's suggestion in a form that clearly signals it is provisional and not yet committed — Copilot uses gray italicized text — and settle the keystrokes for accept/dismiss deliberately.
- 5
Invest in prompt crafting, not just the model
Treat what you feed the model — surrounding variables, class names, method names, comments, the context of the codebase — as a first-class engineering discipline. The same model with a better prompt is a materially better product.
Pro tip Offer multiple suggestions the user can pick from, so acceptance rate becomes a legible quality signal per language.
Watch out Expect quality to vary by domain: languages with more volume and diversity in the training data produce better suggestions (Copilot's acceptance ranged from the upper 20s to the 40s percent by language).
In the wild
An early Copilot design offered whole functions in a right-click side panel. It was rich output, but the user had to shift attention away from the cursor to a different panel — a distraction even without switching windows.
→ GitHub abandoned it for inline multi-line autocomplete in gray italic text, which became the Copilot experience most developers know today.
Rather than guessing, the team ran experiments on how long a suggestion could take before a developer felt interrupted, given that their goal is to stay in the flow.
→ They converged on roughly 200 milliseconds as the latency target, with regional variance, and treated it as a product constraint on the model and prompt design.
Common mistakes
Optimizing suggestion quality while ignoring latency
A better suggestion delivered after a two-second wait is worse than a decent suggestion delivered in 200ms, because the wait is itself the interruption you set out to remove.
Assuming a new interaction surface exists
VS Code had no extensibility point for multi-line inline autocomplete when GitHub needed one. Designing a novel AI UX often requires building the platform affordance too — plan the partnership.
Treating the model as the whole product
'It's not just about the model, but it's also about what you feed the model.' Prompt crafting, context collection, and presentation are where the perceived magic actually comes from.
Is it for you?
Best for
Product designers and PMs embedding AI suggestions into a tool where the user is in a focused, creative flow state (editors, IDEs, writing tools, design tools)
Not ideal for
Asynchronous or batch AI workflows where the user has explicitly handed off a task and is not waiting in flow
From the transcript
“And we eventually came to this idea of inline auto complete.”
“The gray italicized text seemed to be a good way of indicating that it was ephemeral, as it were.”
“it seems like the sweet spot is somewhere around 200 milliseconds.”
“it's not just about the the model, but it's also about what you feed the model.”
“Like, that's just labor. It's not creating, it's just typing.”
From the episode
The role of AI in product development
Ryan J. Salva (VP of Product at GitHub, Copilot)