Don't Default to the Chatbot: Choosing the Right AI Interface
Reject the intuitive AI copy; ask what problem your business actually needs solved.
- Difficulty
- Moderate
- Time to result
- ~weeks to results
- Steps
- 3
- Confidence
- 90%
When a new AI capability arrives, teams reflexively replicate the familiar interface — a chatbot on their own data, or 'an AI agent for X.' Khan's framework says to resist that, describe the problem rather than the solution, and ask what the right interface actually is. Often the best AI experience doesn't look like chat at all; it speeds up a manual process or is so seamlessly embedded that it doesn't feel like AI.
Origin
Aman Khan's observation from Arize AI's vantage point watching customers (Spotify, Instacart, and others) build after ChatGPT's launch, combined with Yuri Timen's point (cited by Khan) about describing solutions instead of problems.
Core principles
- 01Don't do what everyone else is doing just because it feels intuitive and familiar
- 02'We're building an AI agent for X' describes a solution, not a problem
- 03The right AI interface often optimizes a manual process rather than adding a chat box
- 04The strongest AI can be invisible — so seamless it doesn't feel like AI
- 05Ask whether to build the agentic layer yourself or let foundation-model companies build it
How to run it
- 1
Catch the reflex to replicate
Notice when your team's first instinct is to copy the familiar interface — an internal chatbot on your knowledge base, or an 'AI agent' for some task — simply because everyone is bought into it and it feels great.
Watch out Building the same chatbot everyone else built after ChatGPT often produced products with little real usage — 'was that the right thing to do' is poorly understood.
- 2
Describe the problem, not the solution
Force yourself to state the customer problem in plain terms. Saying 'we're building an AI agent for that' is describing a solution; you haven't yet named what's actually broken for the customer.
Pro tip If you can't articulate the problem without naming AI, you're not ready to choose an interface.
- 3
Ask what the right interface actually is
Consider that the best interface may not be a chatbot — it might optimize or speed up a manual step humans do today. Decide whether to build the agentic layer yourself or let the foundation-model companies build it and instead make the experience seamless in your existing product.
Pro tip The highest-innovation move is often to make the AI so seamless it doesn't even feel like AI.
In the wild
After ChatGPT's 'iPhone moment,' nearly every AI PM Khan knew was building an internal chatbot on their knowledge base. Arize instead identified that its team spent a lot of time on data analysis to make decisions, and built AI to automate that. They shipped only a small chatbot to understand the tech, but their real AI interface looked very different.
→ A cutting-edge AI experience shaped around an actual internal bottleneck rather than a copycat chat interface.
Post-ChatGPT, AI PMs gravitated to building 'an internal chatbot on top of our knowledge base' expecting it to answer questions greatly. When examined, how much usage these products actually had was low and poorly understood.
→ A cautionary pattern showing the intuitive interface is frequently the wrong one.
Common mistakes
Replicating a foundation-model product
If your team is building something very similar to a foundational model company's own product (e.g. another ChatGPT on your data), it's a red flag — you're unlikely to have a durable opportunity and usage tends to be low.
Describing the solution instead of the problem
Saying 'we need an AI agent for this' skips defining the customer problem, so you build technology in search of a use case rather than fitting technology to a real, articulated need.
Is it for you?
Best for
AI PMs and product leaders under pressure to 'do something with AI' who must decide what to actually build
Not ideal for
Cases where a conversational interface is genuinely the best fit and validated by the problem
From the transcript
“don't do what everyone else is doing just because it feels intuitive to go and like replicate this interface that feels really great”
“the interface for AI might look really different it won't necessarily look like a chatbot it might look like you're trying to optimize or speed…”
“you're not really describing the problem you're describing the solution an AI agent to do something”
“make that experience so amazingly seamless in your existing product that it doesn't even feel like AI in the first place”
From the episode
Becoming an AI PM
Aman Khan (Arize AI, ex-Spotify, Apple, Cruise)