If Your AI Product Occasionally "Punches the User in the Face," It's Not Viable
Alex critiques teams who treat LLMs as reliable oracles and try to grind down their failure rate. Even at 99% accuracy, an unpredictable 1% that badly hurts the user makes for a non-viable product. Instead of chasing full accuracy, he argues you should design products that assume the tool is 'squishy' and take LLMs for granted as capable-but-imperfect.
- Reducing failure from 5% to 1% still isn't enough if the failure is severe
- LLMs are a 'squishy computer' that does roughly what you meant, not exactly
- Design around unreliability rather than trying to eliminate it
- Ask what you can build now that you have 'magical duct tape', not how to make it perfect
“even if you get it down to like 99% of the time it's fine if it punches in the face that's not a viable product”