Product as Organism: the metabolic loop
Treat an AI product as a living system that ingests signals, tunes on rewards, and improves with every interaction
- Difficulty
- Advanced
- Time to result
- ~months to results
- Steps
- 5
- Confidence
- 90%
Instead of shipping a product as a static artifact and maintaining it, treat it as an organism whose core KPI is metabolism: how fast the team can ingest data, digest it through a rewards model, and produce a tuned outcome. Because modern models can be steered toward price, performance, or quality, the continuous learning loop itself becomes the company's core IP.
Origin
Articulated by Asha Sharma (CVP of Product, Microsoft AI Platform) from observing 80,000+ companies build on Microsoft's AI platform. She cites Cursor (Michael Truell) and Nathan Lambert's leaderboard study on the ~30B-parameter post-training threshold as supporting evidence.
Core principles
- 01The KPI is the metabolism of the team, not a static feature set
- 02Focus on the loop, not the lane — every function obsesses over the whole cycle
- 03Models can be tuned to an outcome (price, performance, or quality), so pick the outcome first
- 04Feedback becomes continuous and observability becomes the culture
- 05It is never one loop — multiple tracks run in parallel like assembly lines
How to run it
- 1
Pick the job-to-be-done and target outcome
Choose the specific use case and the outcome you are optimizing for — price, performance, or quality — before touching the model.
Pro tip Anchor on one clear northstar outcome so the rewards design has something concrete to optimize against.
- 2
Design the rewards model
Define how the system will be scored so that 'better' is measurable and the loop has a gradient to climb.
Watch out Skipping rewards design means you have no way to know if a tuning run actually improved anything.
- 3
Source data to feed the loop
Use your own proprietary interaction data (the strongest signal), or supplement with purchased or synthetically generated data.
Pro tip High-quality expert labeling beats synthetic data — Microsoft's Dragon jumped acceptance from 30-60% to ~83% after annotating 600,000 physician interactions.
- 4
Roll out and A/B test rigorously
Deploy the tuned model into the real product and measure impact against the rewards model with disciplined experimentation.
Watch out Without measurement, observability, and evals set up end-to-end, you are doing 'AI for AI's sake' and cannot tell if it worked.
- 5
Metabolize and repeat across parallel tracks
Feed the generated data back in, tune again, and run several of these loops in parallel like assembly lines so the product keeps improving with every interaction.
Pro tip A small cross-functional group iterating fast beats a large org — the Dragon gains came from a small team, not a big one.
In the wild
Dragon initially used synthetic fine-tuning with modest results. The team annotated 600,000 physician-patient interactions with experts, fed that into the model, and continuously optimized it in the loop.
→ Acceptance rate rose from roughly 30-60% depending on the run to about 83%, achieved by a small cross-functional group.
Cursor captures which code suggestions users accept and reject, using that proprietary interaction data to continuously improve the product.
→ The captured usage data becomes the durable competitive moat rather than any single feature.
Common mistakes
Doing AI for AI's sake
Kicking off many AI projects at once with no blueprint, no measurement, no observability, and no evals — so impact can't be felt in the P&L and nothing compounds.
Over-investing in the UI over the loop
Spending an inordinate amount of time on the canvas/pixels instead of the signals loop that actually makes the product learn.
Is it for you?
Best for
Product teams building AI-forward software who own proprietary interaction data and want a durable, compounding advantage
Not ideal for
Simple, static tools with no meaningful feedback signal or one-shot outputs where continuous tuning adds no value
From the transcript
“the whole KPI is what is the the metabolism of a product team to be able to ingest data and then digest the rewards model…”
“you have to come up with a rewards design you have to actually roll it out you have to AB test it rigorously you have…”
“we saw a massive difference from when we used you know synthetic fine-tuning to when we annotated 600,000 patient um physician interactions by experts”
“I think it's all about the loop not not the lane here”
From the episode
How 80,000 companies build with AI: products as organisms, the death of org charts, and why agents will outnumber employees by 2026
Asha Sharma (CVP of AI Platform at Microsoft)