Workshop Notes

AI Product Discovery: Find the Decision Worth Improving

Listen · 6 min My words, my voice — synthesized.
Cut-paper illustration of a pale blue field of small identical decision-branch diagrams scattered across the frame, one orange circle marking the single branch that's been traced and tested before any model got chosen

AI product discovery is the work of finding the one decision worth improving with AI, before anyone touches a model: a real user pain, data that actually exists to support the fix, the risk if the system gets it wrong, and whether the fix fits how people already work. As of September 2026, after 15 years leading product and engineering teams, including through eight acquisitions inside a regulated clinical research organization, I've watched teams start with the model and work backward to a problem that fits it. That's discovery in reverse. The model is the last decision in this process, not the first, and skipping ahead to it is why so many AI features ship that nobody asked for.

What is AI product discovery?

AI product discovery is the work of testing whether an AI feature is worth building before you build it: confirm a real user pain exists, check that the data to support it exists, weigh the risk of a wrong answer, and test workflow fit. The model comes after those four checks pass, not before.

I've sat in enough of these meetings to know the order usually runs backward. Someone has access to a model, or a mandate to "do something with AI," and the search starts from there: what could we point this at? That question always finds an answer, because almost any workflow can be described in a way that makes AI sound relevant to it. The four checks below exist to stop that search early, before a team spends a quarter building a feature that was really a solution looking for its problem.

How do you find the user pain worth solving with AI?

Find a pain someone already works around today, by hand, often enough that they can describe the workaround without thinking. If the honest answer to "why this matters" is "a competitor has it" or "it would be impressive," the pain isn't real yet, whatever the model can do.

This is the same discipline behind Eric Ries's build-measure-learn loop: you're not allowed to build until you can state, in advance, what you'd need to see to know you were wrong. During the acquisitions I led product through, the target companies almost never lacked ideas for what AI could do. What they lacked was a person who could name, specifically, the task they currently do by hand that they'd stop doing tomorrow if a machine did it for them. When I found that person, the project had a shape. When I couldn't find them, I'd usually stumbled onto a feature someone wanted to have, not a problem anyone wanted solved.

How do you know if the data exists to support an AI feature?

Pull a real sample the feature would run on and read it yourself before writing a spec. If you can't find enough labeled, representative examples of the exact case you're building for, the data problem is the actual project, not a footnote to the AI project.

This step gets skipped more than any other, because it's the least fun one. It's easier to sketch the feature than to spend an afternoon reading two hundred real support tickets or clinical notes and asking whether they actually contain the signal you're assuming is in there. In a regulated environment, this check gets harder, not easier: the data that would make the feature good is often the data with the most restrictions on who can see it and how it can be used, and "we'll figure out access later" is how a promising pilot dies in month four instead of week one.

What risks do you need to test before you pick a model?

Test what a wrong answer costs and who sees it first. A wrong suggestion in a private draft is cheap. A wrong answer sent straight to a customer, or one a person can't easily check, is expensive, and the model choice changes completely depending on which one you're building.

I wrote about this same reversibility test in decision-making under pressure: a decision that's cheap to undo deserves less ceremony before you take it, and one that's expensive to undo deserves more. An AI feature is a decision, made by software, at scale, usually faster than any one person could make it by hand. That's the whole appeal, and it's also the whole risk. Before picking a model, I want to know exactly where a human still checks the output, and what happens the first week that check gets skipped because the queue is long and the model has been right ninety times in a row.

Here's the table I actually walk through before any model gets named:

Check Question it answers What a pass looks like What happens if you skip it
User pain Is there a real, named workaround today? One person describes it without prompting The feature gets built and quietly ignored
Data Does the signal actually exist in real examples? A read sample confirms it, before the spec is written The data problem surfaces mid-build, as a delay
Risk What does a wrong answer cost, and who sees it first? The blast radius is named and someone owns the check A bad output reaches a customer with nobody watching
Workflow fit Is the full loop faster than what it replaces? Prompting, checking, and fixing together beat the old way The model adds a task instead of removing one

The same table doubles as the proof a proof of concept is supposed to produce: not a demo of what the model can do, but an answer to the one question that decides whether it's worth building at all.

How do you test workflow fit before committing to a model?

Time the whole loop a person actually runs: prompting, checking, correcting, and redoing the work when the model is wrong. If that full loop is slower than what it replaces, the feature doesn't fit yet, no matter how good a single output looks in a demo.

I wrote the same test into an AI adoption framework at the rollout stage, and it applies just as hard here, one step earlier. A model that drafts something in ten seconds but needs twenty minutes of correction hasn't saved the twenty minutes it looks like it saved in a demo. It's added a new task, checking the model's work, on top of the old one. The only honest way to know is to hand the loop, start to finish, to the person who'd actually run it, and time it against what they do now.

None of this is a case against building with AI. It's a case against letting the model pick the project. The four checks above are slower than opening a chat window and asking what's possible, and that's the point: they cost an afternoon or a week, and they're supposed to. The alternative costs a quarter, on a feature that was never solving anyone's actual problem, and that bill always comes due later than the checks would have.