There are two ways AI and product discovery meet. The first is using AI to speed up research, which I covered in continuous discovery and AI.
This post is about the other direction, which gets far less attention: using continuous research to build and maintain an AI product. For my MSc thesis I put this question to a product team building AI features, and their answers were sharper than the general commentary on the topic.
AI features decay without fresh data
The team’s top answer, with four votes, was the one that matters most:
AI needs to continually learn. Data sets should be continually updated to reflect the latest learnings and keep everything up to date.
This is the structural argument for continuous discovery in an AI product, and it is stronger than the usual one. A conventional feature ships and stays roughly as good as the day it launched. An AI feature that stops learning gets worse relative to the world around it, because user language, expectations and behaviour keep moving.
Continuous research is one of the mechanisms that keeps the picture current. As the team put it, providing more and fresher data helps AI generate accurate results based on actual facts.
Use research to scope what the AI should do
Before building anything, there is a question worth answering properly: what should this actually do for the user?
The team voted for using user research to scope and help shape AI and data solutions as they get implemented. This sounds obvious and is routinely skipped, because AI features often start from a capability rather than a need. Someone realises the model can summarise, so the product gets a summarise button, and nobody checks whether summarising was the problem.
My interviews found a useful pattern here. Deciding how to incorporate AI is usually a time-bound strategic research project, not continuous work. One participant described exactly this: the organisation needs to determine how it is going to incorporate AI and what should be on the roadmap, which calls for a focused study rather than a weekly rhythm.
So you often need both. A focused project to decide what to build, then continuous discovery to keep it useful once it exists. That dual approach is covered in continuous discovery vs traditional research.
The circular loop
One workshop answer described a loop worth designing for deliberately:
In circular research, by getting responses we can use the insights to get better responses, and so on.
Each round of research improves the questions you ask and the data you feed back. Better data produces better output, which produces more useful feedback from users, which sharpens the next round. It compounds if you set it up, and stalls if you treat research as a one-off input at the start.
The team also noted this helps solidify understanding and tailor how the AI is trained, which is a more concrete link between discovery work and model quality than most teams draw.
What AI can do with the data you already have
The team’s other answers were about a problem most organisations recognise. Two answers, each with three votes:
- Analyse submission data alongside decision data to better inform users, in their case to increase validation rates.
- Collate and categorise the data and findings you already hold. Their phrasing was pointed: we have a lot of data, but it is not readily useable or accessible.
That second one is the real state of most companies. The bottleneck is not collecting more, it is that what exists sits in formats nobody can use. Then, with two votes each:
- Analysing qualitative wording and identifying trends across large volumes of text.
- Consolidating findings, generating reports from analysed data, and helping define problems.
- Deep pattern-finding that would not be obvious to the human eye.
And one forward-looking idea with a single vote, which is where a lot of research tooling is heading: making user research more dynamic by engaging with participants directly as they give feedback, responding and following up in the moment.
Bring the AI team into the research
A practical point from my interviews. Teams shared raw research data with their AI engineers and data scientists, so those people could work with authentic user material rather than a filtered summary.
That works in both directions. The engineers get real context for what they are building, and the research team gets help analysing volumes of data they could not process manually. But the same caveat applies as everywhere else: insights generated by AI are not always actionable, and someone still has to judge them.
The line that does not move
Everything above still sits behind one rule from my research, and it applies with more force when you are building AI products rather than just using AI tools.
Do not put sensitive user data into public models. One participant was categorical that they never put user data into ChatGPT or anything like it, and that recommendation made it into my thesis. If you are training or fine-tuning on user data, that becomes a consent and governance question, not a convenience one.
Build the data agreement before you build the feature.
Further reading
- Teresa Torres, Product Talk. producttalk.org
- My MSc research. Eight expert interviews and a twelve-person workshop. Read the case study
Next, read continuous discovery and AI or the continuous product discovery process.
Based on my MSc thesis on continuous product discovery. The research is written up in the case study.
