Continuous discovery and AI: where it helps and where it doesn't

Every practitioner I interviewed for my MSc thesis had a view on AI in research, and most were using it somewhere. But none of them handed the whole job over to it. The interesting part is where they drew the line.

Here is how practitioners actually used AI in continuous discovery, and where they deliberately kept people in charge.

Where AI helps

AI earned its place on the repetitive, time-consuming parts of research:

• Writing scripts and discussion guides. Several people used it to draft interview questions faster. As one put it, they never had to begin with a blank canvas.
• Transcription. Recording and transcribing calls automatically, which removes hours of manual work.
• Summarising. Generating first-pass summaries of interviews and long documents.
• Sorting survey responses. Grouping open-text answers into categories automatically.
• Getting unstuck. As a starting point when they were not sure how to begin, rather than a final answer.

The common thread is that AI is fastest at the mechanical work around research, freeing people to spend their time on judgement.

The tools practitioners named

• ChatGPT, for drafting and summarising. One team connected it to Confluence so it could read their existing documentation and answer questions from it.
• Dovetail, for transcription, tagging and managing research data.
• Condense, for summarising research material.
• Playbook QX, which participants singled out as being particularly good at sorting survey answers into groups automatically.
• Listen Up, which generated a card holding both the transcript and an AI summary for each session.

Where to keep humans in charge

When it came to making sense of what users said, practitioners were firm that people still do the real work.

• Synthesis stays human. AI summarised individual sessions, but pulling findings together across sessions was not handed to AI at all. As one participant said, the human is still doing the synthesis.
• Quality needs checking. AI summaries were not consistently good enough to trust. The technique participants used was to proofread the summary and check it against the assumption they went into the session with, not just against their notes.
• Manual still wins for the important parts. Even where AI was available, several people preferred to pick out the most important details themselves rather than accept the machine’s selection.

My own view, on top of what the research found: a transcript captures words, but a person in the room catches tone, hesitation and the thing left unsaid. That is a good reason to keep a human in the loop even when the summary looks complete.

The one rule worth keeping

One participant was categorical about this: they never put user data into ChatGPT or anything like it, and did not use it to synthesise. That became one of the recommendations in my thesis, and I would repeat it here.

AI can help you work faster, but user trust and privacy are not worth trading for a quicker summary. If you use AI on research material, use tools that keep your data private and have a clear agreement about how it is stored.

What this means for your team

A sensible starting point:

• Use AI for scripts, transcription and first-draft summaries.
• Keep analysis, synthesis and decisions with your team.
• Always check AI output against the assumption you started with.
• Never feed sensitive or personal user data into public models.

Used this way, AI makes continuous discovery easier to sustain, which is often the hardest part. For more on that, see 11 challenges of continuous product discovery.

Further reading

• Teresa Torres, Product Talk. producttalk.org
• My MSc research. Read the case study

Based on my MSc thesis on continuous product discovery. Read about the research in the case study.