Evidano home
SSRN working paper, 2026Responsible AIColumbia Business School

What AI-Moderated Interviews Gain, and Where Human Moderators Still Matter

Evidano7 min read
Olivier Toubia, Glaubinger Professor of Business at Columbia Business School

Imagine a research team trying to understand why homeowners choose one replacement window or door over another. A static questionnaire can collect answers quickly, but it cannot notice an unfinished thought and ask, “What made that matter to you?” A skilled human moderator can, but the cost and scheduling burden make hundreds of interviews difficult. The practical question is whether an AI interviewer can preserve the adaptive value of a conversation while operating at survey-like scale.

That is the problem examined in “AI-Moderated Interviews for Market Research and Digital Twins Calibration,” a 2026 working paper by Yuting Deng, Jingxuan Liu, Olivier Toubia, and Naman Jain. The study compares AI-moderated interviews not only with live human interviews, but also with a static spoken interview delivered through the same platform. It then asks a second, harder question: whether richer interview data produces better consumer “digital twins” that can predict how the same person will respond to new marketing stimuli.

We spoke with Olivier Toubia, Glaubinger Professor of Business and Senior Vice Dean for Curriculum and Instruction at Columbia Business School, to learn more about what the comparison suggests for market researchers using AI-moderated interviews and consumer digital twins.

About the article

Research question

Can AI-moderated interviews recover the depth, breadth, and customer needs of human-moderated interviews more efficiently than static questioning, and does the richer data they generate improve the predictive performance of individual consumer digital twins?

Key finding

AI moderation produced broader and often deeper qualitative content than static interviews and recovered customer needs efficiently at scale, while live human moderation retained a clear advantage in emotional engagement. Interview data improved digital twins beyond demographics alone, but AI-moderated interviews did not produce more predictive twins than static interviews.

Why this research matters

Market research has long faced a tradeoff between scale and responsiveness. Surveys are standardized and inexpensive, while interviews can follow a participant’s language, clarify ambiguity, and pursue unexpected details. The paper matters because it tests whether AI can narrow that gap under conditions that resemble a real commercial research decision, including a fixed budget and real marketing materials rather than an abstract benchmark.

The distinction is important for organizations deciding what they actually want from an interview. If the goal is discovery, a moderator is valuable because it can probe the reasons behind an answer and surface needs that a fixed question may miss. If the goal is prediction, however, richer narrative data may not automatically make a model better at anticipating future choices. Toubia places that distinction in practical context: “The study, perhaps surprisingly, suggests that AI moderators are as effective as human ones at extracting customer insights and customer needs, for a fraction of the cost per interview, with the important caveat that consumers are more emotionally engaged when conversing with a human. The study also reminds us that consumers’ reactions to marketing stimuli (e.g., advertising, marketing claims) remain hard to predict, even with richer input.”

Key findings from the article

Three findings are especially useful because they separate interview quality, economic efficiency, and predictive validity rather than treating them as the same outcome.

  • Adaptive AI questioning produced richer content than static interviewing, but humans elicited more speech and stronger affect. Participants in the AI condition spoke for 8.1 minutes on average across the open-ended blocks, compared with 4.3 minutes in the static condition, and produced 952 versus 609 words. Human-moderated participants spoke longer still, averaging 11.3 minutes and 1,404 words. Yet the AI interviews covered more predefined themes than human interviews in three of four blocks, while depth was broadly comparable. The emotional pattern moved in the opposite direction: speech-based valence, arousal, and dominance were all higher with human moderators than with AI moderators.
  • Under a fixed research budget, AI moderation recovered customer needs more efficiently. Across all interviews, the researchers consolidated 252 unique customer needs. The AI condition recovered 238 needs from 139 interviews, compared with 188 from 154 static interviews and 109 from 24 human interviews. Using the study’s market-cost assumptions, the implied cost per unique need was about $19 for AI moderation, $27 for static interviewing, and $36 for human moderation. At a matched sample of 24 interviews, AI and human moderation recovered similar numbers of needs, so the advantage came chiefly from being able to run many more adaptive interviews for the same budget.
  • Richer interviews helped digital twins, but AI moderation did not make those twins more predictive than static interviews. Twins built with interview transcripts generally outperformed demographics-only personas, showing that open-ended self-report contained useful predictive signal. But the extra qualitative richness of AI-moderated interviews did not yield better predictions than static interviews on the preregistered rating and ranking measures. The error analysis points to a deeper challenge: twins were more analytical than people when reacting to mailers and commercials, while human responses were more intuitive and stimulus-reactive. Adding a response to a similar task improved rating accuracy from .743 to .787, but produced little improvement in ranking agreement.

Taken together, the results suggest that a stronger qualitative interview and a stronger predictive model are related but distinct achievements. Toubia explained what surprised him: “I was mostly surprised at how well AI moderation performed at extracting consumer insight and consumer needs. Given previous research, I was not as surprised by the somewhat modest performance of digital twins powered by AI-moderated interviews. Digital twins tend to need training that is directly relevant to the prediction task, and they tend to ‘think’ differently from humans.”

Methods and research approach

The preregistered between-subjects study included 317 English-speaking home-improvement decision-makers recruited through Prolific: 139 in AI-moderated interviews, 154 in static spoken interviews, and 24 in live human interviews. All conditions covered the same topics about category experience, purchase journeys, brand awareness, and brand perceptions. The human group was smaller because the researchers matched conditions to an approximate $5,000 market-research budget rather than equal sample sizes.

In the AI condition, a turn-based conversational system transcribed speech, used GPT-5.1 to decide on clarifications, follow-ups, or transitions within a moderator objective, and converted responses back to speech. Importantly, the model adapted only to the verbal transcript and conversation history. It did not use video, facial expression, prosody, or other nonverbal cues. The static condition used the same audio platform but presented fixed questions without probing, while trained moderators from GBK Collective conducted the live interviews.

The researchers assessed volume, thematic breadth, depth, speech-based affect, prosody, and customer-need coverage. Breadth and depth were coded against objective-derived codebooks using five independent LLM passes aggregated by majority vote, with robustness checks. For digital twins, multimodal models received either demographics alone or a fuller persona including interview transcripts and individual-difference measures, then predicted held-out responses to two mailers, two commercials, and two claim-ranking tasks. The authors caution that the study covers one high-involvement category, one AI platform at one point in time, and a relatively small human sample.

Broader implications

For researchers and insight teams, the immediate implication is methodological. AI moderation can make adaptive interviewing easier to scale, but the study does not make human moderators obsolete. Human interviews generated more emotionally engaged speech, which may matter when rapport, empathy, sensitive disclosure, or affect is central. The authors also caution that performance may differ in other product categories or sensitive domains.

The digital-twin results argue for a disciplined notion of calibration. A long interview can tell a system a great deal about a person without revealing how that person will react to a particular advertisement or claim. Closely related behavioral examples helped only modestly, and models still reasoned more deliberatively than the people they represented. Validation against held-out human responses therefore remains essential.

Future work may therefore need to combine better task-relevant calibration with richer signals about immediate human reaction. The authors point to voice tone, facial expression, attention, hesitation, and other affect-rich cues as promising directions. Toubia added a practical note on responsible use: “Insights generated by AI-moderated interviews should be reviewed and audited by humans, and digital twins should be carefully validated before being deployed in a particular use case.”

About Olivier Toubia

Olivier Toubia, Glaubinger Professor of Business at Columbia Business School

Olivier Toubia is the Glaubinger Professor of Business in the Marketing Division at Columbia Business School and serves as Senior Vice Dean for Curriculum and Instruction. His research combines methods from the social sciences and data science to study innovation, customer insights, motivation, choice, and creativity. He is also a co-principal investigator of Columbia Business School’s Digital Twins Lab and previously served as Editor-in-Chief of Marketing Science. He earned his PhD in Marketing and MS in Operations Research from MIT.

About the authors

  • Yuting Deng (Columbia Business School, Columbia University, USA)
  • Jingxuan Liu (Columbia Business School, Columbia University, USA)
  • Olivier Toubia (Columbia Business School, Columbia University, USA)
  • Naman Jain (The Wharton School, University of Pennsylvania, USA)

Read the article

Read the full working paper, “AI-Moderated Interviews for Market Research and Digital Twins Calibration,” published in the SSRN working paper series in 2026.

Full citation

Deng, Y., Liu, J., Toubia, O., & Jain, N. (2026). AI-moderated interviews for market research and digital twins calibration [Working paper]. SSRN. doi.org/10.2139/ssrn.7511818

View the publication

We are grateful to Professor Toubia for sharing his perspective on this study and for his contribution to the careful, validated use of AI in market and qualitative research.

Research, accuracy and responsible AI

How Evidano's output has been evaluated, how its models were built, how your data is handled, and how to disclose or cite AI use in your own work.

  • Resource

    Human vs AI: validated accuracy benchmarks

    Four head-to-head comparisons against expert human analysis — 371 interview transcripts with Arizona State and Penn State, 298 evaluation reports, 224 open-ended survey responses with UT Austin and Wichita State, and a 160-response manual-coding review.

  • Resource

    How Evidano’s models were trained

    Model provenance for research-ethics review: a provenance-tiered curated corpus, and customer uploads, prompts and outputs excluded from training entirely.

  • Resource

    Data security for qualitative research

    Encryption in transit and at rest, customer-controlled deletion, regional storage, and SOC 2 Type 2, GDPR, HIPAA-ready and CCPA/CPRA compliance.

  • Resource

    How to cite and disclose Evidano

    Citation formats in APA, MLA, Chicago, Harvard and Vancouver, methods-section wording per workflow, and an AI-policy review of 41 journals across 8 disciplines.