Evidano home
Published in Discover Artificial IntelligenceResponsible AINorthern Illinois University

What AI Can See in Qualitative Health Data, and What It Still Misses

Evidano7 min read
Portrait of Christy Muasher-Kerwin

Featured researcher: this paper is featured independently and does not evaluate Evidano.

A qualitative researcher can spend days moving between transcripts, tentative codes, candidate themes, and the quotations that give those themes human texture. The difficulty is not simply volume. It is deciding what deserves to be grouped together, what must remain distinct, and which details become flattened when analysis is accelerated.

That tension sits at the center of a 2026 study in Discover Artificial Intelligence examining whether large language models can support several linked stages of qualitative health research. Using nearly 48,000 words from four online brain cancer support forums, the researchers compared GPT-4 and Claude with a prior human-led applied thematic analysis, then explored how AI systems selected quotations and produced visual representations of the same material.

We asked Christy Muasher-Kerwin, a physical therapy researcher and first author of the study, and M. Courtney Hughes, Professor of Public Health at Northern Illinois University and second author, to tell us more about what AI can contribute to qualitative analysis, where it can lose important distinctions, and why researcher oversight remains central.

About the article

Research question

How well can large language models support coding, theme development, representative quotation selection, and visualization in qualitative health research when their outputs are compared with human-led applied thematic analysis?

Key finding

GPT-4 and Claude recovered many of the same broad thematic patterns as human analysis, but they produced fewer and less detailed codes and differed in what they emphasized. The study also exposed concrete failure modes, including omitted practical themes, questionable numerical visualizations, misclassification, and distorted text, making human validation a necessary part of the workflow.

Why this research matters

AI-assisted qualitative analysis is often discussed as a question of speed: can a model process text faster than a researcher? This paper makes the more useful question harder. Even when two analytic approaches arrive at broadly similar themes, do they notice the same kinds of experience, preserve the same distinctions, and choose evidence for the same reasons?

That matters because qualitative analysis is not only classification. A code is a label attached to a segment of data; a theme is a higher-level pattern that interprets how several coded ideas relate. A system may therefore reproduce the general shape of an analysis while still overlooking practical, procedural, or context-specific material that a human researcher considers central.

For researchers considering AI as part of an analytic workflow, the study is valuable precisely because it shows both convergence and friction. Muasher-Kerwin reflected on the broader significance of the work: “I think the broader significance of this work lies in demonstrating AI’s potential to make qualitative research more efficient and accessible. By helping researchers identify patterns and themes more quickly, these tools can allow researchers to devote more time to deeper analysis and exploring the complexities of human experiences.”

Key findings from the article

The most informative results are not a single accuracy score. They are the patterned differences among the human analysis, GPT-4, and Claude across coding, themes, quotations, and visualizations.

  • The models reproduced many broad themes, but with less granularity. Both GPT-4 and Claude generated fewer codes and less detail than the prior applied thematic analysis. Emotional responses, family and social dynamics, coping, and quality of life appeared across approaches, but the models sometimes grouped distinct issues together or placed them under broader labels. In the paper’s error analysis, both models aligned well with human-led themes for emotional and family experiences, while practical and procedural themes were more vulnerable to omission.
  • The two models selected qualitatively different kinds of evidence. GPT-4 tended to choose emotional, personal, and experiential quotations and its quotation selection was described as very similar to the human-led analysis. Claude selected more structured, solution-oriented, and clinically focused material, including treatment barriers, second opinions, financial burdens, and system navigation. The contrast shows that ‘representative’ evidence can depend on the model’s analytic emphasis, even when both systems are working from the same source material.
  • Visualization exposed both new possibilities and visible errors. GPT-4 produced forum comparisons, sentiment displays, and frequency-oriented graphics, while Claude leaned toward categorical charts. DALL·E translated themes into symbolic imagery. But the researchers also identified reasons not to accept visual outputs at face value: Claude generated a quality-of-life-over-time chart without a clear basis for the measurement, separated glioblastoma from astrocytoma in a tumor-type chart, and DALL·E produced distorted or misspelled text.

Taken together, the findings suggest that AI can recover substantial thematic structure while still changing what receives analytical prominence. Hughes reflected on what that means when reading an AI summary: “When I read an AI summary of a caregiver discussion, I want to know whether I can still tell what people needed help with. One detail I appreciated in Claude’s analysis was that it picked out discussions about second opinions. That tells us something specific about the decisions people were facing.”

Methods and research approach

The study reused data that had already been analyzed in earlier work using applied thematic analysis. The dataset came from four public online forums serving people affected by brain tumors and their caregivers: the American Cancer Society Cancer Survivors Network, the American Brain Tumor Association Discussion Forum, Mayo Clinic Connect’s Brain Tumor Support Group, and the MacMillan Cancer Support Online Community. The researchers collected the first 20 discussion posts and associated responses from each forum, producing a combined dataset of 47,824 words. They manually checked for repeated or copy-pasted material and reported no substantial duplication.

The researchers then entered each forum dataset sequentially into GPT-4 and Claude within a continuing conversation. They used the same core prompts for coding and theming, quotation selection, figure generation, and visualization support. Rather than treating the first outputs as final, the team used an iterative process, especially for visualizations, asking for added subthemes or more contextual detail. AI-generated outputs were cross-checked against the underlying data for relevance, depth, hallucinations, and misleading results.

This design is best understood as a methodological comparison rather than a benchmark with a single performance score. It also has limits. The results depend on the prompts used, the source material was drawn from participant-driven forum discussions rather than a standardized interview guide, and only two proprietary language models were tested. The authors therefore call for research across multiple prompting strategies, more structured qualitative datasets, and a broader range of models, including open-source systems.

Broader implications

For qualitative researchers, one practical implication is that AI may be most useful as an additional analytic reader rather than an autonomous analyst. The paper shows that models can rapidly propose codes, themes, quotations, and visual forms, but also that they can collapse categories, omit peer-to-peer or logistical dimensions, and generate confident-looking figures whose underlying logic is unclear.

The study also sharpens the idea of transparency. Because GPT-4 and Claude are proprietary systems, the researchers argue that transparency must come from the workflow: consistent prompts, identical datasets, preserved conversation histories, documented procedures, and manual checking against the original data. This shifts attention from whether a model is internally interpretable to whether the research process surrounding it is reproducible and auditable.

The unresolved issue is how researchers should decide which stages of interpretation can be delegated, accelerated, or cross-checked without weakening context and reflexivity. Muasher-Kerwin’s advice is to keep the researcher’s judgement at the centre: “I would encourage researchers to approach AI-generated findings with the same critical thinking they would apply to any other analytical method. Just because a model produces a convincing theme or a visually appealing figure doesn’t necessarily mean it’s accurate or meaningful. Researchers need to continually return to the original data and ask whether the output truly represents participants’ experiences. AI can help us recognize patterns we might otherwise overlook, but it can also overlook details that are critically important. Ultimately, the responsibility for ensuring that findings are accurate, meaningful, and ethically represented belongs to the researcher.”

Hughes pointed to one way to put that into practice in future studies: “In future iterations and studies, I would be interested in asking the AI model to find passages that challenge our interpretation. Those passages would be interesting to go back and read and could potentially help us see where a theme holds up and where we may need to qualify it.”

That distinction is likely to become more important as AI tools become easier to incorporate into everyday research practice.

About Christy Muasher-Kerwin

Portrait of Christy Muasher-Kerwin

Christy Muasher-Kerwin, MHA, PhD, DPT, is a physical therapy researcher who joins South College Nashville in December 2026. She led this study as a doctoral student at Northern Illinois University. Her recent research spans neuro-oncology, rehabilitation education, caregiving, qualitative health research, and the use of large language and multimodal AI models in health research. Her work has examined brain tumor support communities, family caregiver experiences, AI-assisted interpretation of qualitative data, and the performance of generative AI in rehabilitation education.

About the authors

  • Christy Muasher-Kerwin (Department of Physical Therapy, North Central College, USA)
  • M. Courtney Hughes (Department of Public Health, Northern Illinois University, USA)
  • Samantha M. Econie (Department of Nutrition and Dietetics, Northern Illinois University, USA)

Read the article

Read the full research article, “Using large language models to support qualitative health research through coding, theming, quotation selection, and data visualization,” published in Discover Artificial Intelligence in 2026.

Full citation

Muasher-Kerwin, C., Hughes, M. C., & Econie, S. M. (2026). Using large language models to support qualitative health research through coding, theming, quotation selection, and data visualization. Discover Artificial Intelligence, 6, 357. doi.org/10.1007/s44163-026-01088-4

View the publication

We are grateful to Dr. Muasher-Kerwin and Professor Hughes for sharing their perspectives on this study and for their contribution to the responsible, transparent use of AI in qualitative research.

Research, accuracy and responsible AI

How Evidano's output has been evaluated, how its models were built, how your data is handled, and how to disclose or cite AI use in your own work.

  • Resource

    Human vs AI: validated accuracy benchmarks

    Four head-to-head comparisons against expert human analysis — 371 interview transcripts with Arizona State and Penn State, 298 evaluation reports, 224 open-ended survey responses with UT Austin and Wichita State, and a 160-response manual-coding review.

  • Resource

    How Evidano’s models were trained

    Model provenance for research-ethics review: a provenance-tiered curated corpus, and customer uploads, prompts and outputs excluded from training entirely.

  • Resource

    Data security for qualitative research

    Encryption in transit and at rest, customer-controlled deletion, regional storage, and SOC 2 Type 2, GDPR, HIPAA-ready and CCPA/CPRA compliance.

  • Resource

    How to cite and disclose Evidano

    Citation formats in APA, MLA, Chicago, Harvard and Vancouver, methods-section wording per workflow, and an AI-policy review of 41 journals across 8 disciplines.