What the study analysed
A Study of Unmet Needs Among Highly Intelligent Individuals surveyed 3,443 adults on their academic, career, social-emotional, and mental health needs across life stages, alongside ten follow-up interviews. The 12 open-ended items produced more than 19,000 comments, far more than the team could code by hand on the project’s timeline. The report was approved by the William & Mary and Mensa research review committees and is available open access.
How Evidano fit the workflow
The study describes a transparent, four-step workflow that pairs AI assistance with researcher judgment.
- 1Prepare the data
Open-ended responses were de-identified and split into batches of 150 for analysis.
- 2Run thematic analysis in Evidano
The team used the thematic analysis option to surface emerging themes with supporting quotes.
- 3Check reliability
Each batch was run three times, an approach the authors likened to inter-rater reliability at the AI level, and themes that recurred across batches were retained.
- 4Verify and finalize by hand
A researcher confirmed each quote existed in the data and matched its theme, then collated the final themes.
“This tool was selected for its accuracy and data security profile.”
“It was decided that, to confirm that AILYZE provides similar types of responses each time, every 150 sets of data responses would be run three times through AILYZE. This process was considered similar to 'inter-rater reliability' at the AI level.”
“The final themes were a result of AI and human analysis.”
Manual coding vs. an AI-assisted workflow
Hand-coding thousands of responses. Slow passes across many questions. Quote-finding done from scratch. Reliability is effortful to repeat.
Themes surfaced across batches fast. Supporting quotes returned with themes. Re-runs make reliability checks simple. Researcher verifies and finalizes.
In the study, data was encrypted and deleted on completion, and the authors cite the tool’s GDPR, CCPA, and HIPAA compliance.
Themes came with supporting quotes the researcher could verify against the source data.
AI surfaced candidate themes; the human researcher interpreted, combined, and finalized them.
The tool, steps, and prompts were documented in the methods and a dedicated appendix.