What the study analysed
The mixed-methods study examined which leadership credentials the public values when evaluating police chief candidates. A discrete-choice conjoint experiment with 303 respondents was fielded in April 2026, followed roughly two weeks later by an open-ended survey asking respondents to explain how they distinguished between pairs of applicant qualifications.
The follow-up produced 224 complete responses across two questions, each at least 100 characters. Thematic analysis of the professional standards question produced five themes; the evidence-based policing question produced four.
Manual coding first, AI as the cross-check
The first author read every response, generated codes inductively, and built, refined, and defined the themes by hand. Only then was the qualitative data run through Evidano, to triangulate those findings and to help identify oversights, alternative frames, and personal biases.
Eight of the nine human-coded themes were supported by the AI-assisted analysis with only minor differences in wording, and the ninth was folded into two others as a countervailing perspective. Evidano also surfaced two candidate themes the human coding had not highlighted, one of which the team incorporated as a caveat and one it rejected because the underlying responses did not cohere. All coding and thematic interpretations were reviewed and validated by the research team.
The paper explains why the tool was chosen: five recent peer-reviewed studies had demonstrated its ability to support accurate, low-bias analysis when used with human oversight.
How Evidano fit the workflow
A human-led thematic analysis, cross-checked rather than replaced.
- 1Open-ended responses collected
224 complete responses to two short-answer questions, following a 303-respondent conjoint survey.
- 2Inductive coding by the first author
Codes were generated from the data, then renamed, combined, or split as new meanings emerged.
- 3Themes generated, refined, and defined
Five themes on professional standards, four on evidence-based policing.
- 4Evidano used to triangulate
A second pass to surface oversights, alternative frames, and personal biases.
- 5Research team reviewed and validated
All coding and thematic interpretations were checked by the team, which kept the final call on every theme.
“we analyzed our qualitative data using the online AI-powered qualitative analysis software Evidano (formerly known as AIlyze) to triangulate the findings from our thematic analysis and to "help identify oversights, alternative frames, and personal biases"”
“Evidano was selected because recent qualitative studies […] have demonstrated its ability to support accurate, low-bias qualitative analysis, particularly when used with human oversight.”
“All four themes from the human-coded analysis […] were supported by the AI-assisted analysis with only minor differences in wording.”
“Accordingly, all coding and thematic interpretations were reviewed and validated by the research team.”
What a triangulation pass adds
One analyst coding hundreds of free-text responses carries their own framing into every theme, with nothing to test it against.
An independent pass over the same dataset that surfaces oversights and alternative frames before the findings are published.
The study names Evidano, cites it, and explains why it was selected.
Themes are identified across the entire dataset rather than response by response.
User data is confidential and never used to train models.
Every coding and thematic interpretation was reviewed and validated by the research team.
About Evidano (previously AILYZE)
Evidano is AI-assisted qualitative data analysis software for interviews, focus groups, open-ended survey responses, and documents. It supports AI-assisted thematic analysis with your own or an AI-suggested codebook, transcription and translation in 100+ languages, clickable quotes and citations, and exportable visual reports. End-to-end encryption, no third-party data sharing, and your data is not used to train AI. From USD 50/month for unlimited analysis.
Run the first pass in days, not weeks
Keep the judgment with you. Let Evidano do the heavy lifting on your interviews, focus groups, open-ended responses, and documents.
More case studies
Government & Public SectorIllinois Board of Higher Education & ICCB: AI theme analysis across 62 institutions
Evidano surfaced themes across six quarters of data from a 62-institution consortium, with report staff re-reading all raw data to validate every finding.
Government & Public SectorENAP and UBC used AI-assisted qualitative coding on 39,788 federal survey comments
A peer-reviewed, open-access study coded 39,788 open-ended comments against a predefined thematic grid, then manually verified a random 15% sample.
Government & Public SectorENAP and CUNY used AI-assisted qualitative analysis on 5,211 federal-worker comments
A 2026 study coded 5,211 federal-worker comments against a seven-mechanism framework with AI, then reviewed every result by hand.
Related published evidence
Researcher interviews and published citations covering comparable studies.
- Published research2024
William & Mary’s AI thematic analysis of 19,000+ open-ended responses from 3,443 adults
AI-assisted thematic analysis of 12 open-ended questions in batches of 150 responses, each run three times and verified quote by quote by a human researcher.
3,443 participants in the study
Research report by the William & Mary Center for Gifted Education for the Mensa Foundation, May 2024 (W&M ScholarWorks)
- Published research2024
Monash University study trialled AI thematic analysis to cross-check its themes
See how doctoral education researchers trialled Evidano for thematic analysis, with human researchers leading every interpretation.
23 doctoral writers’ responses analysed, from 29 recruited
Studies in Educational Evaluation, vol. 83, 101379 (Elsevier, 2024), open access
Explore more evidence
Researcher interviews, the wording authors published when citing Evidano, and accuracy comparisons against manual coding.
- Resource
Research spotlights: researchers in their own words
Interviews with researchers at leading universities and institutes about how they actually used Evidano in work they went on to publish.
- Resource
How Evidano was used, in published methods sections
Verbatim excerpts from 20 published studies describing what Evidano did in their analysis — as a thought partner, a second coder, for initial coding, or for synthesis.
- Resource
Methodology guides and research writing
Practical guides to qualitative methods — thematic analysis, grounded theory, evidence synthesis and more — alongside the wider Evidano article library.
- Resource
Human vs AI: validated accuracy benchmarks
Three head-to-head comparisons against expert human analysis — 371 interview transcripts with Arizona State and Penn State, 298 evaluation reports for a UN evaluation group, and UNICEF’s manual-coding review.