AI-Assisted Thematic Analysis: NTIS HPOP Evaluation

Evidano is an AI-powered qualitative data analysis platform that helps researchers analyze interviews, open-ended surveys, and documents. According to the PLoS One article published August 5, 2026, the Northern Territory Institute of Sport (NTIS) evaluated its 2021 High Performance Officiating Program (HPOP) using a mixed-methods, idiographic design with five participating officials. According to the PLoS One article, that evaluation combined four-timepoint quantitative measures and thematic analysis of open responses to report individual changes such as a 37.5% resilience increase for one official and mentor-rated self-efficacy increases as high as 67.57% for another official. This post explains, step by step, how AI-enabled qualitative research methods can reproduce, scale, and extend the PLoS One HPOP thematic analysis to generate actionable program insights.

Key Takeaways

According to the PLoS One article published August 5, 2026, the 2021 NTIS High Performance Officiating Program (HPOP) produced reliable improvements in self-efficacy and resilience for three of five officials, combining quantitative Refs/RCI and thematic analysis to triangulate outcomes (PLoS One).

Three of five officials showed reliable improvements in self-efficacy and resilience between February and November 2021, according to PLoS One (published August 5, 2026).
Official-level changes included a 37.5% resilience increase for Official 1 (RCI = 3.99) and a 67.57% mentor-rated self-efficacy increase for Official 2 (RCI = 7.20), per PLoS One.
The PLoS One study used four measurement timepoints (February, May, August, November 2021) and an idiographic single-case approach to preserve individual patterns of change.
Direct participant quotes in PLoS One reinforced quantitative trends, for example Alex said, "I feel more confident in my abilities. I have been given opportunities to ref high level games with high level referees."

What happened and how the PLoS One study measured it

Answer: The PLoS One study evaluated psychosocial outcomes of the 2021 NTIS HPOP using repeated measures and thematic analysis to capture individual trajectories.

According to the PLoS One article published August 5, 2026, six officials applied to the 2021 HPOP and five completed the study, with data collected at four timepoints: February 2021 (baseline), May 2021, August 2021, and November 2021.

According to the PLoS One article, quantitative measures included the CD-RISC10 for resilience, the Referee Self-Efficacy Scale (REFS) with subscales (game knowledge, decision-making, pressure, communication), the Cognitive Appraisal Scale for challenge and threat, Rotter’s Locus of Control, and the Functional Assertiveness Scale.

According to the PLoS One article, the researchers used percentage change, mean change, and Reliable Change Indices (RCIs) to determine whether individual changes exceeded measurement error (RCI thresholds |RCI| > 1.96).

According to the PLoS One article, qualitative open questions were analysed via a six-stage inductive thematic analysis to create higher-order themes (officiating self-efficacy and officiating opportunities) and five lower-order themes (resilience, technical skills, pre/post strategies, adaptive responses, social support).

Findings snapshot

Date/Timepoint	Metric (source)	Value (per PLoS One)	Implication
February–November 2021 (4 timepoints)	Participants completing study (PLoS One)	n = 5 officials	Idiographic repeated-measures design preserved individual trajectories
Published August 5, 2026	Study report (PLoS One)	Open access article with mixed-methods dataset	Enables replication and secondary analysis of transcripts and RCIs
Baseline to Time 4 (per PLoS One)	Officials with reliable self-efficacy/resilience gains	3 of 5 officials	Program-level signal but strong individual variability
Baseline to Time 4 (per PLoS One)	Example individual RCI: resilience (Official 1)	+37.5% (RCI = 3.99)	Large, reliable individual increase in resilience
Baseline to Time 4 (per PLoS One)	Example individual RCI: mentor-rated self-efficacy (Official 2)	+67.57% (RCI = 7.20)	Substantial mentor-observed confidence gains; cross-source triangulation

Implications for qualitative researchers and program evaluators

Answer: The PLoS One HPOP evaluation shows that combining idiographic quantitative tracking with thematic analysis yields richer program insights than either method alone.

According to the PLoS One article published August 5, 2026, the idiographic approach preserved meaningful individual differences, which is essential for small-cohort program evaluations where averages can hide diverging participant responses.

According to the PLoS One article, triangulating RCIs with participants’ open-text responses increased confidence in program effects, for example the participant Alex stated, "Benchmarking against other referees has been good for me, " linking workshop exposure to perceived mastery.

According to the PLoS One article, program teams should plan for: repeated measurement across at least three timepoints, matched qualitative prompts focused on target constructs (self-efficacy, resilience), and independent thematic coding to reduce facilitator bias.

How Evidano helps reproduce and scale PLoS One–style analyses

Problem: small samples, high individual variability

Solution: Evidano automates idiographic workflows so evaluators can run per-participant time series and Reliable Change Index calculations without manual spreadsheets.

Evidano is an AI-powered qualitative data analysis platform that helps researchers analyze interviews, open-ended surveys, and documents. Evidano features include per-respondent thematic extraction and visualized timelines that mirror the PLoS One idiographic presentation.

Problem: linking quotes to quantitative change

Solution: Evidano ingests transcripts and numeric timepoint data, then produces linked evidence cards that show the exact quotes (with source attribution), timepoint, and associated RCI or percent change.

According to the PLoS One approach, quotes such as Alex’s "I feel more confident in my abilities" are powerful when paired with mentor-rated self-efficacy increases (e.g., +67.57%); Evidano automates that pairing for reporting and stakeholder slides.

Problem: reproducibility and audit trails

Solution: Evidano keeps encrypted audit trails of coding decisions, theme versions, and original source files to support transparent replication of mixed-methods evaluations like the PLoS One HPOP study.

Evidano’s transcript and survey ingestion plus AI chat over your documents accelerate replication of the six-stage thematic analysis workflow used in the PLoS One article.

FAQ: ai-assisted thematic analysis

Can AI help thematic analysis in small-sample program evaluations?

Yes, AI can assist thematic analysis for small samples by standardizing coding, surfacing recurring meaning units, and linking quotes to timepoint metadata.

According to the PLoS One article published August 5, 2026, the authors used an inductive thematic analysis on five officials and benefited from an independent coder to reduce bias; AI can replicate the independent coding step and speed up codebook iterations.

How can I reproduce Reliable Change Indices (RCI) with qualitative context?

RCIs are reproducible by combining pre/post scores, test reliability, and the standard error of the difference, then anchoring significant RCIs to participants’ qualitative statements.

According to the PLoS One article, RCIs greater than |1.96| indicated reliable change; an AI workflow can compute RCIs for each participant and attach supporting quotes when an RCI crosses that threshold.

Do AI tools risk biasing thematic outcomes when the program lead is also an analyst?

AI reduces some forms of bias by offering neutral initial coding and by enabling a separate reviewer to inspect and modify themes, but human oversight remains essential.

According to the PLoS One article, the lead author delivered some HPOP sessions and the team recommended independent interviews and neutral coding; Evidano supports reviewer permissions and blinded coding rounds to mitigate that risk.

What minimum data should evaluators collect to replicate the PLoS One mix-methods approach?

Collect baseline and at least two follow-up numeric measures, matched open-ended questions at each timepoint, and mentor or external ratings for triangulation.

According to the PLoS One article, the study used four timepoints (February, May, August, November 2021), paired quantitative instruments (CD-RISC10, REFS) and open questions: this schedule produced interpretable RCIs and rich thematic evidence.

Conclusion & Next Steps

Answer: The PLoS One evaluation of the 2021 NTIS HPOP demonstrates that pairing idiographic quantitative tracking with rigorous thematic analysis produces actionable program insights for small cohorts.

According to the PLoS One article published August 5, 2026, the study combined RCIs and participant quotes to show meaningful gains in self-efficacy and resilience for several officials, and the same logic scales with AI-assisted workflows.

If you run mixed-methods program evaluations and want to reproduce PLoS One–style idiographic reports faster, consider an AI-enabled platform that preserves quotes, computes RCIs, and produces transparent theme audits.

Try Evidano for free to ingest transcripts, compute per-participant RCIs, and generate linked thematic evidence cards: Try Evidano for free.

Topics

ai-assisted thematic analysis
AI-enabled qualitative analysis
program evaluation thematic analysis
thematic analysis sports officiating

Keep reading

Browse all articles

AI-Assisted Thematic Analysis: NTIS HPOP Evaluation

Key Takeaways

What happened and how the PLoS One study measured it

Findings snapshot

Implications for qualitative researchers and program evaluators

How Evidano helps reproduce and scale PLoS One–style analyses

Problem: small samples, high individual variability

Problem: linking quotes to quantitative change

Problem: reproducibility and audit trails

FAQ: ai-assisted thematic analysis

Can AI help thematic analysis in small-sample program evaluations?

How can I reproduce Reliable Change Indices (RCI) with qualitative context?

Do AI tools risk biasing thematic outcomes when the program lead is also an analyst?

What minimum data should evaluators collect to replicate the PLoS One mix-methods approach?

Conclusion & Next Steps

Topics

Keep reading

Company

About

Preview Evidano

Account

Newsletter