The primary keyword of this post is qualitative analysis of sports officiating. According to the PLOS One study by Grylls et al. published August 5, 2026, the 2021 Northern Territory Institute of Sport High Performance Officiating Program (HPOP) was evaluated using mixed-methods and idiographic repeated measures to track psychological outcomes across four timepoints in February, May, August, and November 2021. The PLOS One study provides concrete statistics, for example a five-official sample size and individual RCI-backed percentage changes in resilience and self-efficacy, which make it a useful case study for AI-enabled qualitative research workflows. This post explains what the study found, how AI can accelerate and deepen thematic and idiographic analysis, and how Evidano’s features map to problems researchers face when analyzing small, rich datasets.
Key Takeaways
According to the PLOS One evaluation (Grylls et al., 2026), the NTIS HPOP produced reliable improvements in self-efficacy and resilience for several participants, with data collected at four timepoints in February, May, August, and November 2021.
- The PLOS One study sampled five officials in 2021 and reported that three of five officials showed reliable improvements in self-efficacy and resilience by program end (published August 5, 2026).
- Official-level changes included a 37.5% increase in resilience for one participant (Jamie, RCI = 3.99) and a 67.57% increase in mentor-rated self-efficacy for another participant (Alex, RCI = 7.20), as reported in PLOS One on August 5, 2026.
- Data were collected at four fixed timepoints in February, May, August, and November 2021, enabling an idiographic longitudinal design emphasized by the PLOS One authors.
What happened and how the PLOS One study measured change
What Happened: The PLOS One study (Grylls et al., 2026) evaluated the 2021 NTIS High Performance Officiating Program using mixed-methods data from five officials collected across four timepoints in 2021.
How it was measured: The PLOS One authors combined validated psychometrics (for example the CD-RISC10 for resilience and the Referee Self-Efficacy Scale for self-efficacy), visual single-case graphs, Reliable Change Indices (RCI), and thematic analysis of open responses to assess individual-level change, as described in PLOS One (Grylls et al., 2026).
Constraints: The PLOS One report (Grylls et al., 2026) notes a small sample (n = 5), data collection disruptions due to Covid-19, and multimodal program content that limits causal attribution to specific HPOP elements.
Findings snapshot
| Date / Timepoint | Metric | Value | Implication (as stated in PLOS One, 2026) |
|---|---|---|---|
| Feb, May, Aug, Nov 2021 | Study design | 4 repeated measurements per participant | Enables idiographic longitudinal analysis of individual change |
| 2021 cohort / published Aug 5, 2026 | Sample size | 5 officials | Findings are preliminary and not broadly generalisable |
| Baseline → Time 4 (2021) | Jamie: Resilience | 37.5% increase (RCI = 3.99) | Reliable improvement in resilience for Jamie (PLOS One, Grylls et al., 2026) |
| Baseline → Time 4 (2021) | Alex: Mentor-rated self-efficacy | 67.57% increase (RCI = 7.20) | Strong mentor-observed growth in self-efficacy for Alex (PLOS One, Grylls et al., 2026) |
Implications for qualitative researchers evaluating small program cohorts
Implication: The PLOS One study shows that mixed-methods idiographic designs can surface meaningful individual change even with small n, but they create heavy transcription, coding, and cross-case synthesis work for researchers.
Practical takeaways: The PLOS One authors used thematic analysis with an independent coder and Excel-based meaning units; the report recommends neutral interview protocols and independent data collection in future work to minimise bias (Grylls et al., 2026).
Analytical challenge: The PLOS One study (Grylls et al., 2026) reports many participant-level numerical RCIs and qualitative excerpts such as Alex’s comment, “I feel more confident in my abilities. I have been given opportunities to ref high level games with high level referees. Benchmarking against other referees has been good for me”, which require integrated coding and quick cross-referencing to draw program-level claims.
How Evidano helps: AI-enabled qualitative workflows for program evaluations
Problem: transcription and PII when analysing rich interviews
Answer: Transcription bottlenecks slow analysis for small-cohort evaluations, and the PLOS One study highlights multiple interview rounds across four timepoints (Feb, May, Aug, Nov 2021) that produced many open responses (Grylls et al., 2026).
Evidano feature mapping: Evidano is an AI-powered qualitative data analysis platform that helps researchers analyze interviews, open-ended surveys, and documents. Evidano’s transcription tools (including custom dictionaries and PII redaction) speed creation of clean transcripts from audio, which is useful when evaluating programs like the NTIS HPOP.
Problem: coding consistency and idiographic synthesis
Answer: Single-case idiographic approaches require consistent coding of meaning units across timepoints to compute RCIs and to map themes to individual trajectories, as the PLOS One authors did using Excel and independent coding (Grylls et al., 2026).
Evidano feature mapping: Evidano automates thematic and code co-occurrence analysis, supports hierarchical codes→subcodes, and produces visualizations such as co-occurrence networks that make it faster to compare participant-level themes to aggregate patterns. See the Evidano features page for relevant capabilities.
Problem: linking quantitative RCIs with qualitative quotes
Answer: The PLOS One study presents paired quantitative RCIs and qualitative excerpts (for example Jade: “My resilience has grown beyond where I started. And I thought I would ever get to. I can deal with confrontation better and bounce back from any setbacks much quicker”), which requires traceable links between numbers and the text that explains them (Grylls et al., 2026).
Evidano feature mapping: Evidano’s document linking and AI chat over your documents allow researchers to query, for example, “Which participants with RCI > 3.0 mentioned mentoring as a factor? ” and immediately retrieve relevant quotes, frequency counts, and cross-segment comparisons.
Problem: reproducible thematic methods and audit trails
Answer: The PLOS One authors used a six-stage thematic analysis with an independent coder to ensure rigor; reproducibility requires an audit trail that connects raw transcripts to theme labels and to final visualizations (Grylls et al., 2026).
Evidano feature mapping: Evidano maintains encrypted data, versioned codebooks, and exportable analysis logs so teams can reproduce thematic decisions and satisfy peer-review or stakeholder transparency requirements. For research-focused security detail see Evidano data security.
FAQ: qualitative analysis of sports officiating
How can AI speed thematic analysis of program evaluations like the NTIS HPOP?
Answer: AI can accelerate transcript cleanup, code suggestion, and cross-case theme frequency counts in minutes rather than days.
Supporting detail: The PLOS One evaluation (Grylls et al., 2026) relied on manual thematic coding of multiple open questions across four timepoints; AI-assisted pipelines reduce manual coding time, surface co-occurrence patterns, and let researchers focus on interpretation and validity checks.
Can AI preserve idiographic rigor when analyzing n = 5 cohorts?
Answer: Yes, if AI tools support per-case tagging, traceable quotes, and RCI-linked metadata rather than only aggregate summaries.
Supporting detail: The PLOS One study (Grylls et al., 2026) emphasises idiographic presentation (participant-by-participant RCIs and quotes). AI platforms that retain case-level linkage and allow export of coded excerpts preserve that rigor.
What are reasonable expectations when combining quantitative RCIs and qualitative themes?
Answer: Expect complementary evidence: RCIs indicate statistical reliability of score change while thematic excerpts explain the mechanisms or context behind those changes.
Supporting detail: Grylls et al. (PLOS One, 2026) paired numeric RCIs (for example 37.5% resilience gain, RCI = 3.99) with participant statements about mentoring and opportunity to explain how changes may have occurred.
Is the PLOS One HPOP evaluation generalisable to other jurisdictions?
Answer: No, the PLOS One authors caution limited generalisability because the sample was small (n = 5) and context-specific to the Northern Territory program (Grylls et al., 2026).
Supporting detail: The PLOS One paper explicitly lists the small cohort, Covid-19 disruptions, and multimodal program delivery as limitations that reduce generalisability beyond the studied cohort.
Conclusion & Next Steps
The PLOS One evaluation (Grylls et al., 2026) demonstrates how mixed-methods idiographic designs can detect reliable individual changes in psychological outcomes for sports officials, while creating a heavy analytic burden for researchers.
AI-enabled qualitative research tools can reduce that burden by automating transcription, suggesting codes, and linking quantitative indices to qualitative excerpts so teams can test hypotheses faster and produce audit-ready reports.
If you run program evaluations with small, rich cohorts and want to accelerate thematic synthesis while preserving idiographic rigor, Try Evidano for free.
Topics
- qualitative analysis of sports officiating
- AI qualitative analysis
- thematic analysis sports officials
- HPOP evaluation
Keep reading
- Commentary on NewsFrontline Voices: Qualitative Analysis of Conflict ReportingAI tools to extract themes, quotes, and counts from frontline reporting on Lebanon’s paramedics. Guide to qualitative analysis of conflict reporting. Try it.
- Commentary on NewsAI Qualitative Analysis of Indigenous Health CampaignsHow to analyze Indigenous tobacco-control campaigns with AI qualitative analysis. Learn methods, stats from ABC News (AU), and how Evidano speeds synthesis.
- Commentary on NewsAI-enabled Interprofessional Learning AnalysisTurn interviews and ward notes into actionable hospital insights: interprofessional learning analysis for nurses and physiotherapists. Learn methods and tools to act.