Fast payoff: If you need to reproduce the PLOS One mixed-methods protocol (published 18 June 2026) that combines NLP web-scraping, administrator interviews, and student focus groups into a standardized evaluation framework, this post lays out a repeatable, AI-enabled workflow, and shows how to run it in Evidano. The primary takeaway: use automated scraping and embedding-clustering to map institutional messaging (Phase I, n=50 schools), then triangulate with interviews (≥25 administrators) and 15–24 student focus group participants to build a robust qualitative framework. See the original protocol at PLOS One and explore how to operationalize the same pipeline on Evidano.
Key Takeaways
Evidano is an AI-powered qualitative data analysis platform that can ingest scraped webpages, transcribe interviews with PII redaction, and run embedding-based clustering to reproduce the PLOS One protocol's Phase I–IV workflow.
This protocol starts with an NLP-assisted web content analysis of public wellbeing pages (Phase I, 50 LCME MD schools), then uses ≥25 administrator interviews and 15–24 student focus group participants to produce a standardized evaluation framework.
- The PLOS One protocol was published on 18 June 2026 and specifies Phase I as a randomized web sample of 50 LCME-accredited MD schools.
- Phase II plans for at least 25 one-hour semi-structured key informant interviews, and Phase III plans 3 focus groups totaling 15–24 students.
- Phase I computational specs include all-miniLM-L6-v2 embeddings, UMAP (n_neighbors=15, min_dist=0.1, n_components=2, random_state=42), and HDBSCAN (min_cluster_size=5, min_samples=3).
Fast take + source
Fast take: Godbole et al. published a sequential-exploratory, mixed-methods protocol on 18 June 2026 that begins with an NLP-assisted web content analysis of wellbeing pages (Phase I), then moves to key informant interviews (Phase II) and student focus groups (Phase III) to produce a standardized evaluation framework.
- Original source: PLOS One
- IRB: NYU Langone Health (IRB ID: i25-00965)
- Why this matters: combines scalable computational capture of public messaging with rich qualitative context, a template many teams can reuse.
Study snapshot (quick numbers & timeline)
| Date / Item | Metric / Detail | Value / Parameter | Source / Note |
|---|---|---|---|
| Protocol published | Date | 18 June 2026 | PLOS One |
| Phase I sample | Web scraping (random sample) | 50 LCME-accredited MD schools | Content analysis of public webpages |
| Phase II sample | Key informant interviews | ≥25 administrators (1 per institution) | 1-hour semi-structured KIIs; purposive sampling |
| Phase III sample | Focus groups | 3 FGDs; 15–24 students total | Groups of 5–8, stratified by training stage |
| Computational specs | Embedding / clustering / models | all-miniLM-L6-v2; UMAP (n_neighbors=15, min_dist=0.1, n_components=2, random_state=42); HDBSCAN (min_cluster_size=5, min_samples=3) | Planned pipeline in Phase I |
| Timeline | Recruitment / Data / Results | Recruit Feb–May 2026; data collection complete Aug 2026; results by Apr 2027 | Protocol status |
| Ethics | Approval | NYU Langone IRB i25-00965 | Research-only; non-diagnostic |
How the mixed-methods workflow for qualitative analysis of medical student wellbeing programs works
This section explains the workflow: Phase I uses automated crawling and sentence-level embedding to extract program descriptions from public medical school webpages, Phase II collects administrator rationales via semi-structured interviews, Phase III runs focus groups to surface student lived experience, and Phase IV triangulates these sources to produce a standardized evaluative framework.
- Phase I: Python crawler (requests and BeautifulSoup), sentence-tokenize, embed with all-miniLM-L6-v2, cosine similarity ranking, UMAP, HDBSCAN, and c-TF-IDF keywords for cluster labels.
- Phase II: Purposive key informant interviews (≥25), verbatim transcripts, double-coding, Cohen’s kappa for inter-rater reliability, and an iterative codebook using inductive and deductive approaches.
- Phase III: Three focus group discussions (5–8 participants each), moderated sessions with field notes, thematic analysis via Dedoose, and comparison of student and administrator themes.
- Phase IV: Joint displays, comparative matrices, and expert panel review to produce a validated framework for program evaluation mapped across preventative, reactive, structural, and cultural domains.
Implications for researchers, program leads, and evaluators
For mixed-methods researchers
For mixed-methods researchers: Use web-scraping as an efficient census of public messaging to reduce non-response bias from survey-only approaches and to inform purposive sampling for interviews.
Mixed-methods researchers should predefine embedding similarity thresholds and manually review low-confidence extractions to avoid missing domain-specific language.
For medical school wellbeing administrators
For medical school wellbeing administrators: Public webpages shape student expectations and computational mapping can reveal mismatches between messaging and available services across institutions.
Administrators can use triangulated insights from web content, administrators, and students to identify cultural or structural gaps that blunt program effectiveness.
For qualitative teams and UX researchers
For qualitative teams and UX researchers: Treat the computational phase as instrument development, using extracted common language and gaps to craft interview and FGD guides that probe real-world friction points.
Qualitative teams should plan for codebook iteration and explicit documentation of coding decisions; the protocol describes double-coding and kappa thresholds to support rigor.
Do more, faster with Evidano (mapped to this protocol)
Problem: Scalable web-based content capture
Evidano can ingest scraped HTML, preserve metadata (URL, capture date), and run sentence-level embedding and similarity ranking so you can replicate Phase I without a custom Python pipeline.
Use Evidano to import scraped pages directly and retain context for later triangulation with interviews and FGDs.
Problem: Extracting themes and clusters reproducibly
Evidano performs thematic and frequency analysis, hierarchical code to subcode visualizations, and co-occurrence networks, all exportable for joint displays and expert review.
Teams can use Evidano's exports to populate comparative matrices and presentable visualizations for Phase IV synthesis.
Problem: Interview & FGD transcription and PII safety
Evidano provides transcription with custom dictionaries and PII redaction to produce verbatim transcripts ready for coding while protecting participant privacy.
Using Evidano transcription helps operationalize the protocol's requirement for secure, research-only data handling and IRB-sensitive safeguards.
Problem: Cross-segment comparisons (admins vs students vs sites)
Evidano’s cross-segment analysis and segment filters let you quantify theme prevalence by cohort, for example preclinical versus clinical students, and produce charts for stakeholder meetings.
Cross-segment filters in Evidano support the protocol's Phase IV goal of triangulating across sources to map domains like preventative and structural programming.
Security & compliance
Evidano uses encrypted storage and proprietary LLMs tuned for qualitative research, and user data is not used to train third-party models, which is critical for IRB-sensitive research.
Teams should document storage and redaction practices when using Evidano to match protocol-level ethics notes and IRB requirements.
Quick integration
Evidano lets you import your codebook, run AI-assisted coding to speed double-coding, and export inter-coder reports (kappa) and joint displays for Phase IV synthesis.
This integration helps teams follow the protocol's recommended sequence from computational extraction to expert-panel review.
Checklist: 7-step workflow to reproduce this protocol in two weeks (pilot)
Checklist: Use these seven pragmatic steps to pilot the study pipeline with Evidano in a short timeframe.
- 1) Assemble a seed list of 50 LCME MD school URLs and randomize with a fixed seed.
- 2) Scrape public wellbeing pages and import HTML and metadata into Evidano.
- 3) Run sentence-tokenization, embeddings, and clustering; flag low-confidence pages for manual review.
- 4) Draft key informant interview and FGD guides using language surfaced by the computational phase.
- 5) Record three pilot interviews and one pilot FGD; transcribe with PII redaction and a custom dictionary.
- 6) Import transcripts into Evidano, run AI-assisted coding, double-code a sample, compute kappa, and finalize the codebook.
- 7) Produce joint displays and a short evaluative rubric mapped to preventative, reactive, cultural, and structural domains; prepare for expert panel review.
Wrapping up: What to do next
Wrapping up: If you are running a similar mixed-methods evaluation, the PLOS One protocol provides concrete parameters (models, clustering, sample sizes, and timeline) that you can operationalize now.
Ethics note: the protocol is non-diagnostic and IRB-approved; researchers must preserve consent and PII safeguards throughout data collection and analysis.
Ready to try the full pipeline: start a pilot by importing scraped pages and interview transcripts into Evidano; Evidano handles transcription with a custom dictionary, thematic and cross-segment analysis, and visual exports for stakeholder review.
- Original study: PLOS One
- Next move: run Phase I on a 10-school subset in Evidano this week and schedule your expert-panel synthesis for Month 2.
If you want to get started right away, Try Evidano for free.
FAQ: medical student wellbeing program evaluation
What does the PLOS One protocol include?
Answer: The protocol includes a sequential-exploratory, mixed-methods design with an NLP-assisted web content analysis (Phase I), key informant interviews (Phase II), focus groups (Phase III), and triangulation into a standardized evaluation framework (Phase IV).
The protocol specifies a randomized web sample of 50 LCME-accredited MD schools for Phase I, at least 25 administrator interviews for Phase II, and three focus groups totaling 15–24 students for Phase III.
How many schools and participants are planned in the study?
Answer: The study plans a Phase I sample of 50 LCME-accredited MD schools, Phase II with at least 25 administrators, and Phase III with 3 focus groups totaling 15–24 students.
The timeline in the protocol lists recruitment from February to May 2026, data collection complete by August 2026, and results by April 2027.
What computational methods does Phase I use?
Answer: Phase I uses sentence-level embeddings with all-miniLM-L6-v2, UMAP for dimensionality reduction, HDBSCAN for clustering, and c-TF-IDF for cluster keyword labeling.
The protocol also recommends cosine similarity ranking and manual review of low-confidence extractions to ensure domain-specific language is captured.
Is the study IRB-approved and non-diagnostic?
Answer: Yes, the protocol notes NYU Langone Health IRB approval with ID i25-00965 and categorizes the work as research-only and non-diagnostic.
Teams using similar pipelines should preserve consent and PII safeguards consistent with the protocol and IRB requirements.