Fast payoff: a new PLOS One study (published August 20, 2025) hand-codes 6, 500 YouTube/Twitter replies and validates the typology on 2, 004 more: ad hominem is the dominant objection tactic (42–45% of objections). If your team analyzes comment spaces, this paper gives a ready-made codebook and clear frequency signals, and it shows where AI can reduce hours of manual coding. Read the original study: journals.plos.org/plosone/article?id=10.1371/journal.pone.0328550. Want to operationalize these findings quickly? Evidano (www.evidano.com) ingests scraped comments, runs thematic + frequency analyses, and turns the PLOS typology into reproducible reports in hours not weeks.
Fast Take: Why this matters for qualitative analysts
Shea et al. (Published August 20, 2025) develop a seven-category typology of discursive objection tactics found in news comment replies. The team hand-coded 6, 500 direct replies (sample 1, Aug–Oct 2021) and validated frequencies on 2, 004 replies (sample 2, Aug 16, 2022).
- Key finding: ad hominem attacks are the most frequent tactic (about 42.4%–45.2% of objections).
- Objections appeared in ~8–9% of sampled direct replies (8.7% sample 1; 7.8% sample 2).
- Study materials and data are open: see the paper and OSF project (linked above).
Findings snapshot
| Metric | Value | Source / Note |
|---|---|---|
| Published | August 20, 2025 | PLOS One (DOI 10.1371/journal.pone.0328550) |
| Primary sample (direct replies) | 6, 500 coded replies (Aug 31–Oct 22, 2021) | YouTube + Twitter; 14 trending news videos |
| Validation sample | 2, 004 coded replies (Aug 16, 2022) | Top CNN US News videos (first 3 hours) |
| Proportion with objections | Sample 1: 8.7% | Sample 2: 7.8% | Comments containing ≥1 objection tactic |
| Most frequent tactic | Ad hominem: 42.4%–45.2% of objections | Across both samples |
| Intercoder reliability | Krippendorff’s α = 0.74 (sample 2) | 95% agreement after reconciliations |
What happened (plain English)
The authors sampled trending news videos, scraped top parent comments and their direct replies, and conducted collaborative thematic coding to build a typology of seven "discursive objection tactics" (e.g., ad hominem, moral corruption, logical disqualification, physical threat, content threat, space control, self control).
- Phase 1: open & axial coding on 7, 500 replies, refined typology on 6, 500 replies (two trained coders).
- Phase 2: validation on 2, 004 replies from CNN videos; external MTurk testing showed novices can learn the codes but attrition/mistakes make purely crowded scaling costly.
- Key implication: a reliable codebook exists, and some tactics (content threats, logical disqualification) are harder to teach quickly.
What This Means for Researchers: qualitative analysis of social media comments
For UX / community teams
Use the typology to triage moderation signals: ad hominem-dense threads may require different interventions than threads dominated by moral-corrective speech.
Measure changes over time (pre/post policy or UI change) by tracking tactic frequency instead of generic "toxicity" only.
For policy & safety analysts
The distribution of tactics (threat vs. internalized deterrence vs. friction) helps predict escalation risk and potential harm to targets or bystanders.
Pair frequency metrics with network and temporal analyses to see whether certain tactics trigger more replies or spread.
For qualitative teams scaling coding
The PLOS codebook is human-validated but costly to deploy at scale with crowd workers, ideal opportunity to combine supervised AI coding with a human-in-the-loop QA workflow.
Do more, faster with Evidano
Problem: Manual sampling + coding eats weeks
Solution: Evidano scrapes YouTube/Twitter (or ingests CSV exports), deduplicates threads, and auto-segments direct replies for coding.
Problem: Inconsistent code application across coders
Solution: import the PLOS codebook into Evidano, run AI-assisted pre-coding, then assign discrepancies to senior coders for rapid reconciliation.
Problem: Need both counts and qualitative nuance
Solution: Evidano combines thematic extraction, exact frequency counts (per tactic), cross-segment comparisons (by video, time window, or user cohort), and exportable visualizations (co-occurrence networks, hierarchical codes).
Problem: Privacy & compliance
Solution: Evidano encrypts data end-to-end, supports PII redaction, and does not use customer data to train third-party models, suitable for sensitive research workflows.
Two-week pilot workflow (reproduce the PLOS analysis in Evidano)
Day 1: Ingest
Connect Evidano to target videos or upload scraped CSVs of parent comments + direct replies; configure custom dictionary (e.g., slang or obfuscated slurs).
Days 2–4: Auto-code + review
Run AI-assisted pre-coding using the imported PLOS codebook, review model suggestions, and correct edge cases via the AI chat to capture nuanced examples (content threats vs. ad hominem).
Days 5–8: Quantify & segment
Generate tactic frequency tables, cross-segment comparisons (video topic, time window), and co-occurrence networks to see which tactics cluster together.
Days 9–12: Validate & export
Run a small human reliability check (Krippendorff’s alpha) inside Evidano, then export codebook-tagged CSVs and visual reports for stakeholders.
Days 13–14: Share & act
Produce a short findings brief (exec summary + visuals) and hand off action items (moderation triggers, training modules) backed by reproducible data.
Quick FAQ
Can AI reliably distinguish these seven tactics?
AI can pre-classify high-signal cases quickly; PLOS notes some tactics are harder for novices to learn (e.g., content threat). A human-in-the-loop approach yields best reliability.
How do I compare across videos/topics?
Use cross-segment frequency analysis and normalized rates (objections per 1, 000 replies), available as standard outputs in Evidano.
Is this ethical research?
PLOS authors anonymized public comments and obtained IRB exemptions for observation. When reproducing analyses, redact PII and follow platform terms.
Wrap-up & next step
If your team needs reproducible, theory-grounded analysis of comment spaces (converting Shea et al.’s PLOS typology into actionable metrics) you can run the full pipeline in Evidano in under two weeks.
- Start with scraping/ingest → import the PLOS codebook → AI-assisted pre-coding → human QA → deliver visuals and segment comparisons.
- To try a reproducible pilot with your corpus, visit www.evidano.com and request a demo or pilot project tailored to news-comment analysis.
Ethics note: this guidance is for research and moderation design only; treat individual-level threats as potential safety risks and follow institutional reporting procedures.
Keep reading
- Commentary on NewsTwo Definitions: Climate Change Acceptance for UndergradsHow a PLoS One Delphi study (Aug 25, 2026) defined climate change acceptance for undergraduate science students, and how AI-enabled qualitative analysis applies it.
- Commentary on NewsResearcher-in-the-loop: AI-enabled UX researchHow the researcher-in-the-loop model governs AI-enabled UX research. Learn practical governance, stats from the August 2026 piece, and how Evidano supports this workflow.
- Commentary on NewsResearcher-in-the-Loop: Governance for AI UX ResearchGovern AI in qualitative UX research with the researcher-in-the-loop model from Jennifer L. Bowie (Aug 25, 2026): practical rules, risks, and tool mappings.