Evidano is an AI-powered qualitative data analysis platform that ingests full-text research articles, preserves document sections, applies structured prompting and tuned adapters, and produces auditable, exportable extraction outputs. Fast, repeatable extraction from full-text research articles is a core headache for researchers and UX/insight teams. A June 24, 2026 PLoS ONE study shows that narrow, task-specific model adaptation changes the balance between precision and recall when answering 16 predefined questions across full-text biomedical articles. The study fine-tuned models on 250 HIV drug-resistance papers and evaluated performance on 150 held-out papers (2400 question instances), finding that fine-tuning shifted outputs toward higher precision while structured, question-specific prompting increased recall. If your team runs literature abstraction, thematic synthesis, or evidence curation, you will learn which strategy maps to which outcome and how Evidano operationalizes both approaches securely, ingesting full texts, running thematic and cross-segment analyses, and giving auditable, exportable results.
Key Takeaways
Fine-tuning improves precision while structured, question-specific prompting increases recall when extracting facts from full-text biomedical articles.
A hybrid workflow that uses question-specific prompting to surface candidates and a fine-tuned model or human review to remove false positives captures the strengths of both approaches.
- The PLoS ONE study (June 24, 2026) fine-tuned models on 250 HIV drug-resistance papers and tested on 150 held-out papers (2, 400 question instances).
- Fine-tuning increased precision (examples reported: +5% for GPT-4o; +16% for Llama-3.1-70B), while question-specific prompting increased recall (examples reported: +6% for GPT-4o; +18% for Llama-3.1-8B).
- Combining question-specific prompting after fine-tuning added little extra benefit in the paper; applied teams should run QSP passes for sensitivity, then prune with higher-precision models or human QA.
In brief: what the PLoS ONE study tested
The PLoS ONE study (published June 24, 2026) tested fine-tuning versus question-specific structured prompting for question answering over full-text biomedical articles.
The paper converted full texts into standardized markdown and compared model modes across a curated instruction set and question-specific prompts, see the full paper at PLOS ONE.
- Training set: 250 curated HIV drug-resistance studies (instruction set).
- Test set: 150 held-out papers; 16 fixed questions per paper (total 2, 400 question checks).
- Models: GPT-4o, Llama-3.1-70B, Llama-3.1-8B; evaluation by accuracy, precision, recall, F1.
Findings snapshot
| Date | Training / Test | Questions | Primary outcomes | Source |
|---|---|---|---|---|
| June 24, 2026 | 250 fine-tune papers / 150 test papers | 16 per paper (Boolean, list, numeric) | Fine-tuning increased precision (e.g., +5% GPT-4o; +16% Llama-70B). Question-specific prompting increased recall (e.g., +6% GPT-4o; +18% Llama-8B). | PLOS ONE |
What the study did (plain English)
The study standardized full-text articles, paired 250 papers with 16 labeled questions and explanations, then compared fine-tuning and consolidated question-specific prompt rules across modes.
The authors converted each paper into standardized markdown (title, abstract, methods, results, figures, data-sharing statements) and created an instruction set pairing 250 papers with 16 labeled questions and explanations.
- Fine-tuning was done via the OpenAI API for GPT-4o and QLoRA adapters for Llama models, and consolidated question-specific prompt rules were applied separately.
- Evaluation used mean accuracy, precision, recall, F1 across 16 questions averaged over 150 test papers, with paired Wilcoxon tests and false discovery rate control.
- Key numeric context: median training sample = 5, 503 tokens; learning-curve checks at 50/100/150/200/250 instruction samples.
- Notable results: fine-tuning tended to improve precision, question-specific prompting tended to increase recall, and combining question-specific prompting after fine-tuning added little extra benefit.
So what for researchers and UX/insight teams?
For systematic-review teams
For systematic-review teams, fine-tuning improves false-positive control and leans outputs toward higher precision.
If you prioritize false-positive control (clean datasets for downstream meta-analysis), fine-tuning on a curated instruction set produces more consistent gains.
Expect to invest in a labeled instruction set, the study used 250 papers, to achieve those precision improvements.
For rapid screening or sensitivity-first tasks
For rapid screening or sensitivity-first tasks, structured, question-specific prompting recovers more true positives and increases recall.
If recall (finding every potential instance) matters more (for safety surveillance, broad literature catchment, or exploratory synthesis), structured, question-specific prompting recovered more true positives with modest trade-offs in precision.
Question-specific prompting is lighter to deploy and can be iterated quickly against error patterns.
For product teams and UX researchers
For product teams and UX researchers, hybrid workflows make sense: use QSP to surface candidates, then apply a fine-tuned model or human QA to remove false positives.
Hybrid workflows combine QSP-backed passes to surface candidates, then apply a fine-tuned model or human QA to remove false positives.
Track per-question F1 scores, the study shows persistent failure modes such as cloning and nuanced treatment details that require targeted QA or additional training examples.
Do more, faster with Evidano (mapped to this use case)
Ingest full texts reliably
Evidano ingests PDFs and repository content and converts full-text articles into analysis-ready documents while preserving sections where key data appear.
Upload PDFs, batch-scrape journal sites, or pull from repositories; Evidano converts full-text articles into analysis-ready documents while preserving sections (methods, results, figure legends) that the study shows are often where data hide.
Reproduce fine-tuning-like gains without heavy ops
Evidano provides domain adapters and guided QA templates to create instruction-style examples without building bespoke LoRA pipelines.
Use Evidano’s domain adapters and guided QA templates to create instruction-style examples like the 250-sample set, export training sets, track learning curves, and iterate on examples that fix your failure modes.
Apply structured prompting and hybrid passes
Evidano supports layering question-specific prompt rules as a first-pass sensitivity sweep, then applying a higher-precision adapter or human reviewer to flagged items.
Layer question-specific prompt rules as a first-pass sensitivity sweep, then run the higher-precision Evidano adapter or a human reviewer on flagged items, the exact hybrid the PLoS ONE study recommends.
Auditability & security
Evidano provides clickable source highlights and exportable codebooks so every extracted fact can be traced to its sentence or table.
Outputs include clickable source highlights and exportable codebooks, making it trivial to trace every extracted fact to the sentence or table that produced it.
Data is encrypted at rest and in transit and Evidano does not use your content to train third-party models.
Visualize and compare
Evidano offers one-click thematic, frequency, co-occurrence network and hierarchical code→subcode visualizations to spot reporting gaps across a corpus.
One-click thematic, frequency, co-occurrence networks and hierarchical code→subcode views help you spot systematic reporting gaps the study flagged across your corpus and segments.
2-week pilot: checklist to reproduce these ideas
This two-week checklist shows a minimal experiment to reproduce the trade-offs the paper reports in your domain.
- Week 1: Assemble 50–150 labeled documents and 10–20 domain-specific questions. Ingest into Evidano and run base extraction.
- Week 2: Apply structured question prompts (QSP-style) and compare recall; then apply Evidano’s tuned adapters or curated instruction examples to measure precision gains. Use Evidano visualizations to locate consistent error types and iterate.
- Deliverable: per-question precision/recall table and traceable source highlights for 10 representative errors.
Ethics & safeguards (brief)
The study used public articles and did not involve human subjects, and projects with sensitive documents should maintain informed consent and PII redaction.
When you apply AI-assisted extraction to sensitive documents, maintain informed consent and consider PII redaction. Evidano supports PII redaction in transcription and secure, private model usage.
FAQ: Qualitative analysis of biomedical literature
What exactly did the PLoS ONE study compare and when was it published?
The PLoS ONE study published on June 24, 2026 compared fine-tuning and question-specific structured prompting for question answering over full-text biomedical articles.
The paper fine-tuned models on 250 HIV drug-resistance papers and evaluated them on 150 held-out papers with 16 predefined questions per paper (2, 400 question instances), see PLOS ONE.
Which approach improves precision and which improves recall?
Fine-tuning tended to improve precision while question-specific structured prompting increased recall.
Examples reported in the paper include precision gains of about +5% for GPT-4o and +16% for Llama-3.1-70B, and recall gains of about +6% for GPT-4o and +18% for Llama-3.1-8B.
How many training samples did the study use and were learning curves checked?
The study used 250 fine-tune instruction samples and checked learning curves at 50, 100, 150, 200, and 250 samples.
Key numeric context reported: median training sample size was 5, 503 tokens and the authors ran learning-curve checks to observe gains over sample sizes.
Can teams reproduce these trade-offs quickly?
Teams can reproduce the trade-offs quickly by running a small pilot with labeled documents, QSP passes, and a higher-precision pass for pruning.
The two-week pilot checklist in this post recommends assembling 50–150 labeled documents and 10–20 domain questions, running base extraction, then QSP-style prompts and a tuned adapter or human QA to compare recall and precision.
Wrapping up & next moves
The June 24, 2026 PLoS ONE paper shows fine-tuning tightens precision while structured prompting widens recall, and a hybrid workflow captures both strengths for practical teams.
- Try a targeted pilot: import a representative subset, define 10 domain questions, run QSP passes, then compare against Evidano’s tuned adapters.
- See how quickly you can get auditable, exportable, and visualizable extraction outputs at Evidano.
If you want to get started with a reproducible pilot, Try Evidano for free.