Qualitative analysis of preprints just got easier with preprint-fulltext and Evidano. On July 18, 2026 Min Dai published preprint-fulltext to PyPI: a CLI/library/MCP tool that returns bioRxiv/medRxiv/arXiv full text as clean, section-labeled, embedding-ready chunks. If your team runs thematic coding, cross-segment comparisons, or evidence synthesis over methods and results, this frees you from HTML/JATS parsing and gives reproducible chunks you can ingest into Evidano (www.evidano.com) for fast thematic, frequency, and cross-segment analysis.
Key Takeaways
Evidano is an AI-powered qualitative data analysis platform that, when combined with preprint-fulltext, lets teams build section-aware, reproducible preprint corpora for faster thematic and cross-segment analysis. preprint-fulltext (published to PyPI on 2026-07-18) converts DOIs or IDs into section-labeled chunks ready for embeddings, and Evidano ingests JSONL/Parquet for deterministic coding and audit trails.
- preprint-fulltext was published to PyPI on 2026-07-18 by Min Dai (Broad Institute).
- preprint-fulltext uses a default chunk size of 512 tokens and a 64-token overlap, and it emits deterministic chunk IDs plus license metadata.
- An OpenAlex API key has been required for discovery/search since 2026-02-13.
- Evidano accepts preprint-fulltext JSONL/Parquet and preserves chunk-level provenance for reproducible thematic, frequency, and cross-segment analyses.
Fast take + source
preprint-fulltext was added to PyPI on 2026-07-18 by Min Dai (Broad Institute), converting DOIs or IDs into structured sections and exporting JSON/Markdown/JSONL/Parquet suitable for embeddings and retrieval.
- Source: PyPI
- Author / contact: Min Dai, dai@broadinstitute.org
- Why it matters: gives direct access to full methods and results for grounded AI reasoning and reproducible qualitative coding.
Snapshot: key facts
| Item | Value | Note |
|---|---|---|
| PyPI publish date | 2026-07-18 | Package page and docs |
| Default chunk size | 512 tokens | PREPRINT_FULLTEXT_CHUNK_TOKENS = 512 |
| Chunk overlap | 64 tokens | PREPRINT_FULLTEXT_CHUNK_OVERLAP = 64 |
| OpenAlex API key | Required since 2026-02-13 | For discover/search |
| License (software) | BSD-3-Clause | Preprint content remains under each paper's license |
What preprint-fulltext does (plain English)
preprint-fulltext routes requests to source-specific parsers and outputs deterministic, section-aware chunks with provenance and license metadata. You can fetch one paper's full text as Markdown/JSON, search/discover lists of IDs, and ingest bulk corpora into chunked JSONL or Parquet, with exports that respect TDM and redistribution rules.
- Outputs include section_kind (methods/results/etc.), token_count, chunk_id, source and license.
- Bulk ingest is resumable and idempotent (no duplicate chunks).
- MCP server exposes search/get methods for coding agents.
Why this changes qualitative analysis of preprints
Structured, chunked full text from preprint-fulltext removes manual scraping and standardizes sections across sources, improving analysis of protocols, tensions between methods and claims, and quote-level evidence. Having section-aware chunks makes Methods and Results first-class inputs to thematic coding, enabling reproducible corpora useful in audits.
- Move from abstract-only sampling to body-aware coding: methods and results often contain decisive detail.
- Create reproducible corpora (JSONL/Parquet) that carry license and version metadata, helpful in audits and reproducibility checks.
- Agent-friendly MCP endpoints let lab automation and code-based pipelines fetch text programmatically for iterative qualitative review.
So what for researchers & UX/Policy teams (implications)
For qualitative researchers
Qualitative researchers can run topic and theme extraction over Methods and Results separately to reduce false positives from abstracts. Run separate methods-coded themes versus claim themes to surface differences in reported practice and claims across corpora.
Compare practices across samples (versions, servers, date windows) using chunk-level metadata as your segment key.
For UX and product teams
UX and product teams can rapidly harvest design-related protocols or implementation details from arXiv and bioRxiv when evaluating open-source ML models. Use precise, section-cited quotes in stakeholder decks without manual copy-paste while preserving provenance.
For policy and evidence teams
Policy and evidence teams can support claim verification by mapping reported results against the paper's Results section programmatically. Use license-aware exports to avoid accidental redistribution, preprint-fulltext enforces TDM compliance.
Do more, faster with Evidano (mapping features to this use case)
Ingest: from preprint-fulltext to Evidano
Evidano accepts preprint-fulltext JSONL or Parquet directly for ingestion and mapping. Map chunk fields (chunk_id, doi, version, section_kind, token_count, text) to document and segment metadata for instant filtering.
Because preprint-fulltext tags section_kind, Methods and Results can be treated as first-class segments and run through separate thematic codebooks.
Automated coding & themes
Evidano runs AI-assisted thematic analysis to generate reproducible codebooks, which you can lock and re-run for deterministic coding across corpus versions. Use frequency and co-occurrence analyses in Evidano to find which methods co-occur with specific claims or reported outcomes.
Cross-segment and cross-cohort analysis
Evidano compares themes by section_kind, by date window, or by server (bioRxiv vs arXiv) using cross-segment analysis and visualizations like co-occurrence networks and hierarchical code maps. Evidano generates stakeholder-ready visuals and quote lists that preserve original provenance for audit trails.
Secure, reproducible research
Evidano ingests encrypted corpora and does not use your data to train third-party models, matching projects that need privacy and auditability. Combine preprint-fulltext's license metadata with Evidano's export controls to respect redistribution rules.
Workflow: 6 steps to build a reproducible qualitative corpus
This checklist shows six steps to move from a DOI list to actionable insights using preprint-fulltext and Evidano.
- 1) Discover: run preprint-fulltext discover/search for your topic or date window and export hits.jsonl.
- 2) Ingest: ingest selected preprints with preprint-fulltext ingest into JSONL/Parquet (Europe PMC for CC subset, S3 for complete corpus).
- 3) Import: upload the JSONL/Parquet into Evidano and map chunk fields to document and segment attributes.
- 4) Code: run AI-assisted codebook generation, review, and lock a codebook for Methods and Results separately.
- 5) Analyze: run thematic, frequency, and cross-segment analyses; produce co-occurrence networks and quote exports with provenance.
- 6) Share: export visuals and quote packs for stakeholders; use Evidano's secure sharing and audit logs.
FAQ: preprint-fulltext
Can I redistribute the preprint text I ingest?
Redistribution depends on each paper's license, preprint-fulltext marks license per chunk and degrades non-redistributable works to link-back stubs unless a license allows redistribution.
How do I compare Methods across 500 papers?
Yes: chunk-level section_kind lets you filter to Methods, then run thematic coding and frequency counts in Evidano and use cross-segment analysis to compare by date or server.
Is this workflow reproducible and auditable?
Yes: preprint-fulltext emits deterministic chunk IDs and manifests, and Evidano preserves provenance and produces exportable audit logs and quote lists.
Conclusion & next steps
If your team needs grounded qualitative analysis that cites paper bodies, combine preprint-fulltext to build section-aware corpora and Evidano to run thematic, frequency, and cross-segment analyses at scale.
- Build a pilot: ingest a curated hits.jsonl from preprint-fulltext and import it into Evidano to run a Methods-vs-Results theme comparison in one sprint.
- Try a two-week pilot or get started now: Try Evidano for free.
- References: preprint-fulltext package and docs at PyPI.