Evidano is an AI-powered qualitative data analysis platform that ingests interview transcripts and portal messages to help teams surface patient preferences and operational risks. Health systems are experimenting with AI to draft patient portal messages, but patients in a July 16, 2026 study of 40 people still want clinician review and clear disclosure. This post shows researchers and UX teams how to run qualitative analysis of AI-drafted patient portal messages to surface patient preferences, tone boundaries, and clinician workflow risks, then translate findings into operational guardrails. Use these steps to design a pilot, analyze transcripts and messages at scale, and present stakeholder-ready evidence using Evidano.
Key Takeaways
A July 16, 2026 qualitative study of 40 patients found that patients will accept AI-drafted portal messages only if clinicians review them and patients are informed that AI assisted. The evidence points teams toward piloting AI for low-stakes, high-volume messages first, adding EHR safeguards and testing disclosure language. Use purposive interviews plus cross-segment frequency analysis to build defensible policies and operational workflows.
- A qualitative sample of 40 patients (reported July 16, 2026) showed acceptance of AI drafts conditioned on clinician review and disclosure.
- Patients view portal messages as transactional for refills and scheduling, while preferring clinician-written or in-person communication for uncertain or sensitive results.
- One cited study reported a 7% response-time reduction (about 20 seconds) when AI assisted, highlighting operational gains for routine tasks.
- Deployment needs: clinician-review workflows, disclosure UX testing, KPIs, and patient advisory involvement to reduce risk.
Fast take: what the study found (source)
A concise summary: a qualitative study of 40 patients at an academic medical center (reported July 16, 2026) found patients accept AI-drafted portal messages provided clinicians review drafts and patients are told AI assisted.
TechTarget covered the JAMA Network Open study.
- Patients view portal messages as transactional (refills, scheduling, quick answers) rather than relationship-building.
- Nearly all participants insisted on clinician review before messages are sent.
- Patients prefer clinician-written or in-person communication for higher-stakes or uncertain results.
- The study informed practical deployment tips: KPIs, clinician-review workflows, disclosure guidelines, and patient advisory involvement.
Findings snapshot
| Metric / Item | Value | Source / Note |
|---|---|---|
| Qualitative sample size | 40 patients | JAMA Network Open cited in TechTarget (16 Jul 2026) |
| Patient portal message volume change since 2020 | 153% increase | Earlier JAMA report referenced in coverage |
| Measured response-time reduction in one 2025 study using AI | 7% (≈20 seconds) | NYU report cited in article |
| Primary patient conditions for preferring human note | Uncertainty, anxiety, abnormal test results | Qualitative theme from interviews |
What happened, study design & core findings
Researchers conducted semi-structured interviews with 40 patients to explore expectations around AI-drafted portal messages.
The interviews revealed that patients saw potential benefits such as faster replies and reduced clinician load, but almost uniformly required human review and explicit disclosure that AI generated a draft.
- Tone matters: clerical tone is acceptable for routine tasks (refills, scheduling), while an empathetic, careful tone is expected for sensitive topics.
- Workflow safeguards are essential: policies and EHR constraints should prevent unreviewed sends and define responsibility pathways for content.
- Patients did not agree on exact disclosure wording or timing, indicating disclosure UX needs testing across populations.
So what for researchers, UX teams, and health ops
Researchers
Researchers should stratify participants by message intent, anxiety level, and prior portal experience to spot differing tolerances for AI involvement.
If you run a qualitative analysis of AI-drafted patient portal messages, stratify participants by message intent (transactional vs. clinical), anxiety level, and prior portal experience to spot differing tolerances for AI involvement.
UX / Product teams
UX and product teams should test disclosure variants and placement and measure trust and understanding, not only click rates.
Test disclosure variants and placement (for example, pre-send banner, message footer, portal settings) and A/B the language; measure trust and understanding in addition to click rates.
Health operations & clinicians
Health operations and clinicians should define KPIs up front and build EHR safeguards so drafts cannot be sent without explicit clinician sign-off.
Define KPIs up front (response time, clinician edit time, patient understanding). Build EHR safeguards so drafts cannot be sent without an explicit clinician sign-off and document responsibility pathways for content.
Do more, faster with Evidano
From interview transcripts to themes
Evidano helps teams convert interview transcripts and portal message logs into themes and quantified preferences.
Ingest interview transcripts and portal message logs into Evidano. Use thematic analysis to surface patient demands (disclosure, tone, review) and frequency analysis to quantify how often each preference appears across segments.
Compare segments reliably
Evidano enables reliable cross-segment analysis to identify where acceptance varies by cohort.
Run cross-segment analyses (age, portal experience, symptom severity) to see where acceptance varies. Evidano’s cross-segment reports let you test if disclosure preferences cluster by cohort and prioritize UX tests accordingly.
Operationalize workflows
Evidano produces operational artifacts such as codebooks and visualizations to guide EHR rule changes.
Map qualitative findings to operational controls: export codebooks, generate clinician-facing edit time estimates, and produce stakeholder-ready visualizations (word clouds, co-occurrence networks, hierarchical code → subcodes) to guide EHR rule changes.
Security & compliance
Evidano runs analyses with encryption and models that are not used to train third-party systems to protect sensitive health data.
Run analyses with data encrypted and models that are not used to train third-party systems, a core Evidano guarantee for sensitive health data.
Pilot checklist: 7 steps to analyze AI-drafted message acceptability
This pilot checklist lists seven practical steps to test AI-drafted portal messaging while capturing robust qualitative evidence.
- 1) Define scope & KPIs: response time, edit time per message, patient trust scores, and safety incidents.
- 2) Recruit & segment participants: aim for diverse portal users; include high-anxiety cases separately (n≥40 per pilot recommended for initial qualitative depth).
- 3) Collect artifacts: export portal messages, AI drafts, clinician edits, and post-message interviews or surveys.
- 4) Ingest data into Evidano: transcripts and message text → auto-code then review; use custom dictionaries for clinical terms.
- 5) Run theme and frequency analysis: identify where patients accept drafts versus demand clinician messaging.
- 6) Prototype disclosure variants and simulate EHR constraints; gather user reaction data.
- 7) Report: deliver a concise decision memo with quantified themes, representative quotes, and recommended EHR rules.
FAQ: qualitative analysis of AI-drafted patient portal messages
When should AI draft messages?
Start AI drafting with low-stakes, high-volume transactional messages such as refills and scheduling.
Start with low-stakes, high-volume transactional messages (refills, scheduling), then expand as evidence and clinician acceptance grow.
How do I measure patient comfort?
Measure patient comfort with a mix of Likert-scale survey items and short interviews, then code for tone and perceived empathy.
Combine Likert-scale survey items (trust, clarity) with short interviews to capture nuance; code for tone and perceived empathy.
Is this research medical advice?
No: this is research-focused guidance about workflow and communication design, not clinical care or diagnosis.
No: this is research-focused guidance about workflow and communication design, not clinical care or diagnosis. Always follow clinical governance.
Conclusion & next steps
The July 16, 2026 qualitative evidence shows patients expect clinician review, tailored tone, and transparent disclosure for AI-drafted portal messages.
AI can speed responses for routine portal tasks, but the July 16, 2026 qualitative evidence shows patients expect clinician review, tailored tone, and transparent disclosure. For teams running qualitative analysis of AI-drafted patient portal messages, combine purposive interviews with cross-segment frequency analysis to build defensible policies.
- Ready to pilot? Ingest transcripts, run thematic and cross-segment analyses, and produce stakeholder-ready visuals with Evidano: Try Evidano for free.
- Primary source: TechTarget (coverage of JAMA Network Open study).