Cognitive interviewing for digital access identifies how respondents understand survey questions and exposes mismatches that harm data quality. The PLOS One study (Taluja et al., published July 28, 2026) ran 101 cognitive interviews between 1 April 2023 and 27 April 2023 in rural Uttar Pradesh to test 118 draft survey questions on phone ownership and internet use. Researchers and survey teams can use these documented mismatches to redesign questions, and AI-enabled qualitative analysis can accelerate synthesis, thematic coding, and cross-segment comparison for such cognitive interview data.
Key Takeaways
According to PLOS One (Taluja et al., published July 28, 2026), cognitive interviewing exposed widespread comprehension problems in questions about mobile phone access and digital use and required revisions for almost all tested items. The PLOS One paper is available at PLOS One.
- 101 cognitive interviews were conducted between 1 April 2023 and 27 April 2023, testing a library of 118 draft questions, according to PLOS One (Taluja et al., 2026).
- The sample included 44 men and 57 women, average age 30 years, and interviews averaged 52 minutes per session, according to PLOS One (Taluja et al., 2026).
- The PLOS One team identified seven recurring problem types: inappropriate terminology, overly complex wording, low resonance of digital concepts, confusion about permission/supervision, problematic question structure, self-practice bias, and unclear time frames (Taluja et al., 2026).
- Smartphone ownership in the study was 84% for men and 56% for women, and only 20% of men and 18% of women had nine or more years of education, as reported in PLOS One (Taluja et al., 2026).
What happened: Cognitive interviewing for digital access
The PLOS One research team conducted 101 cognitive interviews in rural Uttar Pradesh between 1 April 2023 and 27 April 2023 to evaluate 118 draft survey questions about phone ownership, use, norms, and harms (Taluja et al., 2026).
The PLOS One team purposively sampled adults with lower formal education to stress-test comprehension, interviewed 44 men and 57 women, recorded 70 substantive interviews for transcription, and iterated questions across three rounds of testing (Taluja et al., 2026).
Key measurement failures the PLOS One team documented included formal Hindi terms that respondents did not recognize, multipart questions that overwhelmed short-term memory, and digital concepts such as "privacy policy" and "hack" that lacked local resonance (Taluja et al., 2026).
Direct respondent quotations in PLOS One illustrate the problem: a respondent (F27) said, "Please speak in Hindi. What does the question mean? ", and another (F06) explained, "Kaam-dhandha means cooking food, washing utensils. All the household chores."
Findings snapshot
| Date / Period | Metric | Value (from PLOS One) | Implication |
|---|---|---|---|
| 1 April 2023–27 April 2023 | Cognitive interviews | 101 interviews (44 men, 57 women) | Iterative testing across three rounds revealed recurring question failures |
| April 2023 | Questions tested | 118 draft survey items | Almost all items required revision for comprehension |
| April 2023 | Average interview length | 52 minutes | Sufficient time for quantitative item plus qualitative probes |
| Sample characteristics (April 2023) | Smartphone ownership | 84% men, 56% women | Gendered differences in access affect skip patterns and question relevance |
| Sample characteristics (April 2023) | Education (9+ years) | 20% men, 18% women | Low formal education increased sensitivity to translation and complexity |
Implications for survey teams and qualitative researchers
Cognitive interviewing is essential for valid digital access surveys because instrument translations and technical terms often do not map to everyday language, as shown in PLOS One (Taluja et al., 2026).
The PLOS One team found that shorter items, colloquial translations, and examples of local apps (for example WhatsApp and YouTube) improved comprehension of abstract digital concepts (Taluja et al., 2026).
Survey teams should expect to iterate tools in staged rounds: the PLOS One researchers revised items across three rounds of testing, reducing question failures as testing progressed (Taluja et al., 2026).
When measuring norms and attitudes, the PLOS One study recommends simpler response formats (binary or three-point scales) because many respondents struggled with 4–5 point Likert scales and showed "self-practice bias" when asked about others' views (Taluja et al., 2026).
How Evidano helps: problem → AI-enabled solution
Problem: Large volumes of cognitive interview transcripts slow synthesis
Evidano is an AI-powered qualitative data analysis platform that helps researchers analyze interviews, open-ended surveys, and documents.
Evidano's transcription and translation features reduce manual work by creating searchable transcripts with custom dictionaries and PII redaction, which speeds analysis of dozens to hundreds of cognitive interviews.
Evidano supports thematic and frequency analysis that turns repeated comprehension failures into coded themes, enabling teams to track which questions fail for which subgroups and why.
Problem: Iterative revisions across rounds require fast cross-round comparisons
Solution: Evidano’s AI chat over your documents and cross-segment analyses let teams ask targeted questions like "Which draft items caused confusion for women aged 18–30 in round 1? " and get extractable answers and frequency counts.
Evidano visualizes co-occurrence networks and hierarchical codes so teams can see which translation choices, response options, or time-frame phrasings cluster with comprehension problems.
Problem: Technical digital concepts lack local resonance and need tailored explainers
Solution: Evidano’s translation feature with custom dictionaries helps teams build explainer boxes and test alternate phrasing quickly across languages.
For field teams planning large surveys, Evidano integrates transcription (speech-to-text features) and fast thematic synthesis to move from cognitive interview findings to revised question libraries.
Problem: Ensuring data security and non-training of third-party models
Solution: Evidano encrypts data and does not use customer data to train third-party models, supporting the ethical handling of sensitive cognitive interview transcripts (see Data Security).
Evidano also supports iterative workflows so teams can maintain provenance between draft and final question versions.
FAQ: cognitive interviewing for digital access
What is cognitive interviewing and why does it matter for digital access surveys?
Answer: Cognitive interviewing is a qualitative method that tests how respondents interpret survey questions, and it matters because misinterpretation can invalidate measures of phone ownership and internet use (PLOS One, Taluja et al., 2026).
Supporting detail: PLOS One tested 118 questions and found seven categories of failures, showing that standard translations and global instruments often miss local meanings (Taluja et al., 2026).
How many interviews are typical to detect recurrent comprehension problems?
Answer: The PLOS One team used 101 interviews and reached practical saturation for their context between 1 April 2023 and 27 April 2023.
Supporting detail: The study followed guidance on thematic saturation and reduced respondent numbers per round as question comprehension improved (Taluja et al., 2026).
Which question types most commonly fail in rural low-literacy settings?
Answer: The PLOS One study identifies seven common failures, especially inappropriate terminology, overly complex questions, and low resonance of digital concepts (Taluja et al., 2026).
Supporting detail: The PLOS One authors give examples where formal Hindi words confused respondents and where terms like "privacy policy" had no accessible translation in the study context (Taluja et al., 2026).
Can AI accelerate analysis of cognitive interview data without compromising ethics?
Answer: Yes, AI can accelerate coding, thematic synthesis, and cross-segment frequency counts while preserving confidentiality when platforms use encryption and do not train third-party models on customer data (see Data Security).
Supporting detail: Using automated transcription, custom dictionaries, and AI-assisted code suggestions reduces manual time to generate actionable revisions from cognitive interviews.
Conclusion & Next Steps
Cognitive interviewing for digital access, as documented in PLOS One (Taluja et al., published July 28, 2026), uncovered systematic comprehension problems that would have compromised a large-scale survey if left unaddressed.
Survey teams should embed iterative cognitive testing, use local examples for abstract digital terms, simplify response formats, and document changes across rounds as the PLOS One team did with 118 questions over three rounds in April 2023 (Taluja et al., 2026).
If you run cognitive interviews and need to scale transcription, translation, thematic coding, and cross-segment synthesis, Evidano’s AI-enabled platform can shorten the path from raw interviews to a revised question library; see Evidano features for details.
Try a hands-on workflow: Try Evidano for free.