Diary Studies and Experience Sampling: research across time

Interval-contingent: reports at fixed times (end of day, every evening). Cheap in burden, good for routines and daily summaries — at the cost of within-day resolution and some end-of-day retrospection.

Signal-contingent (classic experience sampling): prompts at random or semi-random moments, participants report on the now. The strongest design against memory bias and for capturing states as lived; also the most intrusive.

Event-contingent: participants report whenever a defined event occurs — an argument, a purchase, a symptom episode. The only design that reliably catches rare or irregular phenomena, and wholly dependent on participants recognising and honouring the event definition.

The definitive methodological treatment — designs, trade-offs, and the within-person analytic logic they enable — is Bolger, Davis and Rafaeli’s Diary methods: capturing life as it is lived.

• Within-person change: how the same person varies across contexts and days — the questions averages hide (is it stress that precedes the symptom, for this person?).
• Temporal order: antecedents and consequences observed in sequence, the raw material of process claims no cross-sectional design can touch.
• Rare-event capture: event-contingent designs assemble corpora of moments (conflicts, failures, purchases) that neither interviews nor observation could efficiently collect.
• Ecological texture: qualitative entries written in the moment carry vocabulary, context, and emotion that retrospection launders out.

Diary data are nested — entries within days within people — and the analysis should honour all three levels. Quantitative items support within-person analyses (each participant as their own control); qualitative entries are coded like any corpus, but with the timestamps kept analytic: sequences (what precedes what), trajectories (how week three differs from week one), and day-type contrasts are where diary findings outrun interview findings.

The characteristic product is a process account per pattern: not “users find the feature frustrating” but “frustration entries cluster in the first three days, coincide with imports, and disappear once the workaround appears — which participants then describe as ‘how it works’”.

A product team ran a 14-day diary study with 22 learners: a two-minute interval entry each evening (session yes/no, minutes, one mood item, open text) plus event-contingent reports for “a moment you almost practised but didn’t”. Entry interviews trained the event definition; exit interviews walked each participant through their own fortnight.

Compliance held at 84% with timestamps clean. The event reports — 143 almost-practised moments — were the study’s payload: coded, they showed the app losing its users not at motivation (mood at those moments was fine) but at re-entry cost — the dominant sequence was “opened app → saw due-review pile → closed”. The daily entries added the trajectory: review piles grew after any missed day, so a single skipped evening compounded, and open-text tone shifted from aspiration (“will catch up”) to identity (“I’m behind, I’m bad at this”) between days six and ten — the moment most eventual churners’ last session occurred.

Exit interviews, anchored to each participant’s own entries, converted patterns into mechanism: the pile read as a debt statement. The team shipped a “fresh start” pathway that reframed missed days, targeted at the day-six window the diaries had located — a design decision no retrospective interview could have timestamped.

• Entries designed like surveys. Ten minutes of items per prompt; the study dies by day four and the survivors are a compliance-biased remnant.
• Trusting untimestamped diaries. Sunday-night backfill dressed as daily data — the exact memory bias the method exists to defeat.
• Vague event definitions. “Report when stressed” yields incomparable triggers; entry interviews exist to calibrate the definition.
• Ignoring the attrition structure. Reporting who completed without asking how completers differ from quitters — often the study’s most relevant behavioural finding.
• Analysing entries as a flat corpus. Coding without time kills the design’s advantage; sequence and trajectory are the point.
• Skipping the exit interview. The diary alone shows patterns; the participant walking their own timeline supplies the mechanism.

Self-report remains self-report: in-the-moment entries reduce memory bias, not presentation bias, and repeated measurement is itself an intervention — participants who log their practising practise differently (reactivity the design should measure, or at least admit).

The method selects for the organised and willing; burden falls unevenly, and the populations hardest to retain are often those the question most concerns. Compensation, entry design, and flexible modalities mitigate — never fully.

And per-participant costs cap samples: diary studies trade breadth for temporal depth. They answer process questions with small purposive samples; prevalence claims need other designs, and mixed write-ups should keep the boundary visible.

In-depth interviews bookend diary designs and inherit their anchors; alone, they cannot timestamp anything.

Contextual inquiry observes practice directly but only while present; diaries extend the window at the cost of self-report.

Voice-of-customer interviews capture experience episodically for product teams; diaries catch what happens between the episodes.

Review and feedback mining analyses spontaneous longitudinal traces users already produce — the unprompted cousin.

A diary study’s corpus is large, fragmented, and time-structured: hundreds of short entries plus bookend interviews. Evidano codes the open-text entries at scale with quotes linked to their source (and timestamp metadata preserved in the data), transcribes the entry and exit interviews into the same project, and makes the sequence questions — what precedes the churn-tone shift, which contexts co-occur with skipped sessions — queryable across every participant rather than hand-assembled per case.

Protocol design, the burden budget, and the exit-interview craft are where the study is won; the tool’s job is making 400 small texts analysable with the seriousness the participants’ effort deserves.