Process Evaluation: what was actually delivered, to whom, and why

Implementation: what was delivered — fidelity (consistency with design), dose (how much), reach (who received it), and adaptations (what was changed, why). This is where “the intervention” gets an empirical definition.

Mechanisms of impact: how the intervention produced (or failed to produce) change — participant responses, mediators, and the unexpected pathways and consequences that emerged in practice.

Context: everything external that shaped delivery and effects — organisational capacity, concurrent events, policy environment — including how the same programme met different worlds at different sites.

The linking question makes it an evaluation rather than three monitoring reports: do variations in implementation and context explain variations in outcomes? A process evaluation that never meets the outcome data has answered questions nobody asked.

The field’s reference is the UK Medical Research Council guidance, summarised in Moore and colleagues’ Process evaluation of complex interventions: the three-component model, the case for pre-specifying process questions from the programme’s causal assumptions, and guidance on integrating process and outcome analysis.

The guidance’s core design instruction is to derive process measures from the intervention’s logic model or theory of change — each critical assumption gets an indicator or inquiry line — so the evaluation tests the programme’s theory rather than auditing generic compliance.

It also legitimises mixed methods as the default: quantitative monitoring for fidelity, dose, and reach; qualitative work for adaptation reasoning, participant response, and context — with the two planned to meet in analysis.

• Alongside any outcome evaluation of a complex intervention — without it, trial results cannot distinguish theory failure from implementation failure.
• In multi-site programmes, where between-site variation is both inevitable and the best natural experiment about what matters.
• During pilots and scale-ups, when the honest question is “can this be delivered as designed, and what does it become when it is not?”
• For adaptive interventions, where documenting and reasoning about adaptations is the difference between learning and drift.
• Less useful as a standalone compliance audit — fidelity percentages without mechanism or context questions produce accountability theatre.

A ten-school trial of a classroom mental-health curriculum returned a null primary outcome. The process evaluation, designed up front from the programme’s logic model, made the null interpretable — and useful.

Implementation data showed the curriculum was delivered at high fidelity in seven schools but with dose collapsing in the summer term everywhere (exam pressure); reach analysis showed the students the theory most targeted — those with emerging difficulties — were disproportionately absent on delivery days. Mechanism interviews found teachers systematically softening the sessions on help-seeking (the theorised active ingredient) because they felt unequipped for disclosures; the adaptation log dated the softening to a safeguarding incident in term one. Context work explained the two low-fidelity schools entirely: both had lost their pastoral leads mid-year.

The integrated analysis reframed the trial: the intervention as theorised was barely tested, because its active ingredient was rarely delivered to its target population. The commissioners’ decision — redesign teacher support and re-pilot, rather than abandon — rested on process evidence; the outcome number alone would have killed a programme whose theory had never actually run.

• Process evaluation as afterthought — commissioned once outcomes disappoint, when the delivery data no longer exist.
• Fidelity fundamentalism. Treating all adaptation as contamination; core-function fidelity with documented form-adaptation is the defensible standard.
• Reach without denominators. “400 attended” means nothing without who was eligible and who was missed.
• Mechanism questions skipped — the component that explains results is the one most often cut for budget.
• Parallel reports that never meet. Process and outcome findings published separately, the linking analysis nobody owned.
• Deliverer-only perspectives. Participants’ experience of the intervention is mechanism evidence, not satisfaction garnish.

Process evaluation explains; it does not by itself attribute. Outcome-by-fidelity analyses are observational — sites that implement well differ in other ways — and causal language should stay disciplined.

It is also intrusive by nature: logs, observation, and interviews land on deliverers mid-delivery, and the measurement burden can itself distort the programme. Lean instrumentation is a design duty, not a nicety.

And its findings are time-stamped: the programme it describes is the pilot-era programme. Scale-up changes context, staffing, and dose — which is an argument for keeping light process measurement alive after the evaluation formally ends.

A theory of change or logic model supplies the assumptions a process evaluation tests.

Formative evaluation uses similar methods with an improvement-during-delivery mandate.

Realist evaluation deepens the mechanism component into context–mechanism–outcome theory.

Normalization process theory studies offer a specific lens on how interventions embed into routine work.

The qualitative half of a process evaluation — deliverer and participant interviews, adaptation logs, open-text session notes across sites — is a coding workload with a deadline, since findings must land while delivery can still respond. Evidano codes that corpus against the programme-theory framework with quote-linked evidence, compares themes across sites and implementation bands, and turns “why did site 6 diverge?” into a retrievable answer rather than a war story.

Deciding what counts as core function versus adaptable form, and what the integrated analysis means for the programme, is evaluator judgement — the tool just guarantees the explanation is grounded in what deliverers and participants actually said.