Process Tracing: testing causal mechanisms in a single case

Most evaluation treats evidence additively: more supporting material means more confidence. Process tracing treats it diagnostically. What matters is not how much evidence supports a hypothesis but how unlikely that evidence would be if the hypothesis were false.

A quote from a programme manager saying the intervention worked is abundant, cheap, and almost equally likely under every hypothesis — it barely moves the needle. A dated internal memo from a ministry official, written before the policy shift and citing the programme's specific analysis, is rare and hard to explain any other way. Process tracing is the discipline of preferring the second kind and knowing why.

The method therefore begins from a set of rival explanations, not a single hypothesis. If the only candidate is "the programme did it", the evidence cannot discriminate and the exercise is narrative, not inference.

Process tracing developed in political science and comparative-historical analysis, where single-case explanation is the normal situation. David Collier's Understanding Process Tracing is the standard short introduction and the source most evaluators meet first.

The fullest methodological treatment is Derek Beach and Rasmus Brun Pedersen's Process-Tracing Methods, which distinguishes theory-testing, theory-building and explaining-outcome variants, and their Causal Case Study Methods, which situates it among case-based designs.

Its arrival in evaluation is more recent. Schmitt and Beach set out the case in The contribution of process tracing to theory-based evaluations, arguing it supplies the evidentiary discipline that theory-based evaluation had been missing. Not everyone agrees the transfer is clean; David Waldner's Process Tracing and Qualitative Causal Inference is the most substantial critique of how loosely the label is applied.

The tests are not a scoring rubric applied to evidence after collection. They are a specification written before it: for each hypothesis, what would we expect to find, how surprising would that finding be under the rivals, and what would its absence mean?

Two errors are near-universal in applied work. The first is labelling evidence a smoking gun because it feels persuasive; the label is earned only if the evidence would be very unlikely under every rival explanation, which requires having specified the rivals. The second is treating a passed hoop test as confirmation. A hoop test that is passed tells you only that the hypothesis has not been eliminated — that is genuinely useful, and it is not support.

The most valuable single move in an applied process trace is to ask, for each piece of evidence: would I still expect to see this if the programme had made no difference at all? If the answer is yes, the evidence is not doing inferential work no matter how vivid it is.

A coalition claimed credit for a national regulation restricting marketing of sugary drinks to children. The outcome was specified precisely: the regulation laid before parliament on a stated date, containing an age threshold of 16 rather than the 12 in the earlier draft.

Four rivals were listed alongside the coalition hypothesis: a regional treaty obligation, a change of health minister eight months earlier, an unrelated industry-led voluntary code, and sustained newspaper coverage of an obesity report the coalition had no part in.

The coalition mechanism was written as four links: the coalition's commissioned legal analysis reached the ministry's drafting team; the drafting team found the age-threshold argument persuasive; the threshold was revised in the draft; the revised draft survived consultation.

A hoop test was specified for link one — the analysis should appear in the ministry's document register. It did, dated three weeks before the revision. A predicted smoking gun for link two — internal correspondence citing the analysis specifically on the age question — was requested and not obtained; the ministry declined. Instead a civil servant with no coalition ties confirmed in interview that the legal argument had been discussed by name at the drafting meeting, which was treated as a strong straw in the wind rather than a smoking gun, because a rival explanation (the analysis was noted but not decisive) remained live.

The treaty rival failed its own hoop test: the treaty text contained no age provision, so it could not explain the specific change at issue. The minister rival survived and could not be eliminated.

The reported conclusion was correspondingly bounded: the coalition's legal analysis was almost certainly an input to the age-threshold revision, the ministerial change plausibly created the opening, and the relative weight of the two could not be established on available evidence. That is a weaker claim than the coalition wanted and a considerably more defensible one.

The four tests are a qualitative shorthand for Bayesian updating, and some practitioners make the Bayesian structure explicit, assigning prior probabilities and likelihood ratios. Fairfield and Charman set out how to do this rigorously in Explicit Bayesian Analysis for Process Tracing.

The benefit is transparency: the reasoning becomes auditable, and disagreements localise to specific numbers rather than to the whole conclusion. The cost is that the numbers are themselves judgements, and precise-looking probabilities can lend unearned authority to a guess. Updating Bayesian(s) examines this trade-off directly and is worth reading before committing.

For most evaluation work the qualitative tests, applied honestly, do the job. Formalise when the stakes are high, the audience is sceptical and technically comfortable, or several analysts must reconcile their judgements.

Process tracing establishes whether a mechanism operated in a case. It does not establish how much of the outcome it produced, and it does not generalise: a mechanism confirmed in one setting may be absent in the next.

It is also evidence-hungry in a specific way. The method depends on access to contemporaneous records — minutes, correspondence, dated drafts — and where those do not exist or are withheld, the strongest tests simply cannot be run. Reconstructing them from retrospective interviews reintroduces exactly the hindsight bias the method was designed to escape.

And it is labour-intensive per case. A serious trace of one outcome can take as long as a small survey, which is why it is usually reserved for the outcomes that matter most rather than applied across a portfolio.

Contribution tracing is process tracing adapted for evaluation, adding explicit probability judgements and a workshop format for eliciting them.

Contribution analysis builds the same kind of causal narrative but assesses evidence holistically rather than through discrete diagnostic tests; process tracing is often used to strengthen one link within a contribution analysis.

Realist evaluation shares the mechanism focus but compares configurations across contexts, where process tracing goes deep on one.

Qualitative Comparative Analysis is the cross-case complement, and combining the two — QCA to identify the configurations, process tracing to verify the mechanism in selected cases — is an established multi-method design.

The inferential judgements here are irreducibly human. What is mechanisable is the evidence management: a serious trace accumulates hundreds of documents, each of which has to be linked to a specific mechanism link and a specific test, and kept retrievable when a reviewer asks why a particular piece was called a hoop rather than a straw in the wind.

That is a structured-coding problem, and handling it in a qualitative analysis platform rather than a spreadsheet makes the audit trail real rather than nominal. Evidano supports process tracing as a named methodology; the tests, the rival hypotheses and the weighting stay with the analyst, which is where they belong.