Contribution Analysis: a step-by-step guide

Contribution analysis is a systematic approach to causal inference for programmes where experimental designs are impossible or inappropriate. Rather than asking how much of an observed change the programme caused, it asks whether a contribution story — an explicit account of how the programme was supposed to produce the change — is supported by the available evidence, and whether the plausible alternative explanations can be ruled out or bounded.

The output is a contribution claim with stated confidence and stated weaknesses: the programme plausibly contributed to this result, through these mechanisms, and here is the evidence, here are the assumptions that did not hold, and here is what other factors were doing at the same time.

The method is deliberately iterative. A first contribution story is drafted from the programme's theory of change, tested against evidence, found wanting in specific places, revised, and tested again. The revision is the method working, not a sign the original was wrong-headed.

John Mayne set out the approach in Addressing Attribution through Contribution Analysis: Using Performance Measures Sensibly (2001), written in and for the public-sector performance-measurement context, where managers were being asked attribution questions their monitoring data could not answer.

He revisited and tightened it a decade later in Contribution analysis: Coming of age? (2012), which is the better starting point for a practitioner: it addresses the criticism that the approach was too loose to be falsifiable, and sharpens the role of assumptions and rival explanations.

A national programme trained primary-care staff in a new referral protocol, and referrals to specialist services rose 31% over two years. The attribution question from the ministry was whether the training caused the rise.

The theory of change had four links: training delivered, staff knowledge changed, staff behaviour changed, referrals rose. Evidence was strong on links one and two — attendance records and pre/post knowledge tests. Link three had almost nothing behind it, and link four had a serious rival explanation: a public awareness campaign had run in the same period, and presentations at primary care had themselves risen 24%.

Step 5 therefore targeted exactly two things. Interviews with staff at high- and low-uptake sites tested whether the protocol was actually being used and why. And referral rates were disaggregated to separate the rise attributable to more patients presenting from the rise in referrals per presentation.

The disaggregation was decisive: referrals per presentation rose 9%, not 31%. The revised contribution story claimed the smaller figure, attributed the remainder to the awareness campaign, and reported that uptake of the protocol was concentrated in sites where a clinical lead had championed it — an assumption absent from the original theory of change entirely.

• A theory of change with no assumptions. If every link is asserted rather than conditioned, the analysis has nothing to test and will confirm whatever it started with.
• Listing rival explanations without testing any. Acknowledging that other factors existed is not addressing them. Step 5 exists to rule them out or size them.
• Gathering evidence uniformly. Spending the budget evenly across a chain whose first two links were never in doubt leaves nothing for the link that matters.
• Stopping at the first story. A contribution analysis that never revised its story has almost certainly not tested it.
• Claiming attribution language in the conclusions. The evidence supports a contribution claim; writing it up as "the programme increased X by 31%" throws away the method's honesty in the final paragraph.

Contribution analysis produces a defensible argument, not an estimate. It cannot tell a funder how many units of outcome their money bought, and evaluators are regularly pressed to convert its conclusions into numbers they cannot support.

It is also only as good as the theory of change it starts from. A programme whose logic was never articulated will spend most of the evaluation reconstructing one, and a reconstruction assembled after the results are known is vulnerable to being shaped by them.

The approach has been criticised as insufficiently rigorous — that a sufficiently determined analyst can always assemble a plausible story. The standard response is to formalise the evidentiary step, which is precisely what contribution tracing does.

Mayne, J. (2001). Addressing attribution through contribution analysis: Using performance measures sensibly. Canadian Journal of Program Evaluation, 16(1), 1-24. https://doi.org/10.3138/cjpe.016.001 — the founding statement of the approach.

Mayne, J. (2012). Contribution analysis: Coming of age? Evaluation, 18(3), 270-280. https://doi.org/10.1177/1356389012451663 — the sharper, more practically usable revision, and the better place to start.

Befani, B., & Mayne, J. (2014). Process tracing and contribution analysis: A combined approach to generative causal inference for impact evaluation. IDS Bulletin, 45(6), 17-36. https://doi.org/10.1111/1759-5436.12110 — how process tracing supplies the evidentiary rigour critics found missing.

• Contribution tracing — contribution analysis with Bayesian evidentiary tests attached, for when confidence needs to be stated numerically.
• Process tracing — the causal-inference method that supplies those tests.
• Realist evaluation — asks a different question of the same territory: not whether it contributed, but what works for whom in what circumstances.
• Theory of change — the artefact contribution analysis tests; worth getting right before the evaluation starts.
• Outcome harvesting — the retrospective alternative when no usable theory of change exists at all.

Steps 3 and 5 are evidence-assembly problems: reading across monitoring data, prior evaluations, interview transcripts and administrative records to find what bears on each link in the chain. At any scale, keeping that consistent across a long corpus is the practical difficulty.

AI-assisted analysis helps by classifying material against the links of a stated theory of change and by surfacing passages that bear on a specific assumption — which is a retrieval and coding problem, and a tractable one. UN evaluation offices have used Evidano for exactly this kind of framework-guided document synthesis; the published case studies describe how the analyses were checked.

It does not help with step 2. Deciding what the assumptions are, and which rival explanations are serious enough to test, is the analytic judgement the whole method rests on. Software that proposes a theory of change is proposing the answer.