Contribution analysis exists because most programmes cannot be randomised and most evaluators are still asked whether the programme worked. Its answer is to make the causal claim explicit, state what evidence would have to exist for it to hold, go and look for that evidence, and then deal honestly with the explanations that compete with yours. It produces a reasoned argument rather than an effect size — and where a counterfactual is unavailable, a reasoned argument is what is actually on offer.
What contribution analysis is
Contribution analysis is a systematic approach to causal inference for programmes where experimental designs are impossible or inappropriate. Rather than asking how much of an observed change the programme caused, it asks whether a contribution story — an explicit account of how the programme was supposed to produce the change — is supported by the available evidence, and whether the plausible alternative explanations can be ruled out or bounded.
The output is a contribution claim with stated confidence and stated weaknesses: the programme plausibly contributed to this result, through these mechanisms, and here is the evidence, here are the assumptions that did not hold, and here is what other factors were doing at the same time.
The method is deliberately iterative. A first contribution story is drafted from the programme's theory of change, tested against evidence, found wanting in specific places, revised, and tested again. The revision is the method working, not a sign the original was wrong-headed.
Where it comes from
John Mayne set out the approach in Addressing Attribution through Contribution Analysis: Using Performance Measures Sensibly (2001), written in and for the public-sector performance-measurement context, where managers were being asked attribution questions their monitoring data could not answer.
He revisited and tightened it a decade later in Contribution analysis: Coming of age? (2012), which is the better starting point for a practitioner: it addresses the criticism that the approach was too loose to be falsifiable, and sharpens the role of assumptions and rival explanations.
The six steps
1. Set out the attribution problem
State the causal question precisely, and be explicit about what kind of claim is achievable. "Did the programme cause the change" is usually not answerable; "did the programme contribute, and how much of the change can it reasonably claim" usually is.
Agree at this point what level of confidence the users need. An evaluation that will inform a funding decision needs a different evidentiary bar from one that will inform programme redesign.
2. Develop the theory of change and its risks
Write the postulated causal chain from activities to impact, and — this is the step most often skimped — write down the assumptions at each link, and the external factors that could account for the same result.
A theory of change with no stated assumptions cannot be tested, because nothing about it can fail.
3. Gather the existing evidence
Assemble what is already known against each link: monitoring data, prior evaluations, research literature, administrative records. The point is to find where the chain is already well supported and where it rests on nothing.
4. Assemble and assess the contribution story
Write the story out and interrogate it. Which links are strong? Which assumptions have evidence behind them? Where would a sceptical reader stop believing?
This is the step at which most first drafts fail, and failing here is cheap compared with failing at step 6.
5. Seek out additional evidence
Target new data collection at the weak links identified in step 4 — not at the whole chain. This is what makes contribution analysis affordable: evidence is gathered where the argument is fragile, not uniformly.
It is also where rival explanations get tested directly rather than acknowledged and ignored.
6. Revise and strengthen the contribution story
Rebuild the story with the new evidence, restate the confidence, and be explicit about what remains unresolved. Then stop, or iterate again if the users need more.
A worked example
A national programme trained primary-care staff in a new referral protocol, and referrals to specialist services rose 31% over two years. The attribution question from the ministry was whether the training caused the rise.
The theory of change had four links: training delivered, staff knowledge changed, staff behaviour changed, referrals rose. Evidence was strong on links one and two — attendance records and pre/post knowledge tests. Link three had almost nothing behind it, and link four had a serious rival explanation: a public awareness campaign had run in the same period, and presentations at primary care had themselves risen 24%.
Step 5 therefore targeted exactly two things. Interviews with staff at high- and low-uptake sites tested whether the protocol was actually being used and why. And referral rates were disaggregated to separate the rise attributable to more patients presenting from the rise in referrals per presentation.
The disaggregation was decisive: referrals per presentation rose 9%, not 31%. The revised contribution story claimed the smaller figure, attributed the remainder to the awareness campaign, and reported that uptake of the protocol was concentrated in sites where a clinical lead had championed it — an assumption absent from the original theory of change entirely.
Common mistakes
- A theory of change with no assumptions. If every link is asserted rather than conditioned, the analysis has nothing to test and will confirm whatever it started with.
- Listing rival explanations without testing any. Acknowledging that other factors existed is not addressing them. Step 5 exists to rule them out or size them.
- Gathering evidence uniformly. Spending the budget evenly across a chain whose first two links were never in doubt leaves nothing for the link that matters.
- Stopping at the first story. A contribution analysis that never revised its story has almost certainly not tested it.
- Claiming attribution language in the conclusions. The evidence supports a contribution claim; writing it up as "the programme increased X by 31%" throws away the method's honesty in the final paragraph.
How quality is judged
| Criterion | What a reviewer looks for |
|---|---|
| Explicit theory of change | Causal links are written out, with assumptions stated at each one |
| Falsifiability | The analysis identifies evidence that would have undermined the claim |
| Rival explanations addressed | Alternatives are tested and bounded, not merely acknowledged |
| Targeted evidence | New data collection is concentrated on the weak links |
| Visible iteration | The contribution story changed between first draft and final |
| Calibrated conclusions | Confidence is stated; contribution language is used, not attribution language |
Limitations
Contribution analysis produces a defensible argument, not an estimate. It cannot tell a funder how many units of outcome their money bought, and evaluators are regularly pressed to convert its conclusions into numbers they cannot support.
It is also only as good as the theory of change it starts from. A programme whose logic was never articulated will spend most of the evaluation reconstructing one, and a reconstruction assembled after the results are known is vulnerable to being shaped by them.
The approach has been criticised as insufficiently rigorous — that a sufficiently determined analyst can always assemble a plausible story. The standard response is to formalise the evidentiary step, which is precisely what contribution tracing does.
Key references
Mayne, J. (2001). Addressing attribution through contribution analysis: Using performance measures sensibly. Canadian Journal of Program Evaluation, 16(1), 1-24. https://doi.org/10.3138/cjpe.016.001 — the founding statement of the approach.
Mayne, J. (2012). Contribution analysis: Coming of age? Evaluation, 18(3), 270-280. https://doi.org/10.1177/1356389012451663 — the sharper, more practically usable revision, and the better place to start.
Befani, B., & Mayne, J. (2014). Process tracing and contribution analysis: A combined approach to generative causal inference for impact evaluation. IDS Bulletin, 45(6), 17-36. https://doi.org/10.1111/1759-5436.12110 — how process tracing supplies the evidentiary rigour critics found missing.
Where software helps, and where it does not
Steps 3 and 5 are evidence-assembly problems: reading across monitoring data, prior evaluations, interview transcripts and administrative records to find what bears on each link in the chain. At any scale, keeping that consistent across a long corpus is the practical difficulty.
AI-assisted analysis helps by classifying material against the links of a stated theory of change and by surfacing passages that bear on a specific assumption — which is a retrieval and coding problem, and a tractable one. UN evaluation offices have used Evidano for exactly this kind of framework-guided document synthesis; the published case studies describe how the analyses were checked.
It does not help with step 2. Deciding what the assumptions are, and which rival explanations are serious enough to test, is the analytic judgement the whole method rests on. Software that proposes a theory of change is proposing the answer.
Topics
- contribution analysis
- theory of change
- evaluation
- causal inference
- documents-policy
- process tracing
Other methods in realist and causal analysis approaches
Written guides are linked directly; the rest have a reference entry in the methodology directory.
Keep reading
- Research MethodsContribution Tracing: Bayesian confidence in contribution claimsContribution tracing combines process tracing with explicit Bayesian updating to put a defensible confidence level on a contribution claim. How it works, and its limits.
- Research MethodsOutcome Harvesting: a practical guideHow to run an outcome harvest: the six steps, what counts as an outcome, how substantiation works, and when harvesting beats a conventional results framework.
- Research MethodsRealist Evaluation: context, mechanism, outcomeWhat realist evaluation actually asks, how to build and test a CMO configuration, where the mechanism concept gets misused, and how realist quality is judged.