The Success Case Method: learning from the extremes

SCM rests on an observation about how organisational interventions actually distribute their effects: they are rarely normal. A training programme typically produces substantial change for a minority, no change for a majority, and something in between for the rest. Averaging across that distribution describes nobody.

The method therefore samples the tails on purpose. It asks two questions of the successes — what did they actually do differently, and what made that possible — and one of the non-successes: what got in the way. The answers are usually about the environment surrounding the intervention rather than the intervention itself, which is the finding managers can act on.

The second claim is evidentiary. A verified success case demonstrates that the intervention can produce the outcome under identifiable conditions. That is a weaker claim than an average effect and a considerably more actionable one — and, crucially, it is a claim SCM can actually support, provided the case is verified rather than merely reported.

Robert Brinkerhoff developed SCM in corporate training evaluation, where the field had spent decades on satisfaction sheets and pre/post knowledge tests that told organisations nothing about whether anything changed at work. He set out the approach in The Success Case Method: A Strategic Evaluation Approach to Increasing the Value and Effect of Training, building on the argument in Clarifying and Directing Impact Evaluation.

The practical framing appears in Increasing impact of training investments: an evaluation strategy for building organizational learning capability, which is where the method's central empirical claim is clearest: the constraint on training impact is almost never the training.

The most useful methodological development is Coryn and colleagues' Adding a Time-Series Design Element to the Success Case Method, which addresses the method's weakest point — that a success case, however well documented, does not by itself establish the intervention caused it.

A hospital group trained 340 supervisors in a structured feedback model. Post-course satisfaction averaged 4.4 out of 5; a knowledge test showed a solid gain. Six months later nobody could say whether anything had changed on the wards.

The impact model specified the chain: supervisors learn the model, hold structured feedback conversations monthly, staff report clearer expectations, and — the outcome the board cared about — unplanned turnover falls.

A screening survey went to all 340, with a 71% response rate. Twenty-two reported holding structured conversations at least monthly; 143 reported holding none at all.

Twelve of the twenty-two were interviewed and eight verified through documented conversation records and a corroborating interview with a team member. Their accounts converged on one condition: each had a manager who asked about the conversations in their own one-to-ones. Not encouragement or endorsement — a routine question that made non-compliance visible.

Fifteen non-cases were interviewed. Their barriers were consistent and mundane: no protected time, rosters that put supervisors and staff on opposite shifts, and — most often — a belief that the model was optional because nobody had ever asked about it.

The recommendation was not more training. It was a standing agenda item in the layer of one-to-ones above the supervisors, plus a rostering change, at a fraction of the cost of retraining 340 people. The report also stated plainly that only 22 of 241 respondents had applied the model at the intended frequency — a fact the satisfaction score had entirely concealed.

The obvious criticism is that studying successes and generalising from them is textbook selection bias. It is a fair objection to careless applications and it has three legitimate answers.

• Report the base rates. SCM screens the whole population, so it knows how many successes there were. Publishing "8 verified cases out of 241 respondents" makes the method a distribution finding rather than a selection of favourable examples.
• Study both tails. The non-success interviews are not a courtesy; they are the comparison that makes the success conditions meaningful. An SCM that only interviewed successes has not been done.
• Verify, do not accept. Self-reported success is not a success case. Corroboration is what separates the method from testimonial collection.

What SCM still cannot do is establish that the intervention caused the successes — the eight verified supervisors might have been the eight who would have improved anyway. Coryn and colleagues' time-series extension is the practical response, adding repeated measurement so that the timing of change can at least be checked against the timing of the intervention.

• Skipping the non-successes. The most frequent failure, and it converts the method into internal marketing.
• Not verifying. Enthusiastic self-report is exactly what the method was designed to get past.
• Omitting the base rate. Success stories without the denominator are misleading, and readers are entitled to assume the worst when it is missing.
• Vague success definitions. If success is not behaviourally specified, screening cannot identify it and interviews drift into satisfaction.
• Calling it impact evaluation. SCM demonstrates that impact occurred in verified cases under identifiable conditions. It does not estimate programme effect.
• No decision behind the commission. Without a decision to inform, the findings have nowhere to go.

SCM does not produce an effect estimate, does not control for selection into the extremes, and depends heavily on the honesty of the screening survey — which is administered inside an organisation where reporting non-application may carry a cost.

It is best suited to interventions with a discrete population that can be enumerated and surveyed: training cohorts, tool rollouts, process changes. It fits community and policy work poorly, where the population is open and the outcome diffuse.

And its findings are conditional on organisational context. The conditions that enabled success in one hospital group are a hypothesis elsewhere, not a transferable finding.

Case-based impact evaluation shares the comparative-case logic and is the better choice when cases must be selected on something other than the outcome.

Contribution analysis is what to use when the causal claim itself is the deliverable rather than the conditions for success.

Outcome harvesting identifies outcomes without a defined population, which is why it fits advocacy where SCM fits training.

Qualitative Comparative Analysis formalises what SCM does informally — identifying the combinations of conditions under which the outcome occurs — and is worth the step up once there are enough comparable cases.

Two parts of SCM are qualitative analysis at volume. The screening survey's open-ended responses — often several hundred of them — have to be read for application evidence, not just scored. And the interview corpus from both tails has to be analysed comparatively, since the finding is the contrast between what enabled the successes and what blocked the rest.

That comparison is the analytic core of the method and it is where doing it by hand across two sets of transcripts tends to collapse into impressions. Evidano supports the Success Case Method as a named methodology and handles both the open-ended survey text and the interview corpus; deciding what counts as verified remains a judgement, and the verification evidence has to be obtained rather than inferred.