Most Significant Change: story-based evaluation done rigorously

MSC collects short first-person accounts of significant change, then passes them up through layers of an organisation. At each layer, a group reads the stories from the level below, discusses them, selects the one they consider most significant, and — the essential step — writes down why.

Those documented reasons are the analytic output. A field team that consistently selects stories about individual livelihood gains and a national office that consistently selects stories about policy influence are telling the evaluation something important about a value divergence inside the organisation, and neither group would have reported it if asked directly.

The technique is deliberately dialogical rather than extractive. It does not aim at a representative sample, and it makes no claim to one; it aims at surfacing and negotiating the criteria by which change is judged. This is why MSC is classified as a monitoring and learning technique first and an evaluation method second.

MSC was developed by Rick Davies in the mid-1990s for the Christian Commission for Development in Bangladesh, working on a programme whose outcomes were too diverse and too locally specific for a common indicator set. Jess Dart and Davies later formalised it in A Dialogical, Story-Based Evaluation Tool: The Most Significant Change Technique, published in the American Journal of Evaluation and still the primary reference.

The most useful critical assessment is Willetts and Crawford's The most significant lessons about the Most Significant Change technique, which reports what actually happened when organisations tried to use it and is more informative about failure modes than the founding papers.

Dart returns to the technique with a case study in the SAGE Handbook of Participatory Research and Inquiry, which situates it within the broader participatory tradition.

A livelihoods programme ran MSC across four districts for two years. Field officers collected 60–80 stories per round in three domains.

The district selections were consistent: stories about individual households moving out of acute food insecurity, chosen for their vividness and the scale of change in one life. The national office, reading the district selections, consistently chose differently — stories where a group had negotiated with a local authority, chosen because they suggested change that would persist without the programme.

Two rounds in, the meta-analysis made the divergence explicit. Field staff were being managed against household-level targets and were selecting accordingly; the national office was accountable to a donor interested in systemic change. Neither had noticed, and the disagreement had been showing up as friction over reporting rather than as a strategy question.

The programme revised its field-level indicators and added a domain for changes in relationships with authorities. A separate finding came from verification: one selected story turned out to describe a change that had begun before the programme arrived, which was reported openly and led to a tightening of the collection question.

None of this is an impact claim, and the evaluation did not present it as one. It is organisational learning of a kind an indicator framework cannot produce.

• Treated as a success-story generator. The commonest abuse. If collectors are asked for positive change, the technique becomes a communications exercise, and its evaluative value is gone.
• Selection reasons not recorded. Without the documented criteria there is no analysis, only an anthology.
• No feedback loop. Participation collapses when contributors never learn what happened to their stories.
• No verification. Selected stories carry weight and travel; an unverified one that turns out to be wrong damages the whole exercise.
• Stories edited into the house style. Rewriting for readability strips out the teller's own significance framing, which is the data.
• Presented as representative. MSC selects for the exceptional by design. Reporting selected stories as typical misrepresents the method.
• Run once. A single round produces stories; the findings come from patterns across rounds.

The markers of a well-run MSC process are procedural, not statistical: selection criteria recorded verbatim at every level, including minority views; a documented feedback loop; verification of the stories that were selected; a meta-analysis of criteria across levels and rounds; and negative or unwelcome stories present in the collected set.

The absence of any uncomfortable story in a corpus of two hundred is itself a finding about the collection process, and a reviewer should treat it as one.

MSC cannot tell you how common anything is. It selects for the exceptional, so it says nothing about distribution and should never be used to estimate reach or coverage.

It is also vulnerable to power. Where storytellers depend on the programme, the stories will lean favourable, and no amount of verification fully corrects that. Facilitation quality matters more than in most methods, and a facilitator who signals what they want to hear will get it.

The technique is time-hungry in a way that is easy to underestimate — the deliberation at each level is where the value is, and it cannot be compressed into a form. And its findings are about values and process; a funder wanting an outcome measure will need something else alongside it.

Outcome harvesting also works from what actually changed, but formulates verifiable outcome statements rather than narratives, and substantiates them independently.

Ripple Effects Mapping is the group-mapping counterpart, surfacing unintended and secondary effects collectively rather than through individual accounts.

Success Case Method shares the focus on extremes but samples deliberately from both tails and investigates the mechanism, where MSC lets participants choose.

Participatory evaluation more broadly is the family MSC belongs to; it is one technique within it rather than a whole design.

The corpus grows quickly — four districts, three domains, two rounds a year is several hundred stories, plus the selection rationales from each level, which are themselves qualitative data.

The meta-analysis that gives MSC its distinctive value depends on being able to read the rationales across levels and rounds and see the pattern. That is comparative coding across a structured corpus, and it is the part that gets abandoned when the material lives in a shared drive. Evidano supports Most Significant Change as a named methodology, keeping stories and their selection rationales linked so the criteria can be analysed rather than just filed. The selection itself must stay with the people whose judgements are being surfaced — automating it would remove the only thing the method is measuring.