Document Analysis: reading records as evidence

Documents excel as evidence of what was recorded and how: decisions as minuted, categories as officially defined, the vocabulary an institution required of itself at a given time. They are unbeatable for chronology, for positions taken in writing, and for the evolution of official framings across versions.

They are weak, alone, as evidence of practice: the gap between the procedure manual and the ward, between the minuted consensus and the argument that preceded it, is a standing finding of field research. The document attests that the organisation wrote this — a fact worth having — not that the organisation did this.

The classic appraisal criteria remain the working checklist: authenticity (is it what it purports to be?), credibility (produced with what accuracy and sincerity?), representativeness (typical of what class of records — and what never got filed?), and meaning (clear on its own terms and in its conventions?).

The most-cited procedural statement is Bowen’s Document analysis as a qualitative research method, covering the skim–read–interpret sequence, the combination of content and thematic techniques, and documents’ five research functions (context, questions, supplementary data, tracking change, verification).

For health-policy work, the READ approach — ready your materials, extract data, analyse, distil — set out in Dalglish, Khalid and McMahon’s Document analysis in health policy research: the READ approach, is a practical protocol that scales from a dozen policies to several hundred.

The document-as-artefact sensibility — records as situated products with authors, audiences, and functions — is the documentary-research tradition’s core teaching, and Scott’s four criteria above come from it (A Matter of Record, Polity Press).

• Retrospective questions: how a policy changed across a decade; what an organisation knew, and when, as recorded.
• Inaccessible settings: closed processes (court records, inquiry files) or past events with no interviewable witnesses.
• Official framing as the object: how “risk”, “merit”, or “safeguarding” is constructed in the record itself.
• Triangulation duty: alongside interviews or observation, documents anchor recollection to record — and expose the gaps that are themselves findings.
• Not sufficient alone for questions about practice, experience, or informal process; there the documents scope the study that other methods complete.

A planning researcher analysed 15 years of one city’s consultation practice: 214 documents — statutory consultation reports, council minutes, internal guidance, and three versions of the consultation manual — constituted by rule from the council archive, with refusals and gaps logged (two contentious projects’ files were incomplete; noted, and pursued via FOI).

Content coding tracked the official story: consultation counts rose steadily. Artefact coding told the second story: across manual versions, the definition of consultation migrated from “opportunity to shape options” to “opportunity to comment on the preferred option”; report genres standardised around a template whose “objections addressed” section was, in later years, boilerplate reused verbatim across projects (caught by cross-document comparison); and minutes stopped recording dissenting submissions individually after a 2016 template change — a silence with a date.

Interviews with eight planners triangulated the record: the template change was remembered as an efficiency fix, its effects unintended. The study’s conclusion — consultation was proceduralised into auditability while losing deliberative content — rested on version diffs, genre shifts, and dated silences: findings only the documents could supply, qualified exactly as far as documents warrant.

• Reading records as reality. The minute is evidence of minuting; practice claims need practice evidence.
• Convenience corpora. Analysing what was easy to download, then generalising to the institution’s documentation.
• Skipping provenance. Undated, unattributed extracts quoted as if origin did not condition meaning.
• Ignoring genre. A press release and an internal memo obey different truth conventions; coding them identically flattens the evidence.
• Missing the silences. What the record systematically omits is often the finding; unlogged absences cannot become one.
• Quote-mining. Extracting vivid lines without the document’s function — the artefact plane — reduces the method to decoration for a prior argument.

The record is produced by the powerful side of most relationships: institutions document; clients, patients, and publics are documented. Corpora inherit that asymmetry, and analyses should name whose voice the archive structurally lacks.

Survival bias compounds it — what was kept, digitised, and indexed is a curated remainder, and the curation criteria are rarely recoverable.

And documents cannot be probed: no follow-up question, no clarification. Ambiguity that an interview would resolve in a sentence becomes, in documentary work, either a limitation stated or an inference defended — never silently resolved.

Summative content analysis supplies the term-tracking machinery when official vocabulary is the object.

Critical discourse analysis deepens the artefact plane into ideology and power.

Framework analysis organises large document sets into case-by-theme matrices for applied teams.

Process tracing uses documents as diagnostic evidence for causal claims about single cases.

Document corpora are the native habitat of AI-assisted analysis: hundreds of PDFs coded against content and artefact codebooks, term migrations tracked across versions and years, and every extracted claim linked to its exact source passage. Evidano handles that pipeline — upload the archive, apply the codebook, compare across document types and periods — and its quote-level traceability is what lets documentary claims carry their provenance into the report.

Appraisal and inference stay with the researcher: no tool knows that the 2016 template change explains the silence, or which absences in the archive are meaningful. The software reads everything; deciding what the record can attest remains the craft.