What the evaluation covered
The evaluation assessed UNESCO’s work on museums since the 2015 Recommendation on Museums and Collections, combining the AI-assisted portfolio analysis with interviews with 20 UNESCO staff, 12 external partners, and 15 participants of the 2025 High-Level Forum on Museums, plus benchmarking against 12 other organisations.
An initial keyword search across UNESCO’s SISTER and Core Manager platforms returned 582 candidate projects. Evidano scanned the full set of reports and applied a coding schema provided by the evaluation team, classifying each project against seven criteria as true or false, which refined the portfolio to 230 museum-related records. It then coded 163 narrative reports against the evaluation framework, and two of the report’s published figures are sourced directly to that analysis.
How the AI analysis was governed
The report includes a dedicated methodology note on the Evidano analysis. The process was human-led: an initial sample analysis was submitted to UNESCO-IOS for review before the method was scaled to the full dataset, every key assertion was substantiated with direct quotes and citations from the source reports, and each assigned code came with a written justification reviewers could check. Reports were processed in an encrypted environment, no UNESCO data was used to train AI models, and all source data was deleted on delivery.
How Evidano fit the workflow
The evaluators paired Evidano with human judgment at every step, validating a sample analysis before scaling to the full dataset.
- 1Reports gathered
An initial keyword search across UNESCO’s SISTER and Core Manager platforms returned 582 candidate projects.
- 2Evidano screened and coded
The tool scanned the full set of reports and applied a coding schema provided by the evaluation team, classifying each project against the criteria as true or false, then coded 163 reports against the evaluation framework.
- 3Evaluators reviewed and interpreted
IOS validated a sample analysis before scale-up, researchers decided which projects to include and triangulated findings across data sources, and an Evaluation Reference Group supported quality assurance.
- 4Published evaluation
230 museum projects were mapped across all UNESCO Programme Sectors, and Evidano analysis fed directly into published figures.
“…refined to 230 museum-related records with the AI tool Ailyze (see more below as well as the Methodological Note in Annex).”
“Ailyze scanned the full set of reports and applied a predefined coding schema provided by the evaluation team. The tool was tasked with determining whether each project met specific criteria (see Table 3 below), using a true or false classification.”
“The process was human-led. An initial sample analysis was performed and submitted to UNESCO-IOS for review to validate the LLM's interpretation and coding accuracy. Only after receiving approval was the methodology scaled to the full dataset.”
“No UNESCO data was used to train any AI models. Upon project completion and final delivery, all source data was permanently deleted from the AILYZE system.”
Apply a predefined codebook as the evaluators did, or work from an AI-suggested one.
A sample analysis was reviewed by UNESCO-IOS before the method was applied to the full corpus.
Findings were substantiated with direct quotes and citations, and each code carried a written justification.
Encrypted processing, no training on UNESCO data, and permanent deletion of source data on delivery, as recorded in the report.