Same dataset, two methods
The review coded the same body of evidence by hand in Atlas.ti and with Evidano, then compared the results. The dataset, exported from UNICEF’s evaluation management system, covered every management response to a humanitarian evaluation from 2017 to 2024, with region, office, sector, year, recommendation, and action fields, and around 2,500 actions in total.
Evidano was given the review questions and analytical framework as context and returned three outputs within three business days: a narrative report of more than 60 pages organised by theme, an Excel file with every action reorganised by topic, and trend charts that were adapted into two figures in the published report.
Each method has a clear strength
Human analysts were better at nuance and context, spotting trends over time, and catching data problems.
Evidano was far faster and lower cost. It surfaced the key themes, pulled examples directly from the text in multiple languages, and produced a written draft and trend charts to refine. The human coder then coded the material mainly using the themes Evidano had identified, and the final eight themes were settled with UNICEF Evaluation Office experts.
Pair AI with human review
The review advises treating AI output like the work of a research assistant: a fast, thorough first pass that a skilled researcher verifies, validates, and interprets. Its first recommended use case is exactly this workflow: when UNICEF faces a large body of text, such as evaluations, literature, or interviews, a system like Evidano identifies the main themes before human analysis and validation.
“Overall, the AILyze platform did a very good job in identifying key themes in the data and also in summarizing these key themes in prose for a human to review, verify, and adapt.”
“An important benefit of a machine learning system like AILyze is that, unlike Generative AI models that are known to “hallucinate” results, AILyze references the source documents directly.”
“It provided a high-quality written draft that summarized key themes that a human researcher could review, reorganize and consolidate.”
“It took less than three business days to get the product, which even for its flaws is remarkable, especially given its price point”
Manual coding vs Evidano
| Dimension | Manual coding | Evidano |
|---|---|---|
| Speed | ~54 hours of analyst effort | ~3 business days |
| Cost | $3,000–$5,000 | $500–$2,500 |
| Theme discovery | codebook built by hand | human coders coded mainly using the themes Evidano identified |
| Evidence trail | coded excerpts | clickable quotes linked to their source |
| First output | coded dataset to write up | written draft, reorganized data, and trend charts |
Time and cost figures are the review’s own preliminary estimates (Table 3) for preparing the dataset for analysis. The review called the turnaround and price “remarkable, especially given its price point.”
The review notes every quote in the analysis can be clicked to see its context in the source document.
The review records that the tool does not use clients’ textual data to train its AI models.
Themes were reviewed with UNICEF experts and refined into the final eight.
The review’s first use case for UNICEF is AI theme identification on large text datasets, followed by human analysis and validation.