Graph Pipelines: LLM-assisted Qualitative Analysis

UX and product researchers can codify a coding pipeline (utterance split → sentiment/intent label → theme mapping) and re-run it as new transcripts arrive to improve reproducibility versus manual passes.

DAG pipelines make it easier to version steps (prompt text, chunk size, model) so experiments are auditable.

Policy and health analysts can chain scrubbers and calibration steps before models see data, which is useful when working with sensitive interviews.

License (AGPL) requires attention if you plan to integrate into closed-source systems.

Data and ops teams can map a DAG approach cleanly to CI: test small sample outputs, run calibration jobs, and deploy pipelines that process large corpora reliably.

Built distributions and file hashes are provided on PyPI, which is useful for secure installs.

Evidano ingests and pre-processes transcripts, reports, and survey sheets and drops outputs into a soak pipeline as pre-cleaned text.

Problem: varied inputs, recorded interviews, PDFs, spreadsheets. Evidano: Native ingestion of transcripts, reports, and survey sheets, transcription with custom dictionaries and PII redaction, and export of cleaned text into soak.

Evidano imports codebooks and runs AI-assisted coding with hierarchical codes and calibration tools to compare human versus model labels before committing changes to pipelines.

Problem: prompt drift and inconsistent codes across batches. Evidano: Import codebooks and run AI-assisted coding with hierarchical codes → subcodes, plus calibration tools to compare human vs. model labels.

Evidano runs thematic, frequency, and cross-segment analyses, plus co-occurrence networks and word clouds that map directly to soak’s aggregated outputs for rapid reporting.

Problem: synthesizing themes across cohorts (region, persona, time). Evidano: Thematic, frequency, and cross-segment analyses, plus co-occurrence networks and word clouds that map directly to soak outputs.

Evidano provides end-to-end encryption and a policy of not using customer data to train third-party models, which complements soak’s local-ai and scrub extras for secure pipelines.

Problem: data sensitivity and model training concerns. Evidano: End-to-end encryption and policy of not using customer data to train third-party models, complementing soak features.

soak 0.13 is a DAG-based toolkit that composes text transforms and LLM prompts into repeatable pipelines for labeling and aggregating qualitative text. The toolkit helps replace ad-hoc scripts with multi-step flows: cleaning, chunking, LLM labeling, aggregation, and export, and ships examples and a docs site.

Evidano ingests raw transcripts, applies transcription and PII redaction, and exports cleaned text for use in soak DAGs; Evidano also imports soak outputs for calibration, analysis, and visualization. Evidano supports calibration tools and codebook imports to compare human and model labels before committing changes.

You should review the AGPL-3.0-or-later license for soak before integrating it into closed-source systems, and treat model outputs from sensitive data as research signals requiring governance. Use Evidano’s PII redaction and encryption features to reduce exposure, and validate labels with human coders before taking action.

A compact two-week pilot can take raw files to a stakeholder brief by combining soak pipelines with Evidano’s ingestion, transcription, calibration, and visualization features. The pilot steps include gathering 50–200 transcripts, running transcription and PII redaction, exporting to a soak DAG, importing outputs into Evidano for analysis, and producing a 2-page brief.