Scale Live LLMs in Surveys: In-survey LLM deployment

According to the July 31, 2026 PLOS One article by Banks et al. (PLOS One), DiSCoKit is an open-source middleware toolkit that lets researchers embed live LLM chat interfaces into JavaScript-enabled survey platforms, persistently log every conversational turn, and toggle system prompts by experimental condition.

• DiSCoKit was published on July 31, 2026 and the paper reports the toolkit as "a low-barrier, low-cost, scalable, customizable toolkit" (Banks et al., PLOS One).
• In a deployment reported in July 2026, Banks et al. ran 15 pilot sessions and 218 formal sessions, with the AI maintaining the researcher-specified error behavior in all but 16 of the 218 formal sessions.
• DiSCoKit’s template uses a Linux server running Python 3.12 and an Azure-hosted stateless LLM endpoint by default, and the authors recommend at least 15 trials per experimental condition to test prompt variability.
• The DiSCoKit code and documentation are available on GitHub, according to the PLOS One data availability statement at GitHub.

Answer: DiSCoKit gives survey researchers a three-part architecture to deliver live LLM chat in a survey page, control prompts by condition, and log every turn for downstream analysis.

According to the July 31, 2026 PLOS One article by Banks et al., DiSCoKit comprises: (1) JavaScript embedded in the survey platform that creates the chat iframe, (2) a Flask-based middleware app that brokers prompts, calls to an LLM, and augmentation logic, and (3) a database that records each message with participant ID, experimental condition, and timestamp.

According to the July 31, 2026 PLOS One article by Banks et al., the middleware can switch system prompts mid-conversation because the template targets stateless Azure serverless LLM endpoints, enabling researchers to toggle AI behavior on page advances or by condition.

According to the July 31, 2026 PLOS One article by Banks et al., the toolkit was designed to run on a Linux server or Docker environment with Python 3.12 and can be adapted to other LLM providers and databases (the authors moved from SQLite to recommending PostgreSQL for high-concurrency studies).

Answer: DiSCoKit reduces the trade-off between sample size and ecological validity by enabling live, loggable LLM interactions in web surveys.

According to the July 31, 2026 PLOS One article by Banks et al., researchers can keep the low cost and reach of online surveys while capturing actual interaction behavior instead of relying solely on self-report.

According to the July 31, 2026 PLOS One article by Banks et al., teams should budget for prompt development and testing: the authors recommend at least 15 trials per condition to identify generation variability and to validate that system prompts reliably produce the intended behavior.

According to the July 31, 2026 PLOS One article by Banks et al., privacy and security choices matter: the authors used university-licensed Qualtrics and Azure enterprise licenses to ensure data governance and advised consulting institutional IT for API keys and server configurations.

DiSCoKit, according to Banks et al. in PLOS One (published July 31, 2026), offers researchers a pragmatic path for in-survey LLM deployment: embed a chat iframe, broker prompts with middleware, and log every turn for analysis.

Banks et al. describe the toolkit as "an open-source toolkit for deploying live LLM experiences" and as "a low-barrier, low-cost, scalable, customizable toolkit" that supports experimental manipulations and full conversation capture.

If you plan to run in-survey LLM experiments, export DiSCoKit chat logs and run them through an AI-enabled qualitative analysis workflow to generate themes, cross-condition comparisons, and representative quotes faster.

To explore how to ingest and analyze DiSCoKit exports, see Evidano features and when you are ready, Try Evidano for free.