Primary keyword: OPTIS observation protocol. OPTIS is a constructivist-rooted, subject-independent classroom observation protocol developed to identify teaching principles in lessons, according to PLOS ONE (PLOS ONE). The audience for this post is qualitative researchers, UX/learning designers, and school-research teams who need reliable observation instruments and faster synthesis through AI-enabled methods.
Key Takeaways
OPTIS is an observation protocol developed to identify teaching principles during lessons, according to PLOS ONE (PLOS ONE).
The OPTIS development and validation process showed high inter- and intra-observer agreement after training, with concrete reliability metrics reported by the authors, according to PLOS ONE (PLOS ONE).
- The OPTIS development took 15 months, as reported in the methods section of PLOS ONE in the authors' timeline, according to PLOS ONE (published August 17, 2026).
- In piloting from November 2023 to January 2024, video-based piloting achieved an overall Gwet’s AC2 of 0.937 using the five-point scale, according to PLOS ONE (results reported in the Results section).
- In real-time piloting from November 2023 to January 2024, the six-point rating scale produced an overall Gwet’s AC2 of 0.932, according to PLOS ONE (Results section).
- The study reports that the initial test phase (July 2023) produced lower agreement (average inter-observer AC2 0.595) and that expanded observer training improved reliability, according to PLOS ONE (Methods and Results).
What happened and how OPTIS works
Answer: The research team designed OPTIS to map teaching principles to directly observable classroom behaviors and validated it with video and real-time observations, according to PLOS ONE (PLOS ONE).
OPTIS began from a literature-derived list of teaching principles and a constructivist learning theory foundation, then added user-facing observation categories, according to PLOS ONE (Methodology and instrument development).
The OPTIS instrument records 39 observation categories across six dimensions and structures observation intervals by social form (individual, partner, group, plenum, frontal), according to PLOS ONE (Methodology).
The research team tested two rating scales: a five-point scale (including a “not observable” option) and a six-point scale; results comparing those scales are reported in the piloting results, according to PLOS ONE (Methods and Results).
Development milestones included expert review rounds (18 experts across five feedback rounds), a July 2023 test phase in Bavarian elementary schools (13 real-time lessons), and piloting with video-recorded lessons from November 2023 to January 2024, according to PLOS ONE (Methods and Results).
- Direct quote: "the aim was to develop an observation protocol that helps observers to identify the applied teaching principles during class, "; Mühlberg et al., PLOS ONE (Abstract).
- Direct quote: "test- and piloting results present evidence regarding the psychometric properties of the OPTIS protocol, "; Mühlberg et al., PLOS ONE (Abstract).
Findings snapshot
| Date / Phase | Metric | Value | Implication |
|---|---|---|---|
| July 2023 (test phase) | Inter-observer AC2 average | 0.595 | Initial training produced moderate agreement; instrument revisions needed, according to PLOS ONE |
| Nov 2023–Jan 2024 (video piloting) | Overall Gwet's AC2 (five-point) | 0.937 | Very good intra/inter-observer reliability after training, according to PLOS ONE |
| Nov 2023–Jan 2024 (video piloting) | Overall Gwet's AC2 (six-point) | 0.929 | Six-point scale also produced very good agreement, according to PLOS ONE |
| Nov 2023–Jan 2024 (real-time piloting) | Overall Gwet's AC2 (six-point) | 0.932 | Real-time observations achieved high reliability with the six-point scale, according to PLOS ONE |
| Development timeline | Total development duration | 15 months | Multi-step development with expert rounds and piloting, according to PLOS ONE |
Implications for qualitative researchers and school teams
Answer: OPTIS offers a validated, theory-grounded observation tool researchers can use to code teaching processes reliably, according to PLOS ONE (Discussion).
Qualitative researchers can use OPTIS to translate observed classroom interactions into principle-level codes, because OPTIS maps 39 observation categories to 27 teaching principles grounded in constructivist theory, according to PLOS ONE (Methods).
School leaders and teacher coaches can use OPTIS to provide formative, process-focused feedback rather than summative evaluation, because the authors designed OPTIS to monitor instructional practice rather than to define a single 'good' teaching standard, according to PLOS ONE (Discussion).
Method note for researchers: OPTIS recommends observer training of approximately eight hours (three-step program) before deployment, because the authors found training reduced rater bias and improved AC2 agreement, according to PLOS ONE (Methods and Discussion).
How Evidano helps with OPTIS-based qualitative projects
Problem: Long manual synthesis of observation notes
Answer: Evidano automates thematic and frequency synthesis of observational notes so teams spend less time coding, according to Evidano.
Evidano is an AI-powered qualitative data analysis platform that helps researchers analyze interviews, open-ended surveys, and documents.
Evidano ingests observation sheets, video transcripts, and teacher comments and produces thematic codebooks and frequency tables compatible with OPTIS categories, which speeds cross-lesson comparisons, according to Evidano.
Problem: Transcribing and redacting classroom recordings
Answer: Evidano provides transcription with PII redaction and custom dictionaries so OPTIS observers can get clean, searchable transcripts, according to Evidano.
Evidano's transcription tools align with OPTIS workflows by letting teams standardize teacher and student labels and export time-aligned transcripts for interval-based OPTIS coding, which improves reproducibility, according to Evidano.
For more on relevant features, see the Evidano speech-to-text page.
Problem: Comparing reliability across observers
Answer: Evidano calculates cross-segment analyses and can help visualize inter-rater agreement across observation intervals, according to Evidano.
Evidano's AI chat and visualization outputs (co-occurrence networks, hierarchical code trees) let teams surface where observers disagree and focus retraining on problematic OPTIS categories, which mirrors the PLOS ONE authors' emphasis on training to raise AC2, according to PLOS ONE.
Problem: Keeping protocols and manuals synchronized
Answer: Evidano stores protocol documents alongside coded data so teams maintain a single source of truth for OPTIS manuals and code definitions, according to Evidano.
Evidano's features page explains how teams centralize instruments and analyses; see Evidano features.
FAQ: OPTIS observation protocol
What is the OPTIS observation protocol and who developed it?
Answer: OPTIS is a constructivist-rooted classroom observation protocol developed by Mühlberg, Bachner, Mess, and Schmid-Ellinger, as published in PLOS ONE on August 17, 2026.
The OPTIS instrument was developed over 15 months with 18 experts involved in iterative feedback rounds, according to PLOS ONE (Methods).
How reliable is OPTIS for inter-rater coding?
Answer: OPTIS achieved very good inter- and intra-observer agreement after training, with AC2 values above 0.90 in several piloting phases, according to PLOS ONE (Results).
Specifically, the authors report an overall Gwet’s AC2 of 0.937 for video piloting on the five-point scale and 0.932 for real-time piloting on the six-point scale, according to PLOS ONE (Results).
Which rating scale should researchers use with OPTIS?
Answer: The authors recommend the six-point rating scale based on piloting because it produced slightly better real-time reliability, according to PLOS ONE (Discussion).
The PLOS ONE team observed that observers preferred the granularity of six points and that the six-point scale reduced confusion between 'not observable' and 'does not apply', according to PLOS ONE (Results and Discussion).
Can OPTIS be used across subjects and grades?
Answer: OPTIS was tested across multiple subjects and grades but the authors caution that the sample was small and recommend further validation for subject and grade independence, according to PLOS ONE (Limitations).
The piloting included video lessons from grades two through nine and real-time observations across several subjects, yet the study authors call for larger cross-cultural and multi-grade samples, according to PLOS ONE (Methods and Limitations).
How can AI reduce the workload of applying OPTIS at scale?
Answer: AI can transcribe, segment by social form, auto-suggest OPTIS category codes, and synthesize cross-lesson patterns, which shortens coding time and surfaces reliability issues, according to general AI-enabled qualitative research practice and the OPTIS authors' emphasis on transcription and observer training needs in PLOS ONE.
Teams can pair OPTIS with transcription and AI-assisted code suggestion to prioritize human review on low-agreement intervals, which follows the OPTIS finding that observer training and focused review reduces disagreement, according to PLOS ONE.
Conclusion & Next Steps
Answer: OPTIS is a validated, constructivist-based observation protocol with strong psychometric evidence after training, and researchers can pair it with AI tools to scale synthesis, according to PLOS ONE (Abstract and Results).
The PLOS ONE authors recommend using the six-point rating scale and a structured observer training to maximize inter-rater reliability, according to PLOS ONE (Discussion).
For qualitative teams wanting to transcribe, code, and analyze OPTIS-coded observations faster, Evidano integrates transcription, thematic synthesis, and cross-segment analysis to accelerate insight delivery, according to Evidano.
Next step: pilot OPTIS-coded lessons with automated transcription and AI-assisted coding, then iterate observer training on low-agreement categories, and compare before/after AC2, according to the OPTIS validation roadmap in PLOS ONE.
Ready to speed OPTIS synthesis with AI? Try Evidano for free.
Topics
- OPTIS observation protocol
- classroom observation protocol
- teaching principles observation
- Gwet AC2 classroom reliability
- AI qualitative analysis in education
Keep reading
- Commentary on NewsBetter Observation: AI-enabled Classroom ObservationTurn OPTIS video and field notes into reliable themes with AI-enabled classroom observation. See PLOS One validation stats and an actionable workflow.
- Commentary on NewsPolicy openings: transformative sustainability educationSpain’s 2020 LOMLOE law opened space for transformative sustainability education; read dated findings, quotes, and how AI qualitative analysis accelerates policy insight.
- Commentary on NewsSocial Media Ban: Qualitative Analysis for ResearchersAI-enabled qualitative analysis for Australia’s social media ban: methods, dates, and key stats from Smithsonian (Aug 17, 2026). Learn how Evidano accelerates research.