Process-Focused Assessment: Assessing Coding with AI

According to O'Reilly Radar (July 28, 2026), Eric Freeman argues that assessment must shift from finished artifacts to visible process: show drafts, make in-class work public, and use AI as a conversational assessor.

• Freeman reported on July 28, 2026 that finished programs often reflect prompts more than student thinking, creating an assessment gap.
• A Stanford HAI study (July 10, 2023) found that 61.22% of TOEFL essays were falsely flagged as AI-generated when detectors were applied, showing detector unreliability.
• Freeman documents public, performative practices such as an algorave on November 20, 2025 that force students to demonstrate live fluency rather than submit polished artifacts.

Eric Freeman reported in O'Reilly Radar on July 28, 2026 that generative AI has made finished student code a weaker signal of learning because the code can reflect prompts rather than reasoning.

Freeman described three concrete classroom responses used at the University of Texas at Austin's Arts and Entertainment Technologies Department: public studio work, role-inverted AI conversational assessment, and live coding performances.

"A finished program now tells us more about a student’s prompts than their ideas, " Eric Freeman wrote in O'Reilly Radar (July 28, 2026).

Freeman notes that the studio model collects process evidence: sketches, revisions, prompt histories, and oral explanations, which together provide a richer qualitative record than a final submission alone.

The direct answer is that assessment must measure process, explanation, revision, and recovery, not just final artifacts, as argued by Eric Freeman in O'Reilly Radar (July 28, 2026).

• Design in-class, public milestones so students must explain choices and show drafts, which Freeman implemented at UT Austin starting in 2024 and reported in 2026.
• Use AI as an active conversational assessor that produces transcripts and rubric-aligned summaries, as Freeman described with the Vera Molnár avatar chatbot piloted in 2025.
• Introduce performative assessments such as live coding or demonstrations, exemplified by the AudioPixel Collider algorave on November 20, 2025, to reveal fluency under pressure.

Freeman cautions that these are experimental approaches and not universal solutions; he notes that his observations are based on small-scale experiments rather than controlled trials.

The bottom line from Eric Freeman in O'Reilly Radar (July 28, 2026) is clear: as AI makes polished code easier to produce, assessment must focus on process, explanation, revision, and performance.

Instructors can combine studio practices, AI conversational assessments, and live demonstrations to collect verifiable evidence of understanding.

To pilot these methods, capture transcripts, prompt histories, and revision logs, then run thematic and rubric analyses to surface learning gaps.

Get started by exploring how an AI-native qualitative platform can ingest transcripts and code logs and produce rubric-aligned summaries: Try Evidano for free.