Paper Number

ICIS2026-2832

Paper Type

Completed

Abstract

Generative AI is increasingly embedded in knowledge work, requiring users to verify fluent but error-prone outputs. We report a preregistered between-subjects experiment with 280 participants from a university-centered convenience sample who verified three AI-generated summaries containing planted surface, logic, and hallucination errors. Participants received standard usage or one of three cognitive-forcing implementations: a 20-second delay, pre-AI source notes, or a post-exposure category checklist. No intervention detectably improved error detection in this continuous-text task; the pooled contrast was near zero (d = −0.033, 95% CI [−0.293, 0.226]). Checklist and Decision First increased task time, while Checklist increased self-reported critical evaluation; yet 51.3% of error-linked checklist flags were not accompanied by error correction. These findings establish, within this task, that oversight and effective correction are empirically separable outcomes. They document an oversight-correction gap and motivate task intervention fit as a candidate boundary condition for cognitive forcing functions.

Comments

14_User_behavior

Download

Share

COinS
 
Dec 14th, 12:00 AM

Fluent but False: The Oversight-Correction Gap in Generative AI Verification

Generative AI is increasingly embedded in knowledge work, requiring users to verify fluent but error-prone outputs. We report a preregistered between-subjects experiment with 280 participants from a university-centered convenience sample who verified three AI-generated summaries containing planted surface, logic, and hallucination errors. Participants received standard usage or one of three cognitive-forcing implementations: a 20-second delay, pre-AI source notes, or a post-exposure category checklist. No intervention detectably improved error detection in this continuous-text task; the pooled contrast was near zero (d = −0.033, 95% CI [−0.293, 0.226]). Checklist and Decision First increased task time, while Checklist increased self-reported critical evaluation; yet 51.3% of error-linked checklist flags were not accompanied by error correction. These findings establish, within this task, that oversight and effective correction are empirically separable outcomes. They document an oversight-correction gap and motivate task intervention fit as a candidate boundary condition for cognitive forcing functions.

When commenting on articles, please be friendly, welcoming, respectful and abide by the AIS eLibrary Discussion Thread Code of Conduct posted here.