Paper Number
ICIS2025-1944
Paper Type
Complete
Abstract
Ensuring data quality is a persistent challenge in survey-based research, particularly with the rise of online participant pools prone to inattentiveness and random responding. Traditional quality control methods, such as attention checks and response pattern analyses, add cognitive load and are often domain-specific. In this paper, we explore the use of autoencoders - unsupervised neural networks that learn to reconstruct structured data - as a scalable, domain-agnostic alternative for detecting inattentive survey respondents. Autoencoders can effectively identify response patterns that deviate from typical behavior without requiring labeled data or explicit participant intervention. Across nine real-world survey datasets, our experiments demonstrate that autoencoders consistently improve over baseline predictors, achieving notable reconstruction ability (average Lift > 1.4) and strong inattentiveness detection performance (AUC up to 0.79). We further introduce a modified loss function tailored to survey structures and explore Percentile Loss to enhance detection in challenging cases. These results suggest that autoencoders offer a flexible and automated solution for improving behavioral data reliability, complementing traditional survey quality control techniques.
Recommended Citation
Triantafyllopoulos, Ilias and Ipeirotis, Panos, "Can Autoencoders Replace Attention Checks to Detect Inattentive Survey Respondents?" (2025). ICIS 2025 Proceedings. 11.
https://aisel.aisnet.org/icis2025/is_researchmethods/is_researchmethods/11
Can Autoencoders Replace Attention Checks to Detect Inattentive Survey Respondents?
Ensuring data quality is a persistent challenge in survey-based research, particularly with the rise of online participant pools prone to inattentiveness and random responding. Traditional quality control methods, such as attention checks and response pattern analyses, add cognitive load and are often domain-specific. In this paper, we explore the use of autoencoders - unsupervised neural networks that learn to reconstruct structured data - as a scalable, domain-agnostic alternative for detecting inattentive survey respondents. Autoencoders can effectively identify response patterns that deviate from typical behavior without requiring labeled data or explicit participant intervention. Across nine real-world survey datasets, our experiments demonstrate that autoencoders consistently improve over baseline predictors, achieving notable reconstruction ability (average Lift > 1.4) and strong inattentiveness detection performance (AUC up to 0.79). We further introduce a modified loss function tailored to survey structures and explore Percentile Loss to enhance detection in challenging cases. These results suggest that autoencoders offer a flexible and automated solution for improving behavioral data reliability, complementing traditional survey quality control techniques.
When commenting on articles, please be friendly, welcoming, respectful and abide by the AIS eLibrary Discussion Thread Code of Conduct posted here.
Comments
25-Research