Paper Number
ICIS2025-2504
Paper Type
Complete
Abstract
Advances in AI are giving rise to agentic information systems that increasingly act on users’ behalf—often without their awareness. While such systems promise convenience and support, they also introduce a subtle but ethically significant risk: technological paternalism. This paper explores whether individuals recognize when AI-based assistants overstep, and how they respond to different levels of system proactivity. Based on a field study with 305 participants across 23 everyday scenarios, the findings reveal broad acceptance of soft paternalism, but consistent resistance to fully autonomous actions—even though such actions are not always perceived as autonomy-reducing. These results challenge the assumption that user control is preserved by default and call for a deeper rethinking of agency in ethical AI. As agentic systems become part of everyday life, the field must grapple with where to draw the line between help and interference—and how to ensure that line remains visible to users.
Recommended Citation
Mihale-Wilson, Cristina A., "Invisible Paternalism in Agentic IS: Rethinking Autonomy in the Design of AI-based Support" (2025). ICIS 2025 Proceedings. 11.
https://aisel.aisnet.org/icis2025/ethical_is/ethical_is/11
Invisible Paternalism in Agentic IS: Rethinking Autonomy in the Design of AI-based Support
Advances in AI are giving rise to agentic information systems that increasingly act on users’ behalf—often without their awareness. While such systems promise convenience and support, they also introduce a subtle but ethically significant risk: technological paternalism. This paper explores whether individuals recognize when AI-based assistants overstep, and how they respond to different levels of system proactivity. Based on a field study with 305 participants across 23 everyday scenarios, the findings reveal broad acceptance of soft paternalism, but consistent resistance to fully autonomous actions—even though such actions are not always perceived as autonomy-reducing. These results challenge the assumption that user control is preserved by default and call for a deeper rethinking of agency in ethical AI. As agentic systems become part of everyday life, the field must grapple with where to draw the line between help and interference—and how to ensure that line remains visible to users.
When commenting on articles, please be friendly, welcoming, respectful and abide by the AIS eLibrary Discussion Thread Code of Conduct posted here.
Comments
05-ResponsibleIS