Paper Type
ERF
Abstract
Despite explosive interest and extensive research into Generative AI’s technical architectures, its emergent behavioral characteristics in complex human-AI interactions remain underexplored. As GenAI is increasingly adopted as a collaborative decision-making tool, a critical risk emerges: its tendency to align conclusions with the user's latent preferences or emotional states, often at the expense of objective reasoning. This sycophantic behavior poses severe threats to decision quality. This research proposes a conceptual framework to analyze and evaluate GenAI's behavioral conformity in decision-making contexts. Grounded in cognitive psychology and behavioral economics, we introduce multi-dimensional evaluation metrics to measure whether GenAI alters its conclusions based on objective evidence or user-driven biases. This sets the foundation for future studies to quantify GenAI behavior, ultimately contributing to the design of more robust and objective AI decision support systems.
Paper Number
1395
Recommended Citation
Lee, Heeseung; Son, Jaebong; and Lee, Chang Heon, "The Sycophancy Trap: Measuring the Behavioral Conformity of Generative AI" (2026). AMCIS 2026 Proceedings. 3.
https://aisel.aisnet.org/amcis2026/sig_odis/sig_odis/3
The Sycophancy Trap: Measuring the Behavioral Conformity of Generative AI
Despite explosive interest and extensive research into Generative AI’s technical architectures, its emergent behavioral characteristics in complex human-AI interactions remain underexplored. As GenAI is increasingly adopted as a collaborative decision-making tool, a critical risk emerges: its tendency to align conclusions with the user's latent preferences or emotional states, often at the expense of objective reasoning. This sycophantic behavior poses severe threats to decision quality. This research proposes a conceptual framework to analyze and evaluate GenAI's behavioral conformity in decision-making contexts. Grounded in cognitive psychology and behavioral economics, we introduce multi-dimensional evaluation metrics to measure whether GenAI alters its conclusions based on objective evidence or user-driven biases. This sets the foundation for future studies to quantify GenAI behavior, ultimately contributing to the design of more robust and objective AI decision support systems.
Comments
SIGODIS