Paper Number
ECIS2026-2493
Paper Type
SP
Abstract
Artificial intelligence (AI) increasingly supports organisational decision-making, yet users often miscalibrate how they incorporate AI advice. While prior work highlights cognitive and emotional factors, less is known about how workflow design shapes advice utilisation in repeated interactions with fallible systems. We examine two easily implementable design levers, advice order (AI-First vs. Human-First) and accuracy warnings, and how they influence advice utilisation and its cognitive drivers. We argue that the order determines the initial anchor of the decision process, while accuracy warnings shape perceptions of system performance. We present a 2×2 laboratory experiment in an aviation-inspired forecasting task that systematically manipulates these levers and captures behavioural and cognitive responses across repeated decisions. This research-in-progress outlines the conceptual framework, experimental design, and expected contributions for understanding how workflow design shapes advice utilisation in human-AI collaboration.
Recommended Citation
Suffel, Maren, "Anchor-In-The-Loop: How Advice Timing and Accuracy Warnings Shape The Utilisation Of Artificially Intelligent Advice" (2026). ECIS 2026 Proceedings. 30.
https://aisel.aisnet.org/ecis2026/cog_hbis/cog_hbis/30
Anchor-In-The-Loop: How Advice Timing and Accuracy Warnings Shape The Utilisation Of Artificially Intelligent Advice
Artificial intelligence (AI) increasingly supports organisational decision-making, yet users often miscalibrate how they incorporate AI advice. While prior work highlights cognitive and emotional factors, less is known about how workflow design shapes advice utilisation in repeated interactions with fallible systems. We examine two easily implementable design levers, advice order (AI-First vs. Human-First) and accuracy warnings, and how they influence advice utilisation and its cognitive drivers. We argue that the order determines the initial anchor of the decision process, while accuracy warnings shape perceptions of system performance. We present a 2×2 laboratory experiment in an aviation-inspired forecasting task that systematically manipulates these levers and captures behavioural and cognitive responses across repeated decisions. This research-in-progress outlines the conceptual framework, experimental design, and expected contributions for understanding how workflow design shapes advice utilisation in human-AI collaboration.
When commenting on articles, please be friendly, welcoming, respectful and abide by the AIS eLibrary Discussion Thread Code of Conduct posted here.