Paper Number
ICIS2025-2283
Paper Type
Short
Abstract
When and why do humans overrule algorithmic recommendations? Despite increasing use of AI systems in organizations, answers to this question are ambiguous. Some empirical evidence suggests that humans overrule due to unconscious biases, thereby worsening performance outcomes. Simultaneously, organizational research suggests that professional practices are highly contextual, and workers might have situated reasons to overrule. In response, we study AI overruling in an organizational context where tacit knowledge and intrinsic human engagement are key: the craftwork of baking. We analyze almost 1.000.000 forecasting decisions and qualitative interviews from a bakery chain in Central Europe. Following a computationally-intensive theorizing approach, we find that craft workers deliberately overrule algorithmic forecasts by drawing on craft-specific skills and attitudes. First, they integrate situated and sensory information that would otherwise not be considered by the predictive optimization- focused AI system. Second, they consider motives that defy logics of economic efficiency, yet central to their craftwork.
Recommended Citation
Schmitt, Anuschka; Fahse, Tobias Benjamin; and Grisold, Thomas, "When We Know More than AI Can Tell: A Computational Study of AI Overruling in Craftwork" (2025). ICIS 2025 Proceedings. 13.
https://aisel.aisnet.org/icis2025/isdesign/isdesign/13
When We Know More than AI Can Tell: A Computational Study of AI Overruling in Craftwork
When and why do humans overrule algorithmic recommendations? Despite increasing use of AI systems in organizations, answers to this question are ambiguous. Some empirical evidence suggests that humans overrule due to unconscious biases, thereby worsening performance outcomes. Simultaneously, organizational research suggests that professional practices are highly contextual, and workers might have situated reasons to overrule. In response, we study AI overruling in an organizational context where tacit knowledge and intrinsic human engagement are key: the craftwork of baking. We analyze almost 1.000.000 forecasting decisions and qualitative interviews from a bakery chain in Central Europe. Following a computationally-intensive theorizing approach, we find that craft workers deliberately overrule algorithmic forecasts by drawing on craft-specific skills and attitudes. First, they integrate situated and sensory information that would otherwise not be considered by the predictive optimization- focused AI system. Second, they consider motives that defy logics of economic efficiency, yet central to their craftwork.
When commenting on articles, please be friendly, welcoming, respectful and abide by the AIS eLibrary Discussion Thread Code of Conduct posted here.
Comments
13-DesignDevPM