Paper Number
1461
Paper Type
Complete Research Paper
Abstract
How do physicians integrate AI tools into medical decision-making? Prior research has analyzed extensively whether they exhibit AI algorithmic aversion or appreciation. Yet we argue that these behavioral outcomes arise from underlying decision-making heuristics such as pro-innovation bias, ambiguity aversion, or commitment bias. In this qualitative study, we examined 330 clinical decisions using “think aloud” protocols to identify heuristics employed with AI and explainable AI (XAI). We observed the presence of multiple heuristics, including a “mere exposure effect” and “false confirmation bias”. These heuristics were associated with decision-making errors. The “mere exposure effect” occurred commonly with XAI, when physicians, feeling uncertain about their diagnoses, altered their decision to an incorrect AI diagnosis. False confirmation errors also emerged when AI confirmed an erroneous diagnosis, precluding doctors from seeking alternative information. We also discuss how cognitive interventions could redress these heuristics in decision-making to better optimize accuracy.
Recommended Citation
Rosenbacke, Rikard, "Heuristics and Errors in XAI-Augmented Clinical Decision-Making: Moving Beyond Algorithmic Appreciation and Aversion" (2024). ECIS 2024 Proceedings. 11.
https://aisel.aisnet.org/ecis2024/track03_ai/track03_ai/11
Heuristics and Errors in XAI-Augmented Clinical Decision-Making: Moving Beyond Algorithmic Appreciation and Aversion
How do physicians integrate AI tools into medical decision-making? Prior research has analyzed extensively whether they exhibit AI algorithmic aversion or appreciation. Yet we argue that these behavioral outcomes arise from underlying decision-making heuristics such as pro-innovation bias, ambiguity aversion, or commitment bias. In this qualitative study, we examined 330 clinical decisions using “think aloud” protocols to identify heuristics employed with AI and explainable AI (XAI). We observed the presence of multiple heuristics, including a “mere exposure effect” and “false confirmation bias”. These heuristics were associated with decision-making errors. The “mere exposure effect” occurred commonly with XAI, when physicians, feeling uncertain about their diagnoses, altered their decision to an incorrect AI diagnosis. False confirmation errors also emerged when AI confirmed an erroneous diagnosis, precluding doctors from seeking alternative information. We also discuss how cognitive interventions could redress these heuristics in decision-making to better optimize accuracy.
When commenting on articles, please be friendly, welcoming, respectful and abide by the AIS eLibrary Discussion Thread Code of Conduct posted here.