Abstract

Abstract Contemporary artificial intelligence (AI) systems offer substantial benefits, but they often operate as “black boxes,” making it difficult to understand the reasoning behind their recommendations (Gunning et al., 2019). Although explainable artificial intelligence (XAI) has been widely argued as a remedy, empirical evidence suggests that explanations do not uniformly improve users’ understanding of AI recommendations (Chan, 2023). Thus, a key open question is why different XAI methods produce divergent cognitive and behavioral effects. The proposed study compares two widely used yet conceptually distinct XAI approaches: feature-importance and counterfactual. These methods are grounded in philosophical theories of explanation that enable us to make a theoretically informed comparison. By showing how much each input contributes to the output, feature-importance explanations (e.g., SHAP, LIME) align with the mechanistic view of explanation (Salmon, 1984). In contrast, the counterfactual method aligns with the counterfactual view of explanation (Lewis, 1973) by describing how changes to the inputs would lead to different outcomes. We argue that these differing explanatory logics have important cognitive implications. Specifically, we focus on users’ illusion of explanatory depth (Rozenblit & Keil, 2002), which happens when users believe they understand a system more deeply than they actually do. We hypothesize that counterfactual explanations induce a stronger illusion of explanatory depth than feature-importance explanations. This is because counterfactuals are more intuitive and simplified and might not accurately reflect the original model’s underlying logic. The illusion of explanatory depth further impairs the decision quality of individuals. However, domain expertise acts as a moderator, such that experts are less susceptible to counterfactual-induced illusions due to their existing knowledge of relevant features and outcomes. We plan to use a laboratory experiment where participants will be randomly assigned to receive either feature-importance or counterfactual explanations for the AI recommendation. The illusion of explanatory depth will be operationalized as the discrepancy between perceived and objective understanding. Also, participants will be asked to make a decision that will then be assessed for decision quality. This study contributes to XAI research by grounding XAI effects in philosophical theories and identifying illusion of explanatory depth as a key mechanism linking explainability to decision outcomes. Practically, the findings caution against uncritical use of intuitive explanations.

Download

Share

COinS