Paper Type
Complete
Abstract
By providing value-based explanations using LIME and adjacent methods, explainable AI can provide insights into AI systems’ black box internal decision-making processes. However, this approach often does not aid laypeople without additional text-based annotations. As such annotations are expensive to produce manually and problematic to provide in real-time, we argue that combining proven decision support systems for the generation of a decision, LIME-like value-based explanations, and corresponding LLM-generated annotations may address these two challenges. More specifically, if LLM generated annotations caused no differences in user perceptions, these could be generated with lower cost and in real-time without downsides. Hence, we conducted an experiment to compare the impact of LLM versus human-generated annotations on user perceptions of a system. Based on metrics commonly assessed for explainable AI systems, our findings suggest annotations produced by LLMs to be comparable to human generated ones, thus suggesting the deployment of such annotations.
Paper Number
1197
Recommended Citation
de Zoeten, Marc Christoph; Ernst, Claus-Peter H.; and Staegemann, Daniel, "Perceptions of Annotated Explanations in Explainable AI: Humans versus ChatGPT" (2025). AMCIS 2025 Proceedings. 2.
https://aisel.aisnet.org/amcis2025/sig_svc/sig_svc/2
Perceptions of Annotated Explanations in Explainable AI: Humans versus ChatGPT
By providing value-based explanations using LIME and adjacent methods, explainable AI can provide insights into AI systems’ black box internal decision-making processes. However, this approach often does not aid laypeople without additional text-based annotations. As such annotations are expensive to produce manually and problematic to provide in real-time, we argue that combining proven decision support systems for the generation of a decision, LIME-like value-based explanations, and corresponding LLM-generated annotations may address these two challenges. More specifically, if LLM generated annotations caused no differences in user perceptions, these could be generated with lower cost and in real-time without downsides. Hence, we conducted an experiment to compare the impact of LLM versus human-generated annotations on user perceptions of a system. Based on metrics commonly assessed for explainable AI systems, our findings suggest annotations produced by LLMs to be comparable to human generated ones, thus suggesting the deployment of such annotations.
When commenting on articles, please be friendly, welcoming, respectful and abide by the AIS eLibrary Discussion Thread Code of Conduct posted here.
Comments
SIGSVC