Paper Number
ECIS2026-1425
Paper Type
CRP
Abstract
While the role of transparency in adopting AI-based decision support systems has been widely discussed, the role of explainability in adopting autonomous decision systems, widely known as AI agents, remains widely undiscovered. Empirical findings show that when a system is prescriptive and autonomous in nature, the acceptance of these systems decreases. Against this backdrop, we focus on evaluating autonomous AI agents trained via Reinforcement Learning and provide a functional and human-based evaluation of the effects of explainability approaches from the field of Explainable Reinforcement Learning (XRL). As a result, we reveal issues of faithfulness of causal explanations via functional evaluation. Furthermore, we can observe differing results in the human-based setting between predicting an AI agent’s behavior and understanding it regarding the perceived usefulness of descriptive, generative, and causal explanation approaches.
Recommended Citation
Heinrich, Kai; Keshavarzi, Armin; and Arslan, Sabahat Nagihan Köksalan, "Explain Decisions, Not Just Predictions! An Evaluation Of AI Agent Transparency" (2026). ECIS 2026 Proceedings. 3.
https://aisel.aisnet.org/ecis2026/bus_analytics/bus_analytics/3
Explain Decisions, Not Just Predictions! An Evaluation Of AI Agent Transparency
While the role of transparency in adopting AI-based decision support systems has been widely discussed, the role of explainability in adopting autonomous decision systems, widely known as AI agents, remains widely undiscovered. Empirical findings show that when a system is prescriptive and autonomous in nature, the acceptance of these systems decreases. Against this backdrop, we focus on evaluating autonomous AI agents trained via Reinforcement Learning and provide a functional and human-based evaluation of the effects of explainability approaches from the field of Explainable Reinforcement Learning (XRL). As a result, we reveal issues of faithfulness of causal explanations via functional evaluation. Furthermore, we can observe differing results in the human-based setting between predicting an AI agent’s behavior and understanding it regarding the perceived usefulness of descriptive, generative, and causal explanation approaches.
When commenting on articles, please be friendly, welcoming, respectful and abide by the AIS eLibrary Discussion Thread Code of Conduct posted here.