Paper Number
ECIS2025-1604
Paper Type
CRP
Abstract
Understanding the differences and similarities in human and AI decision-making can help illuminate their individual strengths, enabling more effective human-AI collaboration. This study aims to advance this understanding via a comparative analysis of human and AI decision-making. Image classification has proven to be an ideal platform for such analysis, but previous studies have been limited to routine task settings. In contrast, we conduct our comparative analysis in a non-routine task setting, where humans and AI classify Google Street View images. We perform an eye tracking experiment to unveil human decision-making and utilize an explainable AI method to unveil AI decision-making. Our comparative analysis is based on the earth mover’s distance. The study reveals two key findings. First, we find that inter-human similarity is significantly higher than human-AI similarity. Second, human and AI decision-making is the most dissimilar when both the human and the AI make a correct decision.
Recommended Citation
Tille, Christopher; Sparn, Christian; and Klier, Mathias, "It Takes Two to Tango – A Comparative Analysis of Human and AI Decision-Making through Eye Tracking and Explainable AI" (2025). ECIS 2025 Proceedings. 1.
https://aisel.aisnet.org/ecis2025/ai_org/ai_org/1
It Takes Two to Tango – A Comparative Analysis of Human and AI Decision-Making through Eye Tracking and Explainable AI
Understanding the differences and similarities in human and AI decision-making can help illuminate their individual strengths, enabling more effective human-AI collaboration. This study aims to advance this understanding via a comparative analysis of human and AI decision-making. Image classification has proven to be an ideal platform for such analysis, but previous studies have been limited to routine task settings. In contrast, we conduct our comparative analysis in a non-routine task setting, where humans and AI classify Google Street View images. We perform an eye tracking experiment to unveil human decision-making and utilize an explainable AI method to unveil AI decision-making. Our comparative analysis is based on the earth mover’s distance. The study reveals two key findings. First, we find that inter-human similarity is significantly higher than human-AI similarity. Second, human and AI decision-making is the most dissimilar when both the human and the AI make a correct decision.
When commenting on articles, please be friendly, welcoming, respectful and abide by the AIS eLibrary Discussion Thread Code of Conduct posted here.