Know Thyself: The Relationship between Metacognition and Human-AI Collaboration

Presenter Information

Anna Taudien, University of CologneFollow
Dominik David Walzner, University of CologneFollow
Andreas Fuegener, University of CologneFollow
Alok Gupta, university of minnesotaFollow
Wolfgang Ketter, University of CologneFollow

Paper Number

2324

Paper Type

Short

Abstract

While collaboration between humans and artificial intelligence (AI) can yield enhanced performance due to complementarities of both entities, this potential is often not realized if human users manage the collaboration. This is because users need to be aware of their own capabilities and the capabilities of the AI. The users’ ability to monitor their decision-making process is known as metacognition. In this short paper, we investigate the relationship between metacognition and human-AI collaboration. In an exploratory experimental study (n = 51), we quantify users’ metacognitive efficiency and analyze how this affects their performance when receiving AI advice. We find that higher-performing subjects require higher levels of metacognitive efficiency to achieve improved collaborative performance with AI. We are the first to apply the metric of metacognitive efficiency to human-AI collaboration. Our findings have implications for designing AI advice shown to users based on both their individual capabilities and their metacognitive efficiency.

Comments

21-UserBehavior

Recommended Citation

Taudien, Anna; Walzner, Dominik David; Fuegener, Andreas; Gupta, Alok; and Ketter, Wolfgang, "Know Thyself: The Relationship between Metacognition and Human-AI Collaboration" (2024). ICIS 2024 Proceedings. 7.
https://aisel.aisnet.org/icis2024/user_behav/user_behav/7