HOW MUCH IS THE BLACK BOX? THE VALUE OF EXPLAINABILITY IN MACHINE LEARNING MODELS

Authors

Jonas Wanner, University of WürzburgFollow
Lukas-Valentin Herm, University of WürzburgFollow
Christian Janiesch, University of WürzburgFollow

Abstract

Machine learning enables computers to learn from data and fuels artificial intelligence systems with capabilities to make even super-human decisions. Yet, despite already outperforming preexisting methods and even humans for specific tasks in gaming or healthcare, machine learning faces several challenges related to the uncertainty of the analysis result’s trustworthiness beyond training and validation data. This is because many well-performing algorithms are black boxes to the user who – consequently – cannot trace and understand the reasoning behind a model’s prediction when taking or executing a decision. In response, explainable AI has emerged as a field of study to glass box the former black box models. However, current explainable AI research often neglects the human factor. Against this backdrop, we study from a user perspective the trade-off between completeness, as the accuracy of a model’s prediction, and interpretability, as the way of model prediction understanding. In particular, we evaluate how existing explainable AI model transfers can be used with a focus on the human recipient and derive recommendations for improvements. As a first step, we have identified eleven types of glass box models and defined the fundamentals of a well-founded survey design to understand better the factors that support interpretability and weighing them against improved yet black-boxed completeness.

Recommended Citation

Wanner, Jonas; Herm, Lukas-Valentin; and Janiesch, Christian, "HOW MUCH IS THE BLACK BOX? THE VALUE OF EXPLAINABILITY IN MACHINE LEARNING MODELS" (2020). ECIS 2020 Research-in-Progress Papers. 85.
https://aisel.aisnet.org/ecis2020_rip/85