Paper Number
1307
Paper Type
Short Paper
Abstract
Humans frequently receive algorithmic advice for prediction tasks. However, they often combine their own judgment with advice in a biased way which can be harmful for predictive performance. So far, such kind of biased behavior has been found for humans receiving one algorithmic advice. It is likely that humans will increasingly be advised by more than one algorithm, as it is already the case with ensemble methods. We study the effect of disclosing multiple algorithmic advisors on users’ weight on advice. We conduct a between-subjects experiment (n = 192) in which we manipulate whether the underlying individual algorithmic predictions are shown in addition to an ensemble prediction. In line with findings on naïve diversification, we observe that humans place a higher weight on an ensemble when individual predictions are presented. Our findings contribute to literature on biases in AI-advised settings and have implications for the presentation of ensemble predictions.
Recommended Citation
Taudien, Anna; Schiffels, Sebastian; Fuegener, Andreas; Gupta, Alok; and Ketter, Wolfgang, "One Versus Many: Advice-Taking from Ensemble Algorithms" (2024). ECIS 2024 Proceedings. 9.
https://aisel.aisnet.org/ecis2024/track09_coghbis/track09_coghbis/9
One Versus Many: Advice-Taking from Ensemble Algorithms
Humans frequently receive algorithmic advice for prediction tasks. However, they often combine their own judgment with advice in a biased way which can be harmful for predictive performance. So far, such kind of biased behavior has been found for humans receiving one algorithmic advice. It is likely that humans will increasingly be advised by more than one algorithm, as it is already the case with ensemble methods. We study the effect of disclosing multiple algorithmic advisors on users’ weight on advice. We conduct a between-subjects experiment (n = 192) in which we manipulate whether the underlying individual algorithmic predictions are shown in addition to an ensemble prediction. In line with findings on naïve diversification, we observe that humans place a higher weight on an ensemble when individual predictions are presented. Our findings contribute to literature on biases in AI-advised settings and have implications for the presentation of ensemble predictions.
When commenting on articles, please be friendly, welcoming, respectful and abide by the AIS eLibrary Discussion Thread Code of Conduct posted here.