Paper Number

1307

Paper Type

Short Paper

Abstract

Humans frequently receive algorithmic advice for prediction tasks. However, they often combine their own judgment with advice in a biased way which can be harmful for predictive performance. So far, such kind of biased behavior has been found for humans receiving one algorithmic advice. It is likely that humans will increasingly be advised by more than one algorithm, as it is already the case with ensemble methods. We study the effect of disclosing multiple algorithmic advisors on users’ weight on advice. We conduct a between-subjects experiment (n = 192) in which we manipulate whether the underlying individual algorithmic predictions are shown in addition to an ensemble prediction. In line with findings on naïve diversification, we observe that humans place a higher weight on an ensemble when individual predictions are presented. Our findings contribute to literature on biases in AI-advised settings and have implications for the presentation of ensemble predictions.

Share

COinS
 
Jun 14th, 12:00 AM

One Versus Many: Advice-Taking from Ensemble Algorithms

Humans frequently receive algorithmic advice for prediction tasks. However, they often combine their own judgment with advice in a biased way which can be harmful for predictive performance. So far, such kind of biased behavior has been found for humans receiving one algorithmic advice. It is likely that humans will increasingly be advised by more than one algorithm, as it is already the case with ensemble methods. We study the effect of disclosing multiple algorithmic advisors on users’ weight on advice. We conduct a between-subjects experiment (n = 192) in which we manipulate whether the underlying individual algorithmic predictions are shown in addition to an ensemble prediction. In line with findings on naïve diversification, we observe that humans place a higher weight on an ensemble when individual predictions are presented. Our findings contribute to literature on biases in AI-advised settings and have implications for the presentation of ensemble predictions.

When commenting on articles, please be friendly, welcoming, respectful and abide by the AIS eLibrary Discussion Thread Code of Conduct posted here.