Loading...
Paper Number
1978
Paper Type
Short
Abstract
Large Language Models (LLMs), such as ChatGPT, provide immense potential for forecasting as they can incorporate information from large corpora of text to generate predictions. However, LLMs are also prone to decision-making biases, stemming from their training on human-generated data, which inherently contain a diverse array of biases. We uncover a forecasting optimism bias, where GPT -4 provides overly positive evaluations when making predictions by conducting a forecasting experiment with GPT-4 as participants. Our findings reveal a pronounced optimism bias in GPT-4’s forecasts about individual human outcomes. This bias diminishes when the task is reframed to predict non-human-specific outcomes, despite identical task conditions and data. Our other experimental treatments help isolate that this bias is driven, at least in part, by ChatGPT’s ability to provide a positive narrative using qualitative input data. This study holds implications for organisations and researchers developing and implementing LLM-driven approaches to forecasting.
Recommended Citation
Liu, Nicole and Kirshner, Sam, "The Futures Too Bright: ChatGPT's Optimism Forecasting Bias" (2024). ICIS 2024 Proceedings. 10.
https://aisel.aisnet.org/icis2024/data_soc/data_soc/10
The Futures Too Bright: ChatGPT's Optimism Forecasting Bias
Large Language Models (LLMs), such as ChatGPT, provide immense potential for forecasting as they can incorporate information from large corpora of text to generate predictions. However, LLMs are also prone to decision-making biases, stemming from their training on human-generated data, which inherently contain a diverse array of biases. We uncover a forecasting optimism bias, where GPT -4 provides overly positive evaluations when making predictions by conducting a forecasting experiment with GPT-4 as participants. Our findings reveal a pronounced optimism bias in GPT-4’s forecasts about individual human outcomes. This bias diminishes when the task is reframed to predict non-human-specific outcomes, despite identical task conditions and data. Our other experimental treatments help isolate that this bias is driven, at least in part, by ChatGPT’s ability to provide a positive narrative using qualitative input data. This study holds implications for organisations and researchers developing and implementing LLM-driven approaches to forecasting.
When commenting on articles, please be friendly, welcoming, respectful and abide by the AIS eLibrary Discussion Thread Code of Conduct posted here.
Comments
13-DataAnalytics