Paper Number
2438
Paper Type
Complete
Description
As affective political polarization intensifies, emotionally charged language becomes increasingly pervasive. Simultaneously, large language models (LLMs) play a more central role in information-seeking behavior. In this study, we examine the impact of emotionally loaded language in prompts on the factual accuracy of LLM responses within politically sensitive contexts. Specifically, we investigate how infusing prompts along the six core emotional dimensions affect the accuracy of GPT-4’s responses to political questions compared to neutral prompts. Our findings reveal that emotionally loaded prompts lead to a significant improvement in response accuracy, with an average increase of 11.8%. However, the increase in accuracy depends on whether the factually accurate response reflects positively on Democratic or Republican positions, suggesting that the emotional tone of prompts may introduce or exacerbate political bias in LLM outputs. We discuss the potential underlying mechanisms in the model’s training and the broader societal implications of our findings.
Recommended Citation
Knickrehm, Charlotte and Bauer, Kevin, "GPT, Emotions, and Facts" (2024). ICIS 2024 Proceedings. 31.
https://aisel.aisnet.org/icis2024/aiinbus/aiinbus/31
GPT, Emotions, and Facts
As affective political polarization intensifies, emotionally charged language becomes increasingly pervasive. Simultaneously, large language models (LLMs) play a more central role in information-seeking behavior. In this study, we examine the impact of emotionally loaded language in prompts on the factual accuracy of LLM responses within politically sensitive contexts. Specifically, we investigate how infusing prompts along the six core emotional dimensions affect the accuracy of GPT-4’s responses to political questions compared to neutral prompts. Our findings reveal that emotionally loaded prompts lead to a significant improvement in response accuracy, with an average increase of 11.8%. However, the increase in accuracy depends on whether the factually accurate response reflects positively on Democratic or Republican positions, suggesting that the emotional tone of prompts may introduce or exacerbate political bias in LLM outputs. We discuss the potential underlying mechanisms in the model’s training and the broader societal implications of our findings.
Comments
10-AI