Paper Number
2917
Paper Type
Complete
Description
Current guidelines suggest setting low expectations for chatbots for continued engagement. That is of little use if only high expectations entice users. Such are raised by recent innovations, chatbot agents which generate final responses not directly via Large Language Models but after multiple intermediate generations of text as reasoning. This differentiates chatbot agents from other chatbots. Since these intermediate steps are generated, what impact has displaying them? This study aims to assess the impact of displaying these intermediate reasoning steps, conceptualizing them as enabling mindreading in the semi-literal sense, namely reading the internal reasoning generated by the agent. In 3 studies (N=280), we examine whether non-factive displays, which present reasoning as belief rather than knowledge, enhance user expectation confirmation. Results show that non-factive reasoning displays significantly improved confirmation. Therefore, displaying non-factive reasoning not only differentiates these agents in competitive markets but also improves user interaction.
Recommended Citation
Göldi, Andreas and Rietsche, Roman, "Chatbot Agents Displaying Non-factive Reasoning Enhance Expectation Confirmation" (2024). ICIS 2024 Proceedings. 8.
https://aisel.aisnet.org/icis2024/humtechinter/humtechinter/8
Chatbot Agents Displaying Non-factive Reasoning Enhance Expectation Confirmation
Current guidelines suggest setting low expectations for chatbots for continued engagement. That is of little use if only high expectations entice users. Such are raised by recent innovations, chatbot agents which generate final responses not directly via Large Language Models but after multiple intermediate generations of text as reasoning. This differentiates chatbot agents from other chatbots. Since these intermediate steps are generated, what impact has displaying them? This study aims to assess the impact of displaying these intermediate reasoning steps, conceptualizing them as enabling mindreading in the semi-literal sense, namely reading the internal reasoning generated by the agent. In 3 studies (N=280), we examine whether non-factive displays, which present reasoning as belief rather than knowledge, enhance user expectation confirmation. Results show that non-factive reasoning displays significantly improved confirmation. Therefore, displaying non-factive reasoning not only differentiates these agents in competitive markets but also improves user interaction.
Comments
09-HTI