Paper Type
Complete
Paper Number
1843
Description
This paper investigates the dynamics of human-AI collaboration in content generation, focusing on the use of ChatGPT’s dialog-based interface. Through an exploratory experiment involving 304 participants rating various texts, we examine the impact of dialog-based interaction on the quality of content produced in different settings: Human Control, Human-AI without dialog, Human-AI with dialog and AI-AI with dialog. Our findings reveal that while incorporating AI can enhance content quality, dialog-based interactions often do not improve, and may even degrade, the quality of the initial output. As humans are easily drawn to engage in a dialog with AI, we term this phenomenon “the dialog trap”. This study contributes to the literature by highlighting the nuances of human-AI collaborative writing, particularly in the context of generative AI systems like ChatGPT. It also underscores the importance of considering interaction modes in AI-assisted content generation, offering insights for both researchers and practitioners.
Recommended Citation
Glauben, Adrian; Zöll, Anne; Sharma, Bhavika; and Kristo, Filip, "The Dialog Trap: Exploring Potentially Detrimental Effects of Dialog-Based Interfaces for Generative AI Content Creation" (2024). PACIS 2024 Proceedings. 15.
https://aisel.aisnet.org/pacis2024/track01_aibussoc/track01_aibussoc/15
The Dialog Trap: Exploring Potentially Detrimental Effects of Dialog-Based Interfaces for Generative AI Content Creation
This paper investigates the dynamics of human-AI collaboration in content generation, focusing on the use of ChatGPT’s dialog-based interface. Through an exploratory experiment involving 304 participants rating various texts, we examine the impact of dialog-based interaction on the quality of content produced in different settings: Human Control, Human-AI without dialog, Human-AI with dialog and AI-AI with dialog. Our findings reveal that while incorporating AI can enhance content quality, dialog-based interactions often do not improve, and may even degrade, the quality of the initial output. As humans are easily drawn to engage in a dialog with AI, we term this phenomenon “the dialog trap”. This study contributes to the literature by highlighting the nuances of human-AI collaborative writing, particularly in the context of generative AI systems like ChatGPT. It also underscores the importance of considering interaction modes in AI-assisted content generation, offering insights for both researchers and practitioners.
When commenting on articles, please be friendly, welcoming, respectful and abide by the AIS eLibrary Discussion Thread Code of Conduct posted here.
Comments
AI