Paper Type

Complete

Abstract

Vignettes are widely used in chatbot research to manipulate interface and design cues, but the unique characteristics of dyadic chat interactions raise questions about whether observing a vignette produces the same perceptions as participating in a live conversation. In a 2 × 2 × 2 between-subjects experiment, we compare perceptions of anthropomorphism across vignette and interactive chatbot conditions. We find that humanness cues such as a human name and typing indicator significantly increase perceived anthropomorphism in vignette conditions but have no detectable effect when participants actually interact with the chatbot. These results provide initial evidence that vignette-based chatbot studies may not generalize to real interactions, with implications for how the field studies perception of conversational agents.

Paper Number

1386

Comments

NEXTTRANS

Download

Share

COinS
 
Aug 15th, 12:00 AM

Vignettes or Bots: Can We Study Chatbot Interactions without Actual Interactions?

Vignettes are widely used in chatbot research to manipulate interface and design cues, but the unique characteristics of dyadic chat interactions raise questions about whether observing a vignette produces the same perceptions as participating in a live conversation. In a 2 × 2 × 2 between-subjects experiment, we compare perceptions of anthropomorphism across vignette and interactive chatbot conditions. We find that humanness cues such as a human name and typing indicator significantly increase perceived anthropomorphism in vignette conditions but have no detectable effect when participants actually interact with the chatbot. These results provide initial evidence that vignette-based chatbot studies may not generalize to real interactions, with implications for how the field studies perception of conversational agents.

When commenting on articles, please be friendly, welcoming, respectful and abide by the AIS eLibrary Discussion Thread Code of Conduct posted here.