Paper Number
ICIS2025-1937
Paper Type
Short
Abstract
The rapid adoption of hybrid service systems integrating artificial intelligence (AI) and human agents has fundamentally reshaped service quality metrics and operational efficiency in e-commerce customer service. This transformation underscores the need to optimize human-AI collaboration modalities. However, the mechanisms for optimizing collaboration modalities to enhance service quality remain understudied. This study explores how two types of human-AI collaboration modalities (reactive oversight vs. informed reliance) affect service quality through the lens of regulatory depletion theory, with attention to task complexity and sales phases. A random field experiment is conducted on a large e-commerce platform, and large language models (LLMs) are used for objective service quality evaluation. Results show that informed reliance modality can significantly enhance service quality, especially in high-complexity contexts and pre-sale phases. These findings advance human-AI interaction theory by disentangling mechanisms of cognitive resource allocation and offer actionable strategies for optimizing collaboration modalities based on task characteristics.
Recommended Citation
Lin, Jing; Lan, Zhenming; and Fu, Xin, "Reactive Oversight vs. Informed Reliance: The Impact of Human-AI Collaboration Modality on E-commerce Customer Service Quality" (2025). ICIS 2025 Proceedings. 16.
https://aisel.aisnet.org/icis2025/is_transformwork/is_transformwork/16
Reactive Oversight vs. Informed Reliance: The Impact of Human-AI Collaboration Modality on E-commerce Customer Service Quality
The rapid adoption of hybrid service systems integrating artificial intelligence (AI) and human agents has fundamentally reshaped service quality metrics and operational efficiency in e-commerce customer service. This transformation underscores the need to optimize human-AI collaboration modalities. However, the mechanisms for optimizing collaboration modalities to enhance service quality remain understudied. This study explores how two types of human-AI collaboration modalities (reactive oversight vs. informed reliance) affect service quality through the lens of regulatory depletion theory, with attention to task complexity and sales phases. A random field experiment is conducted on a large e-commerce platform, and large language models (LLMs) are used for objective service quality evaluation. Results show that informed reliance modality can significantly enhance service quality, especially in high-complexity contexts and pre-sale phases. These findings advance human-AI interaction theory by disentangling mechanisms of cognitive resource allocation and offer actionable strategies for optimizing collaboration modalities based on task characteristics.
When commenting on articles, please be friendly, welcoming, respectful and abide by the AIS eLibrary Discussion Thread Code of Conduct posted here.
Comments
03-Transformation