Paper Type
Complete
Paper Number
PACIS2026-1235
Description
AI coding assistants such as GitHub Copilot are rapidly reshaping software development, but their effects on code quality remain uncertain. We analyze internal development logs from a large Chinese automobile manufacturer that adopted a Copilot-like assistant, using a difference-in-differences design to examine defect density and defect severity. Overall, AI adoption is not associated with a statistically significant reduction in defect density, but is associated with higher defect severity when defects occur. Function-level analyses further show that Q&A/chat use is associated with higher defect density and higher severity, whereas code-completion use is associated with lower defect density but higher severity. These findings advance research on human–AI collaboration and offer insights for managing AI-related quality risks in knowledge work.
Recommended Citation
Zhang, Wei; Chen, Yuyuan; Zhang, Yueyue; Ding, Yong; Tong, Yu; and Zhang, Cheng, "The Promise and Peril of AI-Assisted Programming: Effects on Software Defects" (2026). PACIS 2026 Proceedings. 2.
https://aisel.aisnet.org/pacis2026/general_topic/general_topic/2
The Promise and Peril of AI-Assisted Programming: Effects on Software Defects
AI coding assistants such as GitHub Copilot are rapidly reshaping software development, but their effects on code quality remain uncertain. We analyze internal development logs from a large Chinese automobile manufacturer that adopted a Copilot-like assistant, using a difference-in-differences design to examine defect density and defect severity. Overall, AI adoption is not associated with a statistically significant reduction in defect density, but is associated with higher defect severity when defects occur. Function-level analyses further show that Q&A/chat use is associated with higher defect density and higher severity, whereas code-completion use is associated with lower defect density but higher severity. These findings advance research on human–AI collaboration and offer insights for managing AI-related quality risks in knowledge work.
Comments
17-General