Paper Number
ECIS2025-1149
Paper Type
CRP
Abstract
The emergence of advanced large language models enables artificial intelligence (AI) based agents to assist developers in writing computer code to an unprecedented extent. Due to this development, AI-based agents have become integral to developers’ daily workflows. While ongoing enhancements in the code quality provided by AI-based agents aim to improve developers’ productivity, it is far from clear how that development affects developers’ knowledge of how to write functional computer code (i.e., developers’ procedural knowledge). To answer this question, we drew on cognitive load theory and conducted an online experiment with 85 developers. Our results reveal that developers receiving assistance with higher (vs. lower) code quality from AI-based agents exhibit relatively lower procedural knowledge. Furthermore, our results demonstrate that developers’ cognitive load is a crucial explanatory variable for this effect. Overall, our study illustrates a different perspective on the potential consequences of the ever-increasing code quality provided by AI-based agents.
Recommended Citation
Diebel, Christopher; Goutier, Marc; Adam, Martin; and Benlian, Alexander, "The Price of AI Assistance: The Undermining Effect of AI-Generated Code on Developers’ Procedural Knowledge" (2025). ECIS 2025 Proceedings. 2.
https://aisel.aisnet.org/ecis2025/cog_hbis/cog_hbis/2
The Price of AI Assistance: The Undermining Effect of AI-Generated Code on Developers’ Procedural Knowledge
The emergence of advanced large language models enables artificial intelligence (AI) based agents to assist developers in writing computer code to an unprecedented extent. Due to this development, AI-based agents have become integral to developers’ daily workflows. While ongoing enhancements in the code quality provided by AI-based agents aim to improve developers’ productivity, it is far from clear how that development affects developers’ knowledge of how to write functional computer code (i.e., developers’ procedural knowledge). To answer this question, we drew on cognitive load theory and conducted an online experiment with 85 developers. Our results reveal that developers receiving assistance with higher (vs. lower) code quality from AI-based agents exhibit relatively lower procedural knowledge. Furthermore, our results demonstrate that developers’ cognitive load is a crucial explanatory variable for this effect. Overall, our study illustrates a different perspective on the potential consequences of the ever-increasing code quality provided by AI-based agents.
When commenting on articles, please be friendly, welcoming, respectful and abide by the AIS eLibrary Discussion Thread Code of Conduct posted here.