Loading...
Paper Number
ICIS2025-2712
Paper Type
Complete
Abstract
In 2025, major developers of large language models (LLMs) announced that their systems can automatically generate software code from natural language descriptions. Yet these advances largely overlook the implications for privacy and cybersecurity. Drawing on machine learning theory, we propose a framework showing how privacy, utility, confidentiality, integrity, and availability in LLM-generated software are shaped by inherent tensions, tradeoffs, and paradoxes. We empirically test these propositions using an open-source LLM, a curated library of 20,000 code samples, and documented cybersecurity attacks. Results reveal that differential privacy (DP) improves privacy but reduces utility, underscoring a central privacy–utility tradeoff. Moreover, while DP strengthens defenses against attacks on confidentiality, integrity, and availability, it simultaneously weakens detection—creating a protection–detection paradox. We discuss theoretical and practical implications, urging information systems (IS) research to address these challenges and prepare future IS careers for evolving demands in privacy and cybersecurity in AI-driven code generation.
Recommended Citation
Wei, Wenqi; Li, Xiang; and Tanriverdi, Hüseyin, "Software Generation With LLMs: Privacy, Utility, and Cybersecurity Tensions" (2025). ICIS 2025 Proceedings. 38.
https://aisel.aisnet.org/icis2025/gen_ai/gen_ai/38
Software Generation With LLMs: Privacy, Utility, and Cybersecurity Tensions
In 2025, major developers of large language models (LLMs) announced that their systems can automatically generate software code from natural language descriptions. Yet these advances largely overlook the implications for privacy and cybersecurity. Drawing on machine learning theory, we propose a framework showing how privacy, utility, confidentiality, integrity, and availability in LLM-generated software are shaped by inherent tensions, tradeoffs, and paradoxes. We empirically test these propositions using an open-source LLM, a curated library of 20,000 code samples, and documented cybersecurity attacks. Results reveal that differential privacy (DP) improves privacy but reduces utility, underscoring a central privacy–utility tradeoff. Moreover, while DP strengthens defenses against attacks on confidentiality, integrity, and availability, it simultaneously weakens detection—creating a protection–detection paradox. We discuss theoretical and practical implications, urging information systems (IS) research to address these challenges and prepare future IS careers for evolving demands in privacy and cybersecurity in AI-driven code generation.
When commenting on articles, please be friendly, welcoming, respectful and abide by the AIS eLibrary Discussion Thread Code of Conduct posted here.
Comments
12-GenAI