Abstract
The rapid integration of Large Language Models (LLMs) is transforming how developers search for information and solve technical problems. While these tools offer powerful support, they may also encourage cognitive offloading, potentially affecting independent problem-solving. This study examines whether LLM adoption relates to changes in troubleshooting traces in Stack Overflow questions. Drawing on cognitive offloading, transactive memory theory, and SRL, we analyze help-seeking behavior before (2019–2021) and after (2023–2025) widespread LLM adoption. Using machine learning–assisted annotation, we develop a Learning Trace Index (LTI) to capture observable problem-solving processes. Results show a sharp decline in platform activity after 2023, but only modest differences in troubleshooting traces across periods and experience levels. These findings offer empirical insight into cognitive offloading and introduce a scalable measure for studying learning behaviors in developer communities.
Paper Type
Short Paper
DOI
10.62036/ISD.2026.126
Do Large Language Models Change How Developers Seek Help? A Learning Trace-Based Study of Stack Overflow Behavior
The rapid integration of Large Language Models (LLMs) is transforming how developers search for information and solve technical problems. While these tools offer powerful support, they may also encourage cognitive offloading, potentially affecting independent problem-solving. This study examines whether LLM adoption relates to changes in troubleshooting traces in Stack Overflow questions. Drawing on cognitive offloading, transactive memory theory, and SRL, we analyze help-seeking behavior before (2019–2021) and after (2023–2025) widespread LLM adoption. Using machine learning–assisted annotation, we develop a Learning Trace Index (LTI) to capture observable problem-solving processes. Results show a sharp decline in platform activity after 2023, but only modest differences in troubleshooting traces across periods and experience levels. These findings offer empirical insight into cognitive offloading and introduce a scalable measure for studying learning behaviors in developer communities.
Recommended Citation
Yeto, D. & Baj-Rogowska, A.(2026). Do Large Language Models Change How Developers Seek Help? A Learning Trace-Based Study of Stack Overflow Behavior. In M. Valenta, B. Mannová, R. Pergl, A. Przybylek, M. Lang, H. Linger, C. Schneider, N. Iivari, & E. Insfran (Eds.), Making ISD Sustainable: Reloaded with AI and Automation (ISD2026 Proceedings). Prague, Czech Republic: Czech Technical University in Prague. ISBN: 978-80-01-07585-2. https://doi.org/10.62036/ISD.2026.126