Paper Number
2087
Paper Type
Complete Research Paper
Abstract
Ever-increasing volumes of publication data eventually threaten to exceed the limits of human cognition and manual processing. In light of these challenges, new techniques have been called for that extend the capabilities of humans through machine memory and computational power. This paper reflects on a novel "human-in-the-loop" topic modelling approach to systematic literature reviews by combining Latent Dirichlet Analysis (LDA) and human coding to identify key constructs, relationships, and outcomes. Our approach begins by modelling hidden semantic structures through topic modelling to gain insights into embedded patterns and topic compositions in research. Thereafter, we diverge from traditional LDA methods, as we use human coding to extract contextual insights into the socio-technical components of DevOps. Finally, we re-run topic modelling against these socio-technical categories, affording us the opportunity to theorise further. Learnings emerging at each of the three phases are discussed when moving between topic modelling and human coding techniques.
Recommended Citation
Thummadi, Veeresh; McCarthy, Stephen; Sammon, David; and Wibisono, Arif, "Reflections on a ‘Human-in-the-Loop’ Topic Modelling (HLTM) Approach to Systematic Literature Reviews" (2024). ECIS 2024 Proceedings. 5.
https://aisel.aisnet.org/ecis2024/track22_innrm/track22_innrm/5
Reflections on a ‘Human-in-the-Loop’ Topic Modelling (HLTM) Approach to Systematic Literature Reviews
Ever-increasing volumes of publication data eventually threaten to exceed the limits of human cognition and manual processing. In light of these challenges, new techniques have been called for that extend the capabilities of humans through machine memory and computational power. This paper reflects on a novel "human-in-the-loop" topic modelling approach to systematic literature reviews by combining Latent Dirichlet Analysis (LDA) and human coding to identify key constructs, relationships, and outcomes. Our approach begins by modelling hidden semantic structures through topic modelling to gain insights into embedded patterns and topic compositions in research. Thereafter, we diverge from traditional LDA methods, as we use human coding to extract contextual insights into the socio-technical components of DevOps. Finally, we re-run topic modelling against these socio-technical categories, affording us the opportunity to theorise further. Learnings emerging at each of the three phases are discussed when moving between topic modelling and human coding techniques.
When commenting on articles, please be friendly, welcoming, respectful and abide by the AIS eLibrary Discussion Thread Code of Conduct posted here.