Paper Number

2087

Paper Type

Complete Research Paper

Abstract

Ever-increasing volumes of publication data eventually threaten to exceed the limits of human cognition and manual processing. In light of these challenges, new techniques have been called for that extend the capabilities of humans through machine memory and computational power. This paper reflects on a novel "human-in-the-loop" topic modelling approach to systematic literature reviews by combining Latent Dirichlet Analysis (LDA) and human coding to identify key constructs, relationships, and outcomes. Our approach begins by modelling hidden semantic structures through topic modelling to gain insights into embedded patterns and topic compositions in research. Thereafter, we diverge from traditional LDA methods, as we use human coding to extract contextual insights into the socio-technical components of DevOps. Finally, we re-run topic modelling against these socio-technical categories, affording us the opportunity to theorise further. Learnings emerging at each of the three phases are discussed when moving between topic modelling and human coding techniques.

Share

COinS
 
Jun 14th, 12:00 AM

REFLECTIONS ON A ‘HUMAN-IN-THE-LOOP’ TOPIC MODELLING (HLTM) APPROACH TO SYSTEMATIC LITERATURE REVIEWS

Ever-increasing volumes of publication data eventually threaten to exceed the limits of human cognition and manual processing. In light of these challenges, new techniques have been called for that extend the capabilities of humans through machine memory and computational power. This paper reflects on a novel "human-in-the-loop" topic modelling approach to systematic literature reviews by combining Latent Dirichlet Analysis (LDA) and human coding to identify key constructs, relationships, and outcomes. Our approach begins by modelling hidden semantic structures through topic modelling to gain insights into embedded patterns and topic compositions in research. Thereafter, we diverge from traditional LDA methods, as we use human coding to extract contextual insights into the socio-technical components of DevOps. Finally, we re-run topic modelling against these socio-technical categories, affording us the opportunity to theorise further. Learnings emerging at each of the three phases are discussed when moving between topic modelling and human coding techniques.

When commenting on articles, please be friendly, welcoming, respectful and abide by the AIS eLibrary Discussion Thread Code of Conduct posted here.