Abstract
This study aimed to compare the effectiveness of the deep ANN embedding technique with traditional article selection methods in systematic literature reviews. The embedding model utilizes a natural language problem description to find semantically similar publications. Consequently, this technique is accessible to users without experience in data exploration. Traditional methods are represented by precise keyword queries in Scopus and Excel-based selection. Keywords used in these methods are extracted from the description by the GPT-4o model with a temperature set to zero, ensuring repeatability. The obtained results were evaluated using bibliometric metrics, which facilitate the assessment of similarities among filtered publications and enhance understanding of their connections. The findings demonstrated the superiority of the embedding model, achieving higher thematic coherence and more shared references and keywords. This approach improves the identification of relevant publications and significantly contributes to automating systematic literature reviews, which is desired in many scientific disciplines.
Paper Type
Full Paper
DOI
10.62036/ISD.2025.145
A Comparative Analysis of Embedding Models and Traditional Methods for Publication Selection in Systematic Literature Reviews - A Case Study in Gamification Marketing
This study aimed to compare the effectiveness of the deep ANN embedding technique with traditional article selection methods in systematic literature reviews. The embedding model utilizes a natural language problem description to find semantically similar publications. Consequently, this technique is accessible to users without experience in data exploration. Traditional methods are represented by precise keyword queries in Scopus and Excel-based selection. Keywords used in these methods are extracted from the description by the GPT-4o model with a temperature set to zero, ensuring repeatability. The obtained results were evaluated using bibliometric metrics, which facilitate the assessment of similarities among filtered publications and enhance understanding of their connections. The findings demonstrated the superiority of the embedding model, achieving higher thematic coherence and more shared references and keywords. This approach improves the identification of relevant publications and significantly contributes to automating systematic literature reviews, which is desired in many scientific disciplines.
Recommended Citation
Frankowski, P.K., Wiśniewska, J. & Matysik, S. (2025). A Comparative Analysis of Embedding Models and Traditional Methods for Publication Selection in Systematic Literature Reviews - A Case Study in Gamification MarketingIn I. Luković, S. Bjeladinović, B. Delibašić, D. Barać, N. Iivari, E. Insfran, M. Lang, H. Linger, & C. Schneider (Eds.), Empowering the Interdisciplinary Role of ISD in Addressing Contemporary Issues in Digital Transformation: How Data Science and Generative AI Contributes to ISD (ISD2025 Proceedings). Belgrade, Serbia: University of Gdańsk, Department of Business Informatics & University of Belgrade, Faculty of Organizational Sciences. ISBN: 978-83-972632-1-5. https://doi.org/10.62036/ISD.2025.145