Loading...
Paper Type
ERF
Abstract
This study explores the utilization of Large Language Models (LLMs) in managing Design Science Research (DSR) knowledge. It addresses the absence of a consolidated knowledge base for DSR artifacts, proposing an AI-driven approach for extracting and structuring information from scientific articles. Three experiments were conducted to evaluate the effectiveness of LLMs — specifically GPT 3.5 and GPT 4 — when the requested information is 1) spread across multiple articles, 2) contained within a single article, and 3) well-structured inside a single file. Our results indicate the potential of AI tools in efficient information extraction and real-time data processing. However, they also underscore the limitations of current AI models, such as underperforming when extracting DSR-related information spread across many articles. Overall, our research contributes to the understanding of AI capabilities in enhancing DSR knowledge management while identifying promising areas for further exploration.
Paper Number
1330
Recommended Citation
Folha, Rodrigo and Carvalho, Arthur, "Towards Managing Design Science Knowledge with Large Language Models" (2024). AMCIS 2024 Proceedings. 9.
https://aisel.aisnet.org/amcis2024/ai_aa/ai_aa/9
Towards Managing Design Science Knowledge with Large Language Models
This study explores the utilization of Large Language Models (LLMs) in managing Design Science Research (DSR) knowledge. It addresses the absence of a consolidated knowledge base for DSR artifacts, proposing an AI-driven approach for extracting and structuring information from scientific articles. Three experiments were conducted to evaluate the effectiveness of LLMs — specifically GPT 3.5 and GPT 4 — when the requested information is 1) spread across multiple articles, 2) contained within a single article, and 3) well-structured inside a single file. Our results indicate the potential of AI tools in efficient information extraction and real-time data processing. However, they also underscore the limitations of current AI models, such as underperforming when extracting DSR-related information spread across many articles. Overall, our research contributes to the understanding of AI capabilities in enhancing DSR knowledge management while identifying promising areas for further exploration.
When commenting on articles, please be friendly, welcoming, respectful and abide by the AIS eLibrary Discussion Thread Code of Conduct posted here.
Comments
SIGAIAA