Loading...

Media is loading
 

Paper Type

ERF

Abstract

This study explores the utilization of Large Language Models (LLMs) in managing Design Science Research (DSR) knowledge. It addresses the absence of a consolidated knowledge base for DSR artifacts, proposing an AI-driven approach for extracting and structuring information from scientific articles. Three experiments were conducted to evaluate the effectiveness of LLMs — specifically GPT 3.5 and GPT 4 — when the requested information is 1) spread across multiple articles, 2) contained within a single article, and 3) well-structured inside a single file. Our results indicate the potential of AI tools in efficient information extraction and real-time data processing. However, they also underscore the limitations of current AI models, such as underperforming when extracting DSR-related information spread across many articles. Overall, our research contributes to the understanding of AI capabilities in enhancing DSR knowledge management while identifying promising areas for further exploration.

Paper Number

1330

Comments

SIGAIAA

Share

COinS
 
Aug 16th, 12:00 AM

Towards Managing Design Science Knowledge with Large Language Models

This study explores the utilization of Large Language Models (LLMs) in managing Design Science Research (DSR) knowledge. It addresses the absence of a consolidated knowledge base for DSR artifacts, proposing an AI-driven approach for extracting and structuring information from scientific articles. Three experiments were conducted to evaluate the effectiveness of LLMs — specifically GPT 3.5 and GPT 4 — when the requested information is 1) spread across multiple articles, 2) contained within a single article, and 3) well-structured inside a single file. Our results indicate the potential of AI tools in efficient information extraction and real-time data processing. However, they also underscore the limitations of current AI models, such as underperforming when extracting DSR-related information spread across many articles. Overall, our research contributes to the understanding of AI capabilities in enhancing DSR knowledge management while identifying promising areas for further exploration.

When commenting on articles, please be friendly, welcoming, respectful and abide by the AIS eLibrary Discussion Thread Code of Conduct posted here.