Paper Number
ECIS2026-1845
Paper Type
CRP
Abstract
While several approaches leverage Large Language Models (LLMs) to assist process analysts by delegating process discovery tasks to domain experts, they often overlook the requirements of both roles needed to produce usable process knowledge. This study empirically evaluates the capabilities of LLMs in supporting process knowledge acquisition. Across four real-world case studies conducted in three Belgian organizations, 39 augmented elicitation sessions were conducted involving 25 stakeholders. Through a triangulated analysis of computational logs, domain experts' feedback, and process analysts' evaluations of the results of process discovery sessions, this paper positions the performance of an LLM-based approach against industry requirements. Findings formalize 15 challenges of implementing an LLM-based knowledge acquisition approach, such as clarifying governance mechanisms during socialization or ensuring uniform granularity during abstraction, and identify the interdependencies between socialization and externalization challenges. This work offers empirical insights to tailor the design of future LLM-based proposals for delegating process discovery tasks. Keywords: Knowledge acquisition, Domain experts, Large language models, Business process discovery, Case studies.
Recommended Citation
Schinckus, Malik; Klievtsova, Nataliia; Simonofski, Anthony; and Rinderle-Ma, Stefanie, "Delegating Process Knowledge Acquisition To Domain Experts Through Large Language Models: A Multi-Case Study" (2026). ECIS 2026 Proceedings. 7.
https://aisel.aisnet.org/ecis2026/bpm/bpm/7
Delegating Process Knowledge Acquisition To Domain Experts Through Large Language Models: A Multi-Case Study
While several approaches leverage Large Language Models (LLMs) to assist process analysts by delegating process discovery tasks to domain experts, they often overlook the requirements of both roles needed to produce usable process knowledge. This study empirically evaluates the capabilities of LLMs in supporting process knowledge acquisition. Across four real-world case studies conducted in three Belgian organizations, 39 augmented elicitation sessions were conducted involving 25 stakeholders. Through a triangulated analysis of computational logs, domain experts' feedback, and process analysts' evaluations of the results of process discovery sessions, this paper positions the performance of an LLM-based approach against industry requirements. Findings formalize 15 challenges of implementing an LLM-based knowledge acquisition approach, such as clarifying governance mechanisms during socialization or ensuring uniform granularity during abstraction, and identify the interdependencies between socialization and externalization challenges. This work offers empirical insights to tailor the design of future LLM-based proposals for delegating process discovery tasks. Keywords: Knowledge acquisition, Domain experts, Large language models, Business process discovery, Case studies.
When commenting on articles, please be friendly, welcoming, respectful and abide by the AIS eLibrary Discussion Thread Code of Conduct posted here.