Abstract
Conceptual models are essential for designing, analyzing, and communicating complex systems. However, traditional modeling languages such as UML, BPMN, and ArchiMate primarily rely on textual and symbolic representations, which can limit their expressiveness and accessibility, especially for non-expert stakeholders. To address this challenge, we introduce a framework for Multimodal-Enriched Conceptual Modeling (MMeCM) that integrates videos, images, and audio directly into model elements. Our approach enables modelers to attach contextual multimedia references to processes, entities, and relationships, effectively grounding abstract concepts in tangible real-world artifacts. We make three key contributions: (1) a quantitative analysis of concept enrichability using the OntoUML/UFO Catalog, identifying which elements benefit from multimodal representation; (2) the design and implementation of a generalizable framework for embedding multimodal data across different modeling languages; and (3) a qualitative user study, grounded in the Technology Acceptance Model, evaluating the perceived usefulness and usability of multimodal-enriched models, together with a dataset of more than 12K multimodal-enriched natural language elements found in conceptual models. Our evaluation shows that a majority of natural language elements in conceptual models can be effectively augmented with multimedia, and user feedback indicates a strong positive reception of MMeCM.
Paper Type
Full Paper
DOI
10.62036/ISD.2025.15
Towards the Enrichment of Conceptual Models with Multimodal Data
Conceptual models are essential for designing, analyzing, and communicating complex systems. However, traditional modeling languages such as UML, BPMN, and ArchiMate primarily rely on textual and symbolic representations, which can limit their expressiveness and accessibility, especially for non-expert stakeholders. To address this challenge, we introduce a framework for Multimodal-Enriched Conceptual Modeling (MMeCM) that integrates videos, images, and audio directly into model elements. Our approach enables modelers to attach contextual multimedia references to processes, entities, and relationships, effectively grounding abstract concepts in tangible real-world artifacts. We make three key contributions: (1) a quantitative analysis of concept enrichability using the OntoUML/UFO Catalog, identifying which elements benefit from multimodal representation; (2) the design and implementation of a generalizable framework for embedding multimodal data across different modeling languages; and (3) a qualitative user study, grounded in the Technology Acceptance Model, evaluating the perceived usefulness and usability of multimodal-enriched models, together with a dataset of more than 12K multimodal-enriched natural language elements found in conceptual models. Our evaluation shows that a majority of natural language elements in conceptual models can be effectively augmented with multimedia, and user feedback indicates a strong positive reception of MMeCM.
Recommended Citation
Gavric, A., Bork, D. & Proper, H.A. (2025). Towards the Enrichment of Conceptual Models with Multimodal DataIn I. Luković, S. Bjeladinović, B. Delibašić, D. Barać, N. Iivari, E. Insfran, M. Lang, H. Linger, & C. Schneider (Eds.), Empowering the Interdisciplinary Role of ISD in Addressing Contemporary Issues in Digital Transformation: How Data Science and Generative AI Contributes to ISD (ISD2025 Proceedings). Belgrade, Serbia: University of Gdańsk, Department of Business Informatics & University of Belgrade, Faculty of Organizational Sciences. ISBN: 978-83-972632-1-5. https://doi.org/10.62036/ISD.2025.15