Abstract
Most organizations store its data in monolithic data architectures (e.g.: Data Lakes), which over time have demonstrated their limitations when it comes to scalability and cost. It is intended that data engineering follows in the path of software engineering and abandon monolithic architectures, thus creating decentralized data architectures. Data Mesh consists in the implementation of an architecture where data is intentionally distributed among several Mesh nodes, in such a way that there is no chaos or data silos, since there are centralized data governance strategies and the guarantee that the core principles are shared throughout the Mesh nodes. This paper aims to propose a domain model and a conceptual architecture towards the achievement of decentralized data architectures – the Data Mesh.
Paper Type
Short Paper
Data-Driven Information Systems: The Data Mesh Paradigm Shift
Most organizations store its data in monolithic data architectures (e.g.: Data Lakes), which over time have demonstrated their limitations when it comes to scalability and cost. It is intended that data engineering follows in the path of software engineering and abandon monolithic architectures, thus creating decentralized data architectures. Data Mesh consists in the implementation of an architecture where data is intentionally distributed among several Mesh nodes, in such a way that there is no chaos or data silos, since there are centralized data governance strategies and the guarantee that the core principles are shared throughout the Mesh nodes. This paper aims to propose a domain model and a conceptual architecture towards the achievement of decentralized data architectures – the Data Mesh.
Recommended Citation
Machado, I., Costa, C., & Santos, M. Y. (2021). Data-Driven Information Systems: The Data Mesh Paradigm Shift. In E. Insfran, F. González, S. Abrahão, M. Fernández, C. Barry, H. Linger, M. Lang, & C. Schneider (Eds.), Information Systems Development: Crossing Boundaries between Development and Operations (DevOps) in Information Systems (ISD2021 Proceedings). Valencia, Spain: Universitat Politècnica de València.