Paper Type
Complete
Abstract
Machine learning (ML) models are widely used across various critical domains for their ability to process large-scale data and deliver high predictive accuracy. While traditional benchmarking of ML models focuses on performance metrics like accuracy and precision, these metrics often fall short in accounting for the model’s transparency and robustness when the models get complex with varying conditions. This research proposes a comprehensive framework for benchmarking supervised machine learning models that incorporate the model’s performance metrics, explainability techniques, and robustness assessments. The proposed framework combines traditional accuracy- and precision-based performance evaluation with explainability and robustness to ensure the model's efficiency, transparency, and stability in the presence of noise and data shifts. By holistically addressing performance, explainability, and robustness in tandem, the framework supports data-driven decision-making in selecting the model appropriate to organizational contexts, addressing stakeholder concerns related to model interpretability, trustworthiness, and resilience—particularly in critical domains such as healthcare.
Paper Number
1373
Recommended Citation
Malla, Deependra and El-Gayar, Omar, "Framework for Benchmarking Machine Learning Models: Integrating Performance Metrics, Explainability Techniques, and Robustness Assessments" (2025). AMCIS 2025 Proceedings. 7.
https://aisel.aisnet.org/amcis2025/data_science/sig_dsa/7
Framework for Benchmarking Machine Learning Models: Integrating Performance Metrics, Explainability Techniques, and Robustness Assessments
Machine learning (ML) models are widely used across various critical domains for their ability to process large-scale data and deliver high predictive accuracy. While traditional benchmarking of ML models focuses on performance metrics like accuracy and precision, these metrics often fall short in accounting for the model’s transparency and robustness when the models get complex with varying conditions. This research proposes a comprehensive framework for benchmarking supervised machine learning models that incorporate the model’s performance metrics, explainability techniques, and robustness assessments. The proposed framework combines traditional accuracy- and precision-based performance evaluation with explainability and robustness to ensure the model's efficiency, transparency, and stability in the presence of noise and data shifts. By holistically addressing performance, explainability, and robustness in tandem, the framework supports data-driven decision-making in selecting the model appropriate to organizational contexts, addressing stakeholder concerns related to model interpretability, trustworthiness, and resilience—particularly in critical domains such as healthcare.
When commenting on articles, please be friendly, welcoming, respectful and abide by the AIS eLibrary Discussion Thread Code of Conduct posted here.
Comments
SIGDSA