Paper Number
2074
Paper Type
Complete
Description
Machine learning (ML) holds immense potential for enterprise data use cases, but a lack of skilled data scientists hinders its utilization. Automated ML (AutoML) aims to empower business users but often falls short, especially when domain knowledge influences model selection. It remains unclear how human- guided ML (HGML) systems can effectively empower business users. To address this, we establish a design science research project. Drawing on the theory of effective use and interviews with seven business users, we present three design principles that we instantiated in our prototype, MLFeasi. Our formative evaluation with business users and data scientists revealed the impracticality of completely replacing data scientists. Instead, a collaborative approach involving data scientists is advocated when ML use cases are deemed feasible - a process we refer to as HGML feasibility analysis. The summative evaluation, including a small-scale experiment and real-world use cases, demonstrates MLFeasi’s effectiveness in improving HGML feasibility analysis
Recommended Citation
Gunklach, Jonas; Nadj, Mario; Knaeble, Merlin; Bragaglia, Isabela; and Mädche, Alexander, "Designing for Effective Human-Guided Machine Learning Feasibility Analysis" (2024). ICIS 2024 Proceedings. 1.
https://aisel.aisnet.org/icis2024/humtechinter/humtechinter/1
Designing for Effective Human-Guided Machine Learning Feasibility Analysis
Machine learning (ML) holds immense potential for enterprise data use cases, but a lack of skilled data scientists hinders its utilization. Automated ML (AutoML) aims to empower business users but often falls short, especially when domain knowledge influences model selection. It remains unclear how human- guided ML (HGML) systems can effectively empower business users. To address this, we establish a design science research project. Drawing on the theory of effective use and interviews with seven business users, we present three design principles that we instantiated in our prototype, MLFeasi. Our formative evaluation with business users and data scientists revealed the impracticality of completely replacing data scientists. Instead, a collaborative approach involving data scientists is advocated when ML use cases are deemed feasible - a process we refer to as HGML feasibility analysis. The summative evaluation, including a small-scale experiment and real-world use cases, demonstrates MLFeasi’s effectiveness in improving HGML feasibility analysis
Comments
09-HTI