Abstract

This case study provides insights into how the implementation of machine learning (ML) based solution resulted in partial automation of medical transcription task. The analysis reveals that the task performance improvement enabled by ML technology was crucially dependent on the solution’s ability to learn and encapsulate tacit knowledge of human transcribers. These capabilities of the technology, however, required active input from transcribers throughout the development and operation lifecycle of the ML-based solution. This mutual interdependence of human actors and technology signifies the sociotechnical nature of ML-based task automation. Apart from reporting the performance levels before and after ML implementation, as well as providing in-depth analysis of task-level changes leading to performance improvement, this study contributes to the emerging understanding of how the design and implementation of ML-based solutions impact organizational tasks. The empirical material utilized in this work comprises a single case study. The case company, Inscripta, describes its service as “a modern and cost-efficient solution designed for dictating, optimizing and storing patient notes onto a company’s health record system. […] The solution comprises of a dictation application, AI-powered speech recognition, and personnel and workflow management system.” While the data collection for this study is still ongoing, we are currently utilizing four sources of data: (1) interviews with case company employees and founders; (2) interviews with transcribers or professionals working in the context of transcription services and who are not associated with the case company; (3) interviews with practicing medical doctors/physicians; and (4) media and other publicly available materials relating to the focal service offering. The impact of information systems (IS) on organizational task performance has been and continues to be among the core questions investigated by IS scholars. Early work in this area revealed that it is not enough to assume that technology simply substitutes for a task or part of a task that has previously been carried out by a human actor. This observation has been fundamental to developing a more encompassing view of the link between technology implementation and task performance. Goodhue and Thompson (1995) proposed a model depicting the technology-to-performance chain where a fit between task, technology, and individual actor is a key driver of task performance directly, as well as indirectly via the impact on technology utilization. Recent research in this area, however, highlights the importance of detailed analysis of changes taking place over time and recognizing feedback loops within the implementation process. Against this backdrop, we reexamine our understanding of IS impact on task performance in the context of ML implementations. ML-based solutions exhibit previously unseen levels of autonomy and inscrutability, as well as are able to learn from experience. These novel characteristics of IS artifacts based on ML technologies call for a detailed analysis of changes taking place within organizational sociotechnical systems. Based on the preliminary analysis, our case study is set to contribute to the ongoing IS conversation on ML implementation and use in the organizational setting. Despite substantial interest of practitioners and scholars in this topic, there is a scarcity of studies uncovering the details of successful ML implementations. Our case study provides rich insight into the process of changes leading to major improvement in task performance level.

Download

Share

COinS