Location
Online
Event Website
https://hicss.hawaii.edu/
Start Date
3-1-2023 12:00 AM
End Date
7-1-2023 12:00 AM
Description
Teaching research methods is important in any curriculum that prepares students for an academic career. While theoretical frameworks for qualitative theory building can be adequately conveyed through lecturing, the practices of qualitative data analysis (QDA) cannot. However, using experiential learning techniques for teaching QDA methods to large numbers of students presents a challenge to the instructor due to the effort required for the grading of homework. Any homework involving the coding of qualitative data will result in a myriad of different interpretations of the same data with varying quality. Grading such assignments requires significant effort. We approached this problem by using methods of inter-rater agreement and a model solution as a proxy for the quality of the submission. The automated agreement data serves as the foundation for a semi-automated grading process. Within this paper, we demonstrate that this proxy has a high correlation with the manual grading of submissions.
Recommended Citation
Kaufmann, Andreas; Riehle, Dirk; Krause, Julia; and Harutyunyan, Nikolay, "A Solution for Automated Grading of QDA Homework" (2023). Hawaii International Conference on System Sciences 2023 (HICSS-56). 6.
https://aisel.aisnet.org/hicss-56/cl/teaching_and_learning_technologies/6
A Solution for Automated Grading of QDA Homework
Online
Teaching research methods is important in any curriculum that prepares students for an academic career. While theoretical frameworks for qualitative theory building can be adequately conveyed through lecturing, the practices of qualitative data analysis (QDA) cannot. However, using experiential learning techniques for teaching QDA methods to large numbers of students presents a challenge to the instructor due to the effort required for the grading of homework. Any homework involving the coding of qualitative data will result in a myriad of different interpretations of the same data with varying quality. Grading such assignments requires significant effort. We approached this problem by using methods of inter-rater agreement and a model solution as a proxy for the quality of the submission. The automated agreement data serves as the foundation for a semi-automated grading process. Within this paper, we demonstrate that this proxy has a high correlation with the manual grading of submissions.
https://aisel.aisnet.org/hicss-56/cl/teaching_and_learning_technologies/6