SIG ED - IS in Education, IS Curriculum, Education and Teaching Cases
Trust, but Verify! - An Empirical Investigation of Students’ Initial Trust in AI-Based Essay Scoring
Paper Type
Complete
Paper Number
1228
Description
AI is becoming increasingly important in supporting education. Nowadays, AI-based systems can score essays in high-stakes exams not only by comparing words but also by evaluating content. However, for AI-based essay scoring systems to be used, they must be trusted. Based on a scenario-based experiment with 260 students at a German university, we were able to show that their initial trust in AI-based essay scoring systems is significantly lower than in human examiners. Human control of AI-scoring can partially reduce the negative effect. The perceived system characteristics and the personality traits of the students are important factors which positively influence trustworthiness and trust, respectively. Furthermore, we could show that the more complex the essay scoring is perceived, the less trustworthy the AI-based system is classified. No influence could be seen regarding the relevance of the scoring for the students, their AI-experience and technology affinity.
Recommended Citation
Hartmann, Philipp; Hobert, Sebastian; and Schumann, Matthias, "Trust, but Verify! - An Empirical Investigation of Students’ Initial Trust in AI-Based Essay Scoring" (2022). AMCIS 2022 Proceedings. 5.
https://aisel.aisnet.org/amcis2022/sig_ed/sig_ed/5
Trust, but Verify! - An Empirical Investigation of Students’ Initial Trust in AI-Based Essay Scoring
AI is becoming increasingly important in supporting education. Nowadays, AI-based systems can score essays in high-stakes exams not only by comparing words but also by evaluating content. However, for AI-based essay scoring systems to be used, they must be trusted. Based on a scenario-based experiment with 260 students at a German university, we were able to show that their initial trust in AI-based essay scoring systems is significantly lower than in human examiners. Human control of AI-scoring can partially reduce the negative effect. The perceived system characteristics and the personality traits of the students are important factors which positively influence trustworthiness and trust, respectively. Furthermore, we could show that the more complex the essay scoring is perceived, the less trustworthy the AI-based system is classified. No influence could be seen regarding the relevance of the scoring for the students, their AI-experience and technology affinity.
When commenting on articles, please be friendly, welcoming, respectful and abide by the AIS eLibrary Discussion Thread Code of Conduct posted here.
Comments
SIG ED