•  
  •  
 
Communications of the Association for Information Systems

Author ORCID Identifier

Hassan Kalantari Daronkola: 0000-0003-3972-0788

Michael Sadeghi: 0000-0002-3759-2746

Reza Kacouie: 0000-0002-9868-349X

Reza Tajaddini: 0000-0003-4554-3482

Yoga Sembada: 0000-0001-7385-6094

Abstract

Universities are rapidly using AI in assessment, but little is known about how university students evaluate alternative configurations and how they shape students’ acceptance. We conceptualize AI-assisted assessment as a sociotechnical information system and examine how its specific system features influence learners’ acceptance and willingness to enroll in a unit that uses AI-assisted marking. Using a scenario-based conjoint experiment with 340 university students, we manipulate five attributes of an AI-assisted marking configuration: marker type, recognition of creativity, reliability of marking, feedback aggregation, and assessment turnaround time, across three motivational scenarios. Hierarchical Bayes estimation revealed a strong preference for human–AI hybrid marking, and acceptance is most strongly influenced by the presence of human markers. Students also place substantial weight on marking described as reliable and objective, although these should be interpreted as perceived assurance cues rather than verified system properties. Motivational context shapes these trade-offs: students prioritize rich feedback and recognition of creativity for self-development, but value speed in compliance scenarios. We drive design principles for acceptable AI assessment systems, highlighting the importance of visible human oversight, demonstrable marking consistency, and personalized feedback. The study contributes to information systems research on human–AI decision-making and to higher education for governing and implementing AI-assisted assessment.

Download

Share

COinS
 

When commenting on articles, please be friendly, welcoming, respectful and abide by the AIS eLibrary Discussion Thread Code of Conduct posted here.