Introducing Peer Review in an IS Analysis Course

Authors

Guttorm Sindre, Norwegian University of Science and Technology
Daniel L. Moody, Norwegian University of Science and Technology
Terje Brasethvik, Norwegian University of Science and Technology
Arne Sølvberg, Norwegian University of Science and Technology

Abstract

Peer reviews were introduced as a teaching technique for the 2002 offering of the course SIF8035 Information Systems at the Norwegian University of Science and Technology (NTNU). The students handed in conceptual models which were then double-blind reviewed by three independent peers. The review reports contained scores and defect lists, but were not used for grading. This paper conducts an analysis of the outcomes of the peer review exercises. Several approaches are used, both quantitative and qualitative, investigating the students’ performance and perceptions. The main conclusions are: 1) The scores given by the peer reviewers were not reliable enough to recommend their use for grading purposes. 2) The introduction of peer review exercises contributed positively to students’ learning in the course – but not equally so for all students. A substantial fraction of the students did so little that it is hard to claim any learning effect. The main reasons for this seem to have been poor motivation and unclear demands. In hindsight, we have discussed the distinguishing properties of three different purposes of peer-reviews that were not clear to us when the course started, and we have identified several possible ways of improving the peer reviews. In spite of the reported problems, experiences are more positive than negative, and it has been decided to continue with peer reviews in the course.

Recommended Citation

Sindre, Guttorm; Moody, Daniel L.; Brasethvik, Terje; and Sølvberg, Arne (2003) "Introducing Peer Review in an IS Analysis Course," Journal of Information Systems Education: Vol. 14 : Iss. 1 , 101-119.
Available at: https://aisel.aisnet.org/jise/vol14/iss1/11