Are Citations to Academic Journal Articles a Measure of Quality or Something Else? An Exploratory Analysis with Emphasis on Design Science and IS Technical Research
Global, International and Cultural Issues in IS
One pervasive belief about scholarly citations that underlies several studies appearing in the IS literature is that the number ofcitations to a paper reflects the quality of the study. For example, a paper that receives 200 citations is perceived as havinghigher quality than another study that receives 50 citations. While most experts acknowledge that there are other factors thatdrive citations, such as the number of years a paper was available to be cited – or possibly “gaming” of citations by authorswho cite themselves frequently (author self-citations) or over-zealous editors who incent prospective authors to cite theirjournals (journal self-citations), there is an underlying assumption that, barring such unscrupulous behavior, citations are ameasure of research quality. This paper critically examines this assumption using a validated typology identifying 13 distinctsubject areas that characterize IS research. Results show that papers on some topics (e.g., IT adoption and use; newmeasurement development and validation) consistently receive more citations than average – while some topics consistentlyreceive below-average citation rates (e.g., IS development; IT project/risk management). Unless one assumes that all paperson a given topic are consistently of higher quality than all papers on other topics, our data suggest that a key driver ofcitations are the subject area of the study and, in turn, the size of the research community that conducts research on the topic.
Gallivan, Michael J., "Are Citations to Academic Journal Articles a Measure of Quality or Something Else? An Exploratory Analysis with Emphasis on Design Science and IS Technical Research" (2011). AMCIS 2011 Proceedings - All Submissions. 363.