Paper Number

1489

Paper Type

Complete Research Paper

Abstract

Automatic item generation (AIG) is an emerging practice that has the potential to revolutionize empirical research by freeing scholars from the tedious task of creating survey items. Given the theme of the conference, “People First: Constructing Digital Futures Together”, it is relevant to question the role of IS researchers when the item generation task is outsourced to a robot. In this paper, I generated multiple parallel versions of the Cognitive Absorption scale using ChatGPT4.0, a self-sufficient natural language processing algorithm. Specifically, I examined how alternative model creativity parameters influence the linguistic properties of the created scale’s items. The results show that AIG can help create more grammatically diverse and content-valid scales, but increased creativity levels can undermine the scale's readability. This paper contributes to the burgeoning literature on the impact of generative AI on research practices.

Share

COinS
 
Jun 14th, 12:00 AM

A Linguistic Perspective On AI-generated Scales: Readability, Diversity, And Content Validity Considerations

Automatic item generation (AIG) is an emerging practice that has the potential to revolutionize empirical research by freeing scholars from the tedious task of creating survey items. Given the theme of the conference, “People First: Constructing Digital Futures Together”, it is relevant to question the role of IS researchers when the item generation task is outsourced to a robot. In this paper, I generated multiple parallel versions of the Cognitive Absorption scale using ChatGPT4.0, a self-sufficient natural language processing algorithm. Specifically, I examined how alternative model creativity parameters influence the linguistic properties of the created scale’s items. The results show that AIG can help create more grammatically diverse and content-valid scales, but increased creativity levels can undermine the scale's readability. This paper contributes to the burgeoning literature on the impact of generative AI on research practices.

When commenting on articles, please be friendly, welcoming, respectful and abide by the AIS eLibrary Discussion Thread Code of Conduct posted here.