Paper Number
1489
Paper Type
Complete Research Paper
Abstract
Automatic item generation (AIG) is an emerging practice that has the potential to revolutionize empirical research by freeing scholars from the tedious task of creating survey items. Given the theme of the conference, “People First: Constructing Digital Futures Together”, it is relevant to question the role of IS researchers when the item generation task is outsourced to a robot. In this paper, I generated multiple parallel versions of the Cognitive Absorption scale using ChatGPT4.0, a self-sufficient natural language processing algorithm. Specifically, I examined how alternative model creativity parameters influence the linguistic properties of the created scale’s items. The results show that AIG can help create more grammatically diverse and content-valid scales, but increased creativity levels can undermine the scale's readability. This paper contributes to the burgeoning literature on the impact of generative AI on research practices.
Recommended Citation
Pillet, Jean-Charles, "A Linguistic Perspective on AI-Generated Scales: Readability, Diversity, and Content Validity Considerations" (2024). ECIS 2024 Proceedings. 4.
https://aisel.aisnet.org/ecis2024/track22_innrm/track22_innrm/4
A Linguistic Perspective on AI-Generated Scales: Readability, Diversity, and Content Validity Considerations
Automatic item generation (AIG) is an emerging practice that has the potential to revolutionize empirical research by freeing scholars from the tedious task of creating survey items. Given the theme of the conference, “People First: Constructing Digital Futures Together”, it is relevant to question the role of IS researchers when the item generation task is outsourced to a robot. In this paper, I generated multiple parallel versions of the Cognitive Absorption scale using ChatGPT4.0, a self-sufficient natural language processing algorithm. Specifically, I examined how alternative model creativity parameters influence the linguistic properties of the created scale’s items. The results show that AIG can help create more grammatically diverse and content-valid scales, but increased creativity levels can undermine the scale's readability. This paper contributes to the burgeoning literature on the impact of generative AI on research practices.
When commenting on articles, please be friendly, welcoming, respectful and abide by the AIS eLibrary Discussion Thread Code of Conduct posted here.