Location
Hilton Waikoloa Village, Hawaii
Event Website
https://hicss.hawaii.edu/
Start Date
7-1-2025 12:00 AM
End Date
10-1-2025 12:00 AM
Description
In the information systems (IS) discipline, ensuring the accuracy of psychometric measurement through construct validation is paramount for theoretical advancement. Construct validity delineates the degree to which an instrument genuinely measures the concept it purports to assess. Traditional methods like principal components analysis can bias results by employing the same dataset to validate the instrument and test the proposed theory. In contrast, our research introduces a novel computational tool for evaluating construct validity, which validates the instrument separately from the data collected to test the theory. By finetuning a RoBERTa-based transformer model, we estimate the likelihood of indicator pairs being associated with the same construct. These probabilities serve as measures of indicator-to-construct concordance. Our empirical findings reveal significant congruence between our method's probabilistic evaluations and established factor loading measures. The proposed approach improves construct validity assessments and refines IS measurement tools, paving the way for more precise and reliable theoretical model development.
Recommended Citation
Sang, Lan; Larsen, Kai; and Gandhi, Fairy, "Synthetic Construct Validity: A Transformational Solution" (2025). Hawaii International Conference on System Sciences 2025 (HICSS-58). 2.
https://aisel.aisnet.org/hicss-58/os/design_science_research/2
Synthetic Construct Validity: A Transformational Solution
Hilton Waikoloa Village, Hawaii
In the information systems (IS) discipline, ensuring the accuracy of psychometric measurement through construct validation is paramount for theoretical advancement. Construct validity delineates the degree to which an instrument genuinely measures the concept it purports to assess. Traditional methods like principal components analysis can bias results by employing the same dataset to validate the instrument and test the proposed theory. In contrast, our research introduces a novel computational tool for evaluating construct validity, which validates the instrument separately from the data collected to test the theory. By finetuning a RoBERTa-based transformer model, we estimate the likelihood of indicator pairs being associated with the same construct. These probabilities serve as measures of indicator-to-construct concordance. Our empirical findings reveal significant congruence between our method's probabilistic evaluations and established factor loading measures. The proposed approach improves construct validity assessments and refines IS measurement tools, paving the way for more precise and reliable theoretical model development.
https://aisel.aisnet.org/hicss-58/os/design_science_research/2