Advances in Research Methods
Loading...
Paper Type
Short
Paper Number
1784
Description
Considering the challenges of measuring the perceptual dimensions of the IT artifact, we propose a computational method for developing latent concepts through the lines of code that make up a software artifact. The proposed “machine-sourcing human judgement” approach, combines a novel technique of extracting the semantic properties (meanings) of the code from the software engineering literature with the machine-learning techniques used in the IS literature. Using the illustration of open source software (OSS), we demonstrate that the ‘contextual’ and ‘economic’ limitations of evaluating the creativity of OSS code contributions can be overcome through our approach. The performance of the proposed approach is tested by using a labelled dataset of code contributions created by two experienced OSS developers. We find that our approach of using semantic properties from the “software code” matches in performance to evaluating “textual descriptions” of the code. Potential methodological improvements and future research opportunities are also discussed.
Recommended Citation
Medappa, Poonacha K. and Srivastava, Shirish C., "Can Human Judgement be Machine-Sourced? An Approach to Measure the Perceptual Dimensions Embedded in Software" (2020). ICIS 2020 Proceedings. 4.
https://aisel.aisnet.org/icis2020/adv_research_methods/adv_research_methods/4
Can Human Judgement be Machine-Sourced? An Approach to Measure the Perceptual Dimensions Embedded in Software
Considering the challenges of measuring the perceptual dimensions of the IT artifact, we propose a computational method for developing latent concepts through the lines of code that make up a software artifact. The proposed “machine-sourcing human judgement” approach, combines a novel technique of extracting the semantic properties (meanings) of the code from the software engineering literature with the machine-learning techniques used in the IS literature. Using the illustration of open source software (OSS), we demonstrate that the ‘contextual’ and ‘economic’ limitations of evaluating the creativity of OSS code contributions can be overcome through our approach. The performance of the proposed approach is tested by using a labelled dataset of code contributions created by two experienced OSS developers. We find that our approach of using semantic properties from the “software code” matches in performance to evaluating “textual descriptions” of the code. Potential methodological improvements and future research opportunities are also discussed.
When commenting on articles, please be friendly, welcoming, respectful and abide by the AIS eLibrary Discussion Thread Code of Conduct posted here.
Comments
19-Methods