Paper Type

Complete

Paper Number

1837

Description

This study explores the application of Mixed Methods (MM) in evaluating user experience (UX) quality through sentiment analysis and large language models (LLMs) like ChatGPT-4, focusing on the legitimacy of mixed-method data. This research uses advanced Natural Language Processing tools to analyze qualitative data transformed into quantitative metrics with the User Experience Quality (UEQ) survey. By comparing these metrics with original respondent scores, the study aims to assess the consistency of MM data and explore potential discrepancies between quantitative and qualitative findings. Our methodology also investigates the reliability of MM approaches by integrating sentimentR and ChatGPT-4, highlighting their roles in identifying and rectifying inconsistencies in survey data. The findings are expected to offer significant insights into the integration of MM in UX studies, providing a robust framework for ensuring data validity and enhancing survey methodologies.

Comments

User

Share

COinS
 
Jul 2nd, 12:00 AM

Measuring Internal Validity in Mixed Methods User Feedback Survey by Finding Inconsistencies with sentimentR and ChatGPT-4

This study explores the application of Mixed Methods (MM) in evaluating user experience (UX) quality through sentiment analysis and large language models (LLMs) like ChatGPT-4, focusing on the legitimacy of mixed-method data. This research uses advanced Natural Language Processing tools to analyze qualitative data transformed into quantitative metrics with the User Experience Quality (UEQ) survey. By comparing these metrics with original respondent scores, the study aims to assess the consistency of MM data and explore potential discrepancies between quantitative and qualitative findings. Our methodology also investigates the reliability of MM approaches by integrating sentimentR and ChatGPT-4, highlighting their roles in identifying and rectifying inconsistencies in survey data. The findings are expected to offer significant insights into the integration of MM in UX studies, providing a robust framework for ensuring data validity and enhancing survey methodologies.

When commenting on articles, please be friendly, welcoming, respectful and abide by the AIS eLibrary Discussion Thread Code of Conduct posted here.