Location
Hilton Waikoloa Village, Hawaii
Event Website
https://hicss.hawaii.edu/
Start Date
7-1-2025 12:00 AM
End Date
10-1-2025 12:00 AM
Description
Artificial Intelligence (AI) is revolutionizing education, particularly with advancements in generative AI conversational agents. Our study investigates the effectiveness of these generative AI agents in enhancing English-speaking skills compared to human agents. Through a randomized field experiment involving 363 participants, we found that unrestricted use of AI tools led to a 5.90% improvement in lexical diversity, as measured by the Type-Token Ratio (TTR), highlighting the benefits of self-paced learning. Notably, learners with below-average proficiency experienced a 9.53% improvement in TTR, suggesting AI's potential to bridge educational equity gaps. Moreover, AI tools significantly reduced evaluation apprehension, further enhancing learning outcomes. These findings underscore AI's capacity to provide personalized, anxiety-free learning environments, particularly for students with lower proficiency, and offer valuable insights for integrating AI into educational strategies to foster more inclusive learning experiences.
Recommended Citation
Cheon, Gyeombi; Choi, Yunmin; Lee, Dongwon; and Baek, Jiye, "Generative AI Agents in Language Learning: A Randomized Field Experiment" (2025). Hawaii International Conference on System Sciences 2025 (HICSS-58). 6.
https://aisel.aisnet.org/hicss-58/ks/higher_education/6
Generative AI Agents in Language Learning: A Randomized Field Experiment
Hilton Waikoloa Village, Hawaii
Artificial Intelligence (AI) is revolutionizing education, particularly with advancements in generative AI conversational agents. Our study investigates the effectiveness of these generative AI agents in enhancing English-speaking skills compared to human agents. Through a randomized field experiment involving 363 participants, we found that unrestricted use of AI tools led to a 5.90% improvement in lexical diversity, as measured by the Type-Token Ratio (TTR), highlighting the benefits of self-paced learning. Notably, learners with below-average proficiency experienced a 9.53% improvement in TTR, suggesting AI's potential to bridge educational equity gaps. Moreover, AI tools significantly reduced evaluation apprehension, further enhancing learning outcomes. These findings underscore AI's capacity to provide personalized, anxiety-free learning environments, particularly for students with lower proficiency, and offer valuable insights for integrating AI into educational strategies to foster more inclusive learning experiences.
https://aisel.aisnet.org/hicss-58/ks/higher_education/6