Paper Type

Complete

Abstract

Generative AI systems are increasingly used to support learning and other knowledge-based tasks, but their effectiveness depends not only on the capability of the model but also on how people interact with them. Drawing on socio-technical systems theory, this study examines prompt design as a way to better align AI-generated explanations with users’ knowledge and learning needs. We introduce Iterative Prompt Optimization (IPO), a structured framework in which prompts are refined through repeated user feedback to improve explanation quality, clarity, and confidence in complex domains. Using quantum algorithm explanation as the study context, we conducted a multi-phase study involving baseline model evaluation, iterative prompt refinement with novice users, and the development of a prototype conversational application, AlgoQ. Preliminary findings suggest that structured prompt scaffolding helps users better understand complex concepts, improves the clarity of AI-generated explanations, and increases learner confidence. This research contributes to a better understanding of human–AI collaboration and offers design insights for building effective AI-supported learning systems.

Paper Number

1884

Comments

AI SYSTEM

Download

Share

COinS
 
Aug 15th, 12:00 AM

Iterative Prompt Optimization: A Socio-Technical Framework for Human–AI Alignment in Generative AI

Generative AI systems are increasingly used to support learning and other knowledge-based tasks, but their effectiveness depends not only on the capability of the model but also on how people interact with them. Drawing on socio-technical systems theory, this study examines prompt design as a way to better align AI-generated explanations with users’ knowledge and learning needs. We introduce Iterative Prompt Optimization (IPO), a structured framework in which prompts are refined through repeated user feedback to improve explanation quality, clarity, and confidence in complex domains. Using quantum algorithm explanation as the study context, we conducted a multi-phase study involving baseline model evaluation, iterative prompt refinement with novice users, and the development of a prototype conversational application, AlgoQ. Preliminary findings suggest that structured prompt scaffolding helps users better understand complex concepts, improves the clarity of AI-generated explanations, and increases learner confidence. This research contributes to a better understanding of human–AI collaboration and offers design insights for building effective AI-supported learning systems.

When commenting on articles, please be friendly, welcoming, respectful and abide by the AIS eLibrary Discussion Thread Code of Conduct posted here.