Paper Type
ERF
Abstract
Large language models (LLMs) are increasingly used in policy analysis, where evaluation often relies on prompt optimization followed by output assessment. This ERF paper examines evaluative misalignment in two heuristic cases from our applied LLM work in policy analysis: qualitative coding and policy summarization. We identify two potential forms of misalignment in LLM evaluation processes: between human reasoning and model behavior, and among human evaluators. The cases show that plausible or high-scoring outputs may still diverge from the reasoning and contextual interpretation required in policy tasks. We suggest that LLM output evaluation in such settings should be understood as a process-centered, role-aligned activity. The paper offers a preliminary account of how evaluative misalignment emerges when output scores, model behavior, and human judgment are positioned within policy analysis workflows.
Paper Number
1709
Recommended Citation
Yang, Shengnan and Zhu, Xiaohua Awa, "Evaluative Misalignment in LLM-Supported Policy Analysis: Two Heuristic Cases" (2026). AMCIS 2026 Proceedings. 26.
https://aisel.aisnet.org/amcis2026/conftheme/conftheme/26
Evaluative Misalignment in LLM-Supported Policy Analysis: Two Heuristic Cases
Large language models (LLMs) are increasingly used in policy analysis, where evaluation often relies on prompt optimization followed by output assessment. This ERF paper examines evaluative misalignment in two heuristic cases from our applied LLM work in policy analysis: qualitative coding and policy summarization. We identify two potential forms of misalignment in LLM evaluation processes: between human reasoning and model behavior, and among human evaluators. The cases show that plausible or high-scoring outputs may still diverge from the reasoning and contextual interpretation required in policy tasks. We suggest that LLM output evaluation in such settings should be understood as a process-centered, role-aligned activity. The paper offers a preliminary account of how evaluative misalignment emerges when output scores, model behavior, and human judgment are positioned within policy analysis workflows.
When commenting on articles, please be friendly, welcoming, respectful and abide by the AIS eLibrary Discussion Thread Code of Conduct posted here.
Comments
NEXTTRANS