Abstract
The peer review process is the bedrock of knowledge validation in the Information Systems discipline. Traditional reviewer matching relies heavily on semantic similarity or keyword matching. With the advancement of generative artificial intelligence (GenAI), the methods currently used for reviewer matching can be upgraded. We propose a reviewer matching method based on GenAI that integrates deep retrieval with deep thinking. The deep retrieval part in our method performs decomposition based on IMRaD (Kavak & Odabaş), breaking the manuscript into sub-queries corresponding to its core components, such as theoretical background, methodological innovation, and evaluation metrics. Based on sub-queries, the system searches for the previous publication of reviewers to gather evidence. To synthesize retrieved evidence into a reliable reviewer recommendation through deep thinking, we train a large language model using a novel fine-tuning approach grounded in information theory. We further introduce an information-gain reward mechanism: during fine-tuning, every reasoning step in the model's trace receives a reward proportional to how much that step reduces the system's residual uncertainty about the correct match decision. A user study will be conducted on one funding agency to test and validate the proposed method. Ultimately, we aim to urge the research community to rethink and redesign the peer reviewer matching, moving beyond similarity-driven statistical searching toward reasoning-driven interpretable matching.
Recommended Citation
Ren, Runtao; Ma, Jian; and Qiu, Zhouqiang, "Enhancing Peer Reviewer Matching through the Deep Thinking of Large Language Models" (2026). AMCIS 2026 TREOs. 38.
https://aisel.aisnet.org/treos_amcis2026/38