Paper Number
ICIS2025-1718
Paper Type
Complete
Abstract
Supplier qualification assessment is pivotal for robust supply chain management, particularly in industries with complex production ecosystems. Traditional multi-dimensional indicator systems often suffer from subjective biases and fail to delineate specific strengths or weaknesses in supplier evaluations. This study proposes a novel framework integrating large language models (LLMs) with Chain-of-Thought (CoT) and Retrieval-Augmented Generation (RAG) technologies—termed Eval-RCoT—to enhance the precision and interpretability of supplier assessments. Leveraging a novel dataset from a major Chinese manufacturing enterprise, we evaluate the performance of ChatGPT and GLM in executing supplier evaluations. Our experiments reveal that Eval-RCoT effectively generates accurate rating levels and interpretable management insights. Quantitative validation via classification metrics and LLM-based metrics underscores the framework's reliability. These findings highlight the potential of LLM-driven approaches to mitigate subjectivity and streamline supplier evaluation processes in high-stakes industries, offering a scalable solution for dynamic supply chain demands.
Recommended Citation
Li, Xuerong; Shang, Wei; Niu, Zihang; Xie, Bingxin; and Zhu, Lexiang, "Can large language models evaluate suppliers’ qualifications of complex production lines? An approach incorporating retrieval-augmented generation and Chain-of-Thought" (2025). ICIS 2025 Proceedings. 5.
https://aisel.aisnet.org/icis2025/da_bus/da_bus/5
Can large language models evaluate suppliers’ qualifications of complex production lines? An approach incorporating retrieval-augmented generation and Chain-of-Thought
Supplier qualification assessment is pivotal for robust supply chain management, particularly in industries with complex production ecosystems. Traditional multi-dimensional indicator systems often suffer from subjective biases and fail to delineate specific strengths or weaknesses in supplier evaluations. This study proposes a novel framework integrating large language models (LLMs) with Chain-of-Thought (CoT) and Retrieval-Augmented Generation (RAG) technologies—termed Eval-RCoT—to enhance the precision and interpretability of supplier assessments. Leveraging a novel dataset from a major Chinese manufacturing enterprise, we evaluate the performance of ChatGPT and GLM in executing supplier evaluations. Our experiments reveal that Eval-RCoT effectively generates accurate rating levels and interpretable management insights. Quantitative validation via classification metrics and LLM-based metrics underscores the framework's reliability. These findings highlight the potential of LLM-driven approaches to mitigate subjectivity and streamline supplier evaluation processes in high-stakes industries, offering a scalable solution for dynamic supply chain demands.
When commenting on articles, please be friendly, welcoming, respectful and abide by the AIS eLibrary Discussion Thread Code of Conduct posted here.
Comments
07-DataAnalytics