Paper Number
ECIS2025-1108
Paper Type
CRP
Abstract
We present an evaluation of ten state-of-the-art large language models (LLMs) taking the role of management consultants. The assessment is performed in a simulated interview setting where the LLMs are tasked with solving case studies – a popular testing procedure during job interviews at management consulting firms. We compose a dataset of eight annotated case studies with a total of 37 business questions and corresponding reference answers. We record the LLMs’ answers and evaluate their performance in terms of outcome and process validity. Our findings indicate that eight out of the ten evaluated LLMs would pass these case interviews, exceeding the minimum level of performance expected from human candidates. Further, the LLMs seem to take measures to ensure high-quality decisions. These findings highlight the potential of LLMs to serve as strategic advisors to managers in charge of making reliable decisions.
Recommended Citation
Malberg, Simon; Grigorev, Nikita; Rein, Victoria Lea; Borne Bass, Barbara; Gelvez Alvarez, Maria Alejandra; and Groh, Georg, "Bridging AI and Business: Are Large Language Models Good Management Consultants?" (2025). ECIS 2025 Proceedings. 3.
https://aisel.aisnet.org/ecis2025/ai_org/ai_org/3
Bridging AI and Business: Are Large Language Models Good Management Consultants?
We present an evaluation of ten state-of-the-art large language models (LLMs) taking the role of management consultants. The assessment is performed in a simulated interview setting where the LLMs are tasked with solving case studies – a popular testing procedure during job interviews at management consulting firms. We compose a dataset of eight annotated case studies with a total of 37 business questions and corresponding reference answers. We record the LLMs’ answers and evaluate their performance in terms of outcome and process validity. Our findings indicate that eight out of the ten evaluated LLMs would pass these case interviews, exceeding the minimum level of performance expected from human candidates. Further, the LLMs seem to take measures to ensure high-quality decisions. These findings highlight the potential of LLMs to serve as strategic advisors to managers in charge of making reliable decisions.
When commenting on articles, please be friendly, welcoming, respectful and abide by the AIS eLibrary Discussion Thread Code of Conduct posted here.