Abstract
This work explores the integration of large language models (LLMs) with Retrieval Augmented Generation (RAG) to enhance internal support systems. Focusing on closed-domain knowledge bases, the study develops a chatbot deployed within a team communication platform to improve accessibility and efficiency. Through a modular framework, various vector storage, document structure and retrieval methods were evaluated, alongside multiple LLM architectures, balancing performance and resource constraints. Initial experiments highlight challenges such as handling multi-question inputs, user language variability, and context-aware response generation. A manual evaluation process helped refine system configurations, with plans for comprehensive user testing in collaboration with the technical team. This project demonstrates the practical potential of LLMs with RAG for scalable, context-aware internal support systems.
Recommended Citation
Brito, Catarina; Lopes, António Luís; and Batista, Fernando, "Chatbot based on LLMs for Support and Development Teams" (2025). CAPSI 2025 Proceedings. 20.
https://aisel.aisnet.org/capsi2025/20