SIG ODIS - Artificial Intelligence and Semantic Technologies for Intelligent Systems
Loading...
Paper Type
Complete
Paper Number
1713
Description
Chatbots have been used for basic conversational functionalities and task performance in today's world. With the surge in the use of chatbots, several design features have emerged to cater to its rising demands and increasing complexity. Researchers have grappled with the issues of modeling and evaluating these tools because of the vast number of metrics associated with their measure of successful. This paper conducted a literature survey to identify the various conversational metrics used to evaluate chatbots. The selected evaluation metrics were mapped to the various layers of The Open Group Architecture Framework (TOGAF) architecture. TOGAF architecture helped us divide the metrics based on the various facets critical to developing successful chatbot applications. Our results show that the metrics related to the business layer have been well studied. However, metrics associated with the data, information, and system layers warrant more research. As chatbots become more complex, success metrics across the intermediate layers may assume greater significance.
Recommended Citation
THIMMANAYAKANAPALYA, Sagarika Suresh; Mulgund, Pavankumar; and Sharman, Raj, "A TOGAF Based Chatbot Evaluation Metrics: Insights from Literature Review" (2022). AMCIS 2022 Proceedings. 18.
https://aisel.aisnet.org/amcis2022/sig_odis/sig_odis/18
A TOGAF Based Chatbot Evaluation Metrics: Insights from Literature Review
Chatbots have been used for basic conversational functionalities and task performance in today's world. With the surge in the use of chatbots, several design features have emerged to cater to its rising demands and increasing complexity. Researchers have grappled with the issues of modeling and evaluating these tools because of the vast number of metrics associated with their measure of successful. This paper conducted a literature survey to identify the various conversational metrics used to evaluate chatbots. The selected evaluation metrics were mapped to the various layers of The Open Group Architecture Framework (TOGAF) architecture. TOGAF architecture helped us divide the metrics based on the various facets critical to developing successful chatbot applications. Our results show that the metrics related to the business layer have been well studied. However, metrics associated with the data, information, and system layers warrant more research. As chatbots become more complex, success metrics across the intermediate layers may assume greater significance.
When commenting on articles, please be friendly, welcoming, respectful and abide by the AIS eLibrary Discussion Thread Code of Conduct posted here.
Comments
SIG ODIS