Automating the Measurement of Heterogeneous Chatbot Designs
Entity
UAM. Departamento de Ingeniería InformáticaPublisher
ACMDate
2022-04-25Citation
10.1145/3477314.3507255
SAC’22. Brno, Czech Republic, April 25- April 29, 2022
ISBN
978-1-4503-8713-2DOI
10.1145/3477314.3507255Funded by
Work funded by the Spanish Ministry of Science (RTI2018-095255B-I00) and the R&D programme of Madrid (P2018/TCS-4314).Project
Gobierno de España. RTI2018-095255B-I00; Comunidad de Madrid. P2018/TCS-4314Editor's Version
https://doi.org/10.1145/3477314.3507255Subjects
Chatbot design; metrics; quality assurance; InformáticaRights
© 2022 Association for Computing MachineryAbstract
Chatbots are being increasingly used to provide a natural language interface to all kinds of software services. However, while there are many platforms and tools for chatbot development, they typically lack support to statically measure properties of the designed chatbots, as indicators of their size, complexity, quality or usability, and facilitating comparison. To attack this problem, in this paper we propose a suite of 20 metrics for chatbot designs. The metrics are defined on a neutral chatbot design language, becoming independent of the implementation platform. We have developed a tool, called Asymob, which supports the translation of chatbots defined in several platforms into this neutral format to perform the measurements. As a proofof-concept, we evaluate the metrics over a collection of Dialogflow and Rasa chatbots from several sources and open-source repositories. Our metrics helped detecting quality issues statically, and served as a basis for comparing chatbots from different origins and built using different technologies.
Files in this item
Google Scholar:Cerro Cañizares, Pablo
-
Guerra, Esther
-
Pérez-Soler, Sara
-
De Lara, Juan
This item appears in the following Collection(s)
Related items
Showing items related by title, author, creator and subject.