The multi-domain international search on speech 2020 ALBAYZIN evaluation: Overview, systems, results, discussion and post-evaluation analyses
EntityUAM. Departamento de Tecnología Electrónica y de las Comunicaciones
10.3390/app11188519Applied Sciences-Basel 11.18 (2021): 8519
Funded byThis research was funded by the Ministry of Science, Innovation and Universities of Spain, grant number RTI2018-095324-B-I00, and project DSForSec (grant number RTI2018-098091-B-I00). The APC was funded by the project DSForSec (grant number RTI2018-098091-B-I00) from the Ministry of Science, Innovation and Universities of Spain
ProjectGobierno de España. RTI2018-095324-B-I00; Gobierno de España. RTI2018-098091-B-I00
SubjectsInternational evaluation; Query-by-example spoken term detection; Search on speech; Spanish language; Spoken term detection; Telecomunicaciones
Rights© The author(s)
Esta obra está bajo una Licencia Creative Commons Atribución 4.0 Internacional.
The large amount of information stored in audio and video repositories makes search on speech (SoS) a challenging area that is continuously receiving much interest. Within SoS, spoken term detection (STD) aims to retrieve speech data given a text-based representation of a search query (which can include one or more words). On the other hand, query-by-example spoken term detection (QbE STD) aims to retrieve speech data given an acoustic representation of a search query. This is the first paper that presents an internationally open multi-domain evaluation for SoS in Spanish that includes both STD and QbE STD tasks. The evaluation was carefully designed so that several post-evaluation analyses of the main results could be carried out. The evaluation tasks aim to retrieve the speech files that contain the queries, providing their start and end times and a score that reflects how likely the detection within the given time intervals and speech file is. Three different speech databases in Spanish that comprise different domains were employed in the evaluation: the MAVIR database, which comprises a set of talks from workshops; the RTVE database, which includes broadcast news programs; and the SPARL20 database, which contains Spanish parliament sessions. We present the evaluation itself, the three databases, the evaluation metric, the systems submitted to the evaluation, the evaluation results and some detailed post-evaluation analyses based on specific query properties (in-vocabulary/out-of-vocabulary queries, single-word/multi-word queries and native/foreign queries). The most novel features of the submitted systems are a data augmentation technique for the STD task and an end-to-end system for the QbE STD task. The obtained results suggest that there is clearly room for improvement in the SoS task and that performance is highly sensitive to changes in the data domain
Google Scholar:Tejedor, Javier - Toledano, Doroteo T. - Ramirez, Jose M. - Montalvo, Ana R. - Alvarez-Trejos, Juan Ignacio
This item appears in the following Collection(s)
Showing items related by title, author, creator and subject.
Tejedor, Javier; Toledano, Doroteo T.; Lopez-Otero, Paula; Docío-Fernández, Laura; García-Mateo, Carmen García; Cardenal, Antonio; Echeverry-Correa, Julián David; Coucheiro-Limeres, Alejandro; Olcoz, Julia; Miguel, Antonio
Tejedor, Javier; Toledano, Doroteo T.; Lopez-Otero, Paula; Docío-Fernández, Laura; García-Mateo, Carmen García