Using TREC for cross-comparison between classic IR and ontology-based search models at a Web scale
Entity
UAM. Departamento de Ingeniería InformáticaPublisher
Marko Grobelnik; Peter Mika; Thanh Tran Douc; Haofen WangDate
2009Citation
SemSearch 2009: Proceedings of the Workshop on Semantic Search (SemSearch 2009) at the 18th International World Wide Web Conference (WWW 2009). CEUR Workshop Proceedings, Volumen 491, 2009. 41-50ISSN
1163-0073Editor's Version
http://ceur-ws.org/Vol-491/Subjects
InformáticaNote
Also published online by CEUR Workshop Proceedings (CEUR-WS.org, ISSN 1613-0073)Proceedings of the Workshop on Semantic Search (SemSearch 2009) at the 18th International World Wide Web Conference (WWW 2009)
Rights
© 2009 The author/sAbstract
The construction of standard datasets and benchmarks to evaluate
ontology-based search approaches and to compare then against
baseline IR models is a major open problem in the semantic technologies
community. In this paper we propose a novel evaluation
benchmark for ontology-based IR models based on an adaptation
of the well-known Cranfield paradigm (Cleverdon, 1967) traditionally
used by the IR community. The proposed benchmark
comprises: 1) a text document collection, 2) a set of queries and
their corresponding document relevance judgments and 3) a set of
ontologies and Knowledge Bases covering the query topics. The
document collection and the set of queries and judgments are
taken from one of the most widely used datasets in the IR community,
the TREC Web track. As a use case example we apply the
proposed benchmark to compare a real ontology-based search
model (Fernandez, et al., 2008) against the best IR systems of
TREC 9 and TREC 2001 competitions. A deep analysis of the
strengths and weaknesses of this benchmark and a discussion of
how it can be used to evaluate other ontology-based search systems
is also included at the end of the paper.
Files in this item
Google Scholar:Fernández Sánchez, Miriam
-
López, Vanessa
-
Sabou, Marta
-
Uren, Victoria S.
-
Vallet Weadon, David Jordi
-
Motta, Enrico
-
Castells Azpilicueta, Pablo
This item appears in the following Collection(s)
Related items
Showing items related by title, author, creator and subject.