The Importance of Evaluation for Cross-Language System Development: the CLEF Experience


Carol Peters (IEI-CNR, Area di Ricerca CNR, 56124 Pisa, Italy)

Martin Braschler (Eurospider Information Technology, Zürich, Switzerland)


EO1: Information Retrieval & Information Extraction Evaluation


The aim of the Cross-Language Evaluation Forum (CLEF) is to develop and maintain an infrastructure for the evaluation of information retrieval systems operating on European languages in both monolingual and cross-language contexts, and to create test-suites of reusable data that can be employed by system developers for benchmarking purposes. Two CLEF evaluation campaigns have been held so far (CLEF 2000 and CLEF 2001); CLEF 2002 is now under way. The paper describes the objectives and the organisation of these campaigns, and gives a first assessment of the results. In conclusion, plans for future CLEF campaigns are reported.


Cross-language information retrieval, System evaluation, Multilingual test collections, Benchmarking

Full Paper