The Importance of Evaluation for Cross-Language System Development: the CLEF Experience
Carol Peters (IEI-CNR, Area di Ricerca CNR, 56124 Pisa, Italy)
Martin Braschler (Eurospider Information Technology, Zürich, Switzerland)
EO1: Information Retrieval & Information Extraction Evaluation
The aim of the Cross-Language Evaluation Forum (CLEF) is to develop and maintain an infrastructure for the evaluation of information retrieval systems operating on European languages in both monolingual and cross-language contexts, and to create test-suites of reusable data that can be employed by system developers for benchmarking purposes. Two CLEF evaluation campaigns have been held so far (CLEF 2000 and CLEF 2001); CLEF 2002 is now under way. The paper describes the objectives and the organisation of these campaigns, and gives a first assessment of the results. In conclusion, plans for future CLEF campaigns are reported.
Cross-language information retrieval, System evaluation, Multilingual test collections, Benchmarking