| LREC 2000 2nd International Conference on Language Resources & Evaluation | ||||||
|
|
|
|
|
|
|
| Title | Evaluation of Word Alignment Systems |
| Authors | Ahrenberg Lars (Department of Computer and Information Science Linköping University, Institute of Technology, S-581 83 Linköping, Sweden, email:lah@ida.liu.se) Merkel Magnus (Department of Computer and Information Science Linköping University, Institute of Technology, S-581 83 Linköping, Sweden, email: magme@ida.liu.se) Sågvall Hein Anna (Department of Linguistics Uppsala University, Box 527, S-751 20 Uppsala, Sweden, email: anna@ling.uu.se) Tiedemann Jörg (Department of Linguistics Uppsala University, Box 527, S-751 20 Uppsala, Sweden, email: joerg@stp.ling.uu.se) |
| Keywords | Automatic Evaluation, Bilingual Lexicon Extraction, Evaluation Metrics, Gold Standard, Parallel Text, Word Alignment |
| Session | Session EO3 - Evaluation and Semantics |
| Full Paper | 137.ps, 137.pdf |
| Abstract | Recent years have seen a few serious attempts to develop methods and measures for the evaluation of word alignment systems, notably the Blinker project (Melamed, 1998) and the ARCADE project (Véronis and Langlais, forthcoming). In this paper we discuss different approaches to the problem and report on results from a project where two word alignment systems have been evaluated. These results include methods and tools for the generation of reference data and a set of measures for system performance. We note that the selection and sampling of reference data can have a great impact on scoring results. |