LREC 2000 2nd International Conference on Language Resources & Evaluation

Previous Paper   Next Paper

Title Inter-annotator Agreement for a German Newspaper Corpus
Authors Brants Thorsten (Computational Linguistics, Saarland University, 66041 Saarbrücken, Germany,
Keywords Corpus Annotation, Evaluation, Inter-Annotator Agreement
Session Session WO18 - Morphology in Lexical and Textual Resources
Full Paper, 333.pdf
Abstract This paper presents the results of an investigation on inter-annotator agreement for the NEGRA corpus, consisting of German newspaper texts. The corpus is syntactically annotated with part-of-speech and structural information. Agreement for part-of-speech is 98.6%, the labeled F-score for structures is 92.4%. The two annotations are used to create a common final version by discussing differences and by several iterations of cleaning. Initial and final versions are compared. We identify categories causing large numbers of differences and categories that are handled inconsistently.