Summary of the paper

Title Extrinsic Corpus Evaluation with a Collocation Dictionary Task
Authors Adam Kilgarriff, Pavel Rychlý, Milos Jakubicek, Vojtěch Kovář, Vit Baisa and Lucia Kocincová
Abstract The NLP researcher or application-builder often wonders ``what corpus should I use, or should I build one of my own? If I build one of my own, how will I know if I have done a good job?'' Currently there is very little help available for them. They are in need of a framework for evaluating corpora. We develop such a framework, in relation to corpora which aim for good coverage of `general language'. The task we set is automatic creation of a publication-quality collocations dictionary. For a sample of 100 headwords of Czech and 100 of English, we identify a gold standard dataset of (ideally) all the collocations that should appear for these headwords in such a dictionary. The datasets are being made available alongside this paper. We then use them to determine precision and recall for a range of corpora, with a range of parameters.
Topics Evaluation Methodologies, MultiWord Expressions & Collocations
Full paper Extrinsic Corpus Evaluation with a Collocation Dictionary Task
Bibtex @InProceedings{KILGARRIFF14.52,
  author = {Adam Kilgarriff and Pavel Rychlý and Milos Jakubicek and Vojtěch Kovář and Vit Baisa and Lucia Kocincová},
  title = {Extrinsic Corpus Evaluation with a Collocation Dictionary Task},
  booktitle = {Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)},
  year = {2014},
  month = {may},
  date = {26-31},
  address = {Reykjavik, Iceland},
  editor = {Nicoletta Calzolari (Conference Chair) and Khalid Choukri and Thierry Declerck and Hrafn Loftsson and Bente Maegaard and Joseph Mariani and Asuncion Moreno and Jan Odijk and Stelios Piperidis},
  publisher = {European Language Resources Association (ELRA)},
  isbn = {978-2-9517408-8-4},
  language = {english}
Powered by ELDA © 2014 ELDA/ELRA