Summary of the paper

Title VERTa: Facing a Multilingual Experience of a Linguistically-based MT Evaluation
Authors Elisabet Comelles, Jordi Atserias, Victoria Arranz, Irene Castellon and Jordi Sesé
Abstract There are several MT metrics used to evaluate translation into Spanish, although most of them use partial or little linguistic information. In this paper we present the multilingual capability of VERTa, an automatic MT metric that combines linguistic information at lexical, morphological, syntactic and semantic level. In the experiments conducted we aim at identifying those linguistic features that prove the most effective to evaluate adequacy in Spanish segments. This linguistic information is tested both as independent modules (to observe what each type of feature provides) and in a combinatory fastion (where different kinds of information interact with each other). This allows us to extract the optimal combination. In addition we compare these linguistic features to those used in previous versions of VERTa aimed at evaluating adequacy for English segments. Finally, experiments show that VERTa can be easily adapted to other languages than English and that its collaborative approach correlates better with human judgements on adequacy than other well-known metrics.
Topics Tools, Systems, Applications, Evaluation Methodologies
Full paper VERTa: Facing a Multilingual Experience of a Linguistically-based MT Evaluation
Bibtex @InProceedings{COMELLES14.1032,
  author = {Elisabet Comelles and Jordi Atserias and Victoria Arranz and Irene Castellon and Jordi Sesé},
  title = {VERTa: Facing a Multilingual Experience of a Linguistically-based MT Evaluation},
  booktitle = {Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)},
  year = {2014},
  month = {may},
  date = {26-31},
  address = {Reykjavik, Iceland},
  editor = {Nicoletta Calzolari (Conference Chair) and Khalid Choukri and Thierry Declerck and Hrafn Loftsson and Bente Maegaard and Joseph Mariani and Asuncion Moreno and Jan Odijk and Stelios Piperidis},
  publisher = {European Language Resources Association (ELRA)},
  isbn = {978-2-9517408-8-4},
  language = {english}
Powered by ELDA © 2014 ELDA/ELRA