Title Agreement in Human Factoid Annotation for Summarization Evaluation
Author(s) Simone Teufel (1), Hans van Halteren (2)

(1) Computer Laboratory, University of Cambridge,UK; (2) Department of Language and Speech, University of Nijmegen, The Netherlands

Session O39-EW
Abstract Factoid analysis was introduced by vanHalteren and Teufel (2003) as an objective, yet semantics-oriented way of measuring overlap of information rather than surface strings in summaries. In this paper, we report on annotation experiments with two sets of summaries, and on a factoid-pairing program which finds correlations between factoids semi-automatically.
Keyword(s) Evaluation, summarization, semantic overlap
Language(s) English
Full Paper 723.pdf