Construction of a Word Sense Tagged Corpus for SENSEVAL-2 Japanese Dictionary Task


Kiyoaki Shirai (Japan Advanced Institute of Science and Technology  1-1 Asahidai, Tatsunokuchi, Ishikawa, Japan)


EP1: Evaluation


This paper reports the details of a Japanese word sense tagged corpus developed as an evaluation data for SENSEVAL-2 Japanese dictionary task. The corpus made up of 2,130 newspaper articles. Not all but only 10,000 words in the articles were manually annotated with sense IDs, which was used as a gold standard data. Word senses were defined according to the Iwanami Kokugo Jiten, a Japanese dictionary published by Iwanami Shoten. Two annotators chose a sense ID for each instance separately. If they did not agree, the third annotator chose the correct sense ID between them. Inter-tagger agreement and Cohen's \kappa was 86.3% and 0.677, respectively.


Word sense tagged corpus, SENSEVAL-2, Manual annotation, Inter-Tagger agreement

Full Paper