NIST Rich Transcription 2002 Evaluation: A Preview

John Garofolo (National Institute of Standards and Technology 100 Bureau Drive, Mail Stop 8940 Gaithersburg, MD 20899-8940, USA)

Jonathan G. Fiscus (National Institute of Standards and Technology 100 Bureau Drive, Mail Stop 8940 Gaithersburg, MD 20899-8940, USA)

Alvin Martin (National Institute of Standards and Technology 100 Bureau Drive, Mail Stop 8940 Gaithersburg, MD 20899-8940, USA)

David Pallett (National Institute of Standards and Technology 100 Bureau Drive, Mail Stop 8940 Gaithersburg, MD 20899-8940, USA)

Mark Przybocki (National Institute of Standards and Technology 100 Bureau Drive, Mail Stop 8940 Gaithersburg, MD 20899-8940, USA)

SO3: Dialogue-Conversation Evaluation

The National Institute of Standards and Technology (NIST) has been implementing evaluations of automatic speech transcription technologies for over 15 years. NIST has  helped guide progress in these technologies by: creating increasingly challenging and realistic tests, helping to provide associated linguistic resources, employing uniform metrics and analyses across systems to assess performance, and sponsoring evaluation-related technology workshops. Over time, this approach has shown great progress in the technology as the test domains have become more difficult and error rates  have almost consistently decreased. In conjunction with the new DARPA Effective,  Affordable, Reusable Speech (EARS) Program, NIST has begun an evaluation effort to help move the stateof- the-art to the next level in the form of a Rich Transcription (RT)  evaluation program. RT is defined to be an integrated combination of speech-to-text  generation (STT) and metadata (MD) annotation as applied to multiple domains such as speech from Broadcast  News, telephone conversations, and meetings. The Rich Transcription 2002 (RT-02) evaluation will have been the first in an annual  series of evaluations and workshops focusing on this technology.

Rich transcription

130.pdf