NIST Language Technology Evaluation Cookbook
Alvin F. Martin, John S. Garofolo, Jonathan C. Fiscus, Audrey N. Le, David S. Pallett, Mark A. Przybocki, Gregory A. Sanders
National Institute of Standards and Technology, Gaithersburg, MD, USA
We review some of the methodology applied to the various NIST language technology evaluations. We discuss the elements included in each evaluation plan, and suggest what we believe are key practices for successful evaluations, and what pitfalls should be avoided. A couple of lessons learned are noted.
evaluation methodology, language technology, evaluation plan