LREC 2000 2nd International Conference on Language Resources & Evaluation
 

Previous Paper   Next Paper

Title Obtaining Predictive Results with an Objective Evaluation of Spoken Dialogue Systems: Experiments with the DCR Assessment Paradigm
Authors Antoine Jean-Yves (EQUIPAGE team, VALORIA, Université de Bretagne Sud, IUP Vannes, r. Y. Mainguy, F-56000 Vannes, France., Email : Jean-Yves.Antoine@univ-ubs.fr)
Siroux Jacques (CORDIAL team, IRISA-LLI, ENSSAT, 6 r. de Kerampont, F-22305 Lannion, France., Email : siroux@enssat.fr)
Caelen Jean (CLIPS-IMAG, BP 53, F-38041 Grenoble Cedex 9, France., Email : Jean.Caelen@univ-ubs.fr @imag.fr)
Villaneau Jeanne (EQUIPAGE team, VALORIA, Université de Bretagne Sud, IUP Vannes, r. Y. Mainguy, F-56000 Vannes, France., Email : berthele@univ-ubs.fr)
Goulian Jérôme (EQUIPAGE team, VALORIA, Université de Bretagne Sud, IUP Vannes, r. Y. Mainguy, F-56000 Vannes, France., Email : Jerome.Goulian@univ-ubs.fr)
Ahafhaf Mohamed (CLIPS-IMAG, BP 53, F-38041 Grenoble Cedex 9, France., Email : Mohamed.Ahafhaf@univ-ubs.fr @imag.fr)
Keywords DCR Evaluation, Dialogue, Genericity, Multi-Criteria Diagnosis, Spoken Language Understanding
Session Session SO5 - Evaluation of Dialogue
Full Paper 36.ps, 36.pdf
Abstract The DCR methodology is a framework that proposes a generic and detailed evaluation of spoken dialog systems. We have already detailed (Antoine et al., 1998) the theoretical bases of this paradigm. In this paper, we present some experimental results on spoken language understanding that show the feasibility and the reliability of the DCR evaluation as well as its ability to provide a detailed diagnosis of the system’s behaviour. Finally, we highlight the extension of the DCR methodology to dialogue management.