Summary of the paper
| Title | Task-Based Evaluation of Meeting Browsers: from Task Elicitation to User Behavior Analysis |
|---|---|
| Authors | Andrei Popescu-Belis, Mike Flynn, Pierre Wellner and Philippe Baudrion |
| Abstract | This paper presents recent results of the application of the task-based Browser Evaluation Test (BET) to meeting browsers, that is, interfaces to multimodal databases of meeting recordings. The tasks were defined by browser-neutral BET observers. Two groups of human subjects used the Transcript-based Query and Browsing interface (TQB), and attempted to solve as many BET tasks - pairs of true/false statements to disambiguate - as possible in a fixed amount of time. Their performance was measured in terms of precision and speed. Results indicate that the browser’s annotation-based search functionality is frequently used, in particular the keyword search. A more detailed analysis of each test question for each participant confirms that despite considerable variation across strategies, the use of queries is correlated to successful performance. |
| Language | Single language |
| Topics | Evaluation methodologies, Multimedia annotation and processing, Tools, systems, applications |
Full paper |
Task-Based Evaluation of Meeting Browsers: from Task Elicitation to User Behavior Analysis |
Slides |
- |
| Bibtex | @InProceedings{POPESCUBELIS08.580, author = {Andrei Popescu-Belis, Mike Flynn, Pierre Wellner and Philippe Baudrion}, title = {Task-Based Evaluation of Meeting Browsers: from Task Elicitation to User Behavior Analysis}, booktitle = {Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC'08)}, year = {2008}, month = {may}, date = {28-30}, address = {Marrakech, Morocco}, editor = {Nicoletta Calzolari (Conference Chair), Khalid Choukri, Bente Maegaard, Joseph Mariani, Jan Odijk, Stelios Piperidis, Daniel Tapias}, publisher = {European Language Resources Association (ELRA)}, isbn = {2-9517408-4-0}, note = {http://www.lrec-conf.org/proceedings/lrec2008/}, language = {english} } |