Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

The TRECVid 2008 BBC Rushes Summarization Evaluation



Paul D. Over, Alan Smeaton, George M. Awad


This paper describes an evaluation of automatic video summarization systems run on rushes from several BBC dramatic series. It was carried out under the auspices of the TREC Video Retrieval Evaluation (TRECVid) as a followup to the 2007 video summarization workshop held at ACM Multimedia 2007. 31 research teams submitted video summaries of 40 individual rushes video files, aiming to compress out redundant and insignificant material. Each summary had a duration of at most 2\% of the original. The output of a baseline system, which simply presented each full video at 50 times normal speed was contributed by Carn\-egie Mellon University (CMU) as a control. The 2007 procedures for developing ground truth lists of important segments from each video were applied at the National Institute of Standards and Technology (NIST) to the BBC videos. At Dublin City University (DCU) each summary was judged by 3 humans with respect to how much of the ground truth was included and how well-formed the summary was. Additional objective measures included: how long it took the system to create the summary, how long it took the assessor to judge it against the ground truth, and what the summary's duration was. Assessor agreement on finding desired segments averaged 81\%. Results indicated that while it was still difficult to exceed the performance of the baseline on including ground truth, the baseline was outperformed by most other systems with respect to avoiding redundancy/junk and presenting the summary with a pleasant tempo/rhythm.
Proceedings Title
TVS'08: Proceedings of the International Workshop on TRECVid Video Summarization
Conference Dates
October 27-31, 2008
Conference Location
Vancouver, CA
Conference Title
ACM Conference on Multimedia 2008


benchmarking, evaluation, TRECVid, video summarization


Over, P. , Smeaton, A. and Awad, G. (2009), The TRECVid 2008 BBC Rushes Summarization Evaluation, TVS'08: Proceedings of the International Workshop on TRECVid Video Summarization, Vancouver, CA, [online], (Accessed June 16, 2024)


If you have any questions about this publication or are having problems accessing it, please contact

Created May 28, 2009, Updated February 19, 2017