Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

The TRECVID 2007 BBC Rushes Summarization Evaluation Pilot

Published

Author(s)

Paul D. Over, Alan Smeaton, Phillip Kelly

Abstract

This paper provides an overview of a pilot evaluation of video summaries produced automatically from unedited BBC dramatic series video. It was carried out under the auspices of the TREC Video Retrieval Evaluations (TRECVID.) Twenty-two research teams submitted video summaries compressed to at least 4% of the original video's duration to remove redundant and insignificant material. The output of two baseline systems was contributed by Carnegie Mellon University. Ground truth lists of important segments from each video were created by Dublin City University. At NIST each summary was judged by three humans with respect to how much of the ground truth was included, how easy the summary was to understand, and how much repeated material the summary contained. Additional objective measures included: how long it took the assessor to judge the included ground truth and how much system effort was expended creating each summary. Pairwise agreement on judging the inclusion of ground truth averaged 78%. Assessor agreement on amount of repeated video was less than on the ease of understanding the summary. While it appears it was difficult to exceed the two baselines, a few systems did.
Proceedings Title
2007 ACM Multimedia Conference: TRECVID Video Summarization Workshop Proceedings
Conference Title
2007 ACM Multimedia Conference: TRECVID Video Summarization Workshop

Keywords

BBC rushes, evaluation, TRECVID, video summarization

Citation

Over, P. , Smeaton, A. and Kelly, P. (2007), The TRECVID 2007 BBC Rushes Summarization Evaluation Pilot, 2007 ACM Multimedia Conference: TRECVID Video Summarization Workshop Proceedings, [online], https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=51190 (Accessed May 19, 2024)

Issues

If you have any questions about this publication or are having problems accessing it, please contact reflib@nist.gov.

Created November 14, 2007, Updated February 17, 2017