The Interactive Surveillance Event Detection (SED) evaluation is a track within the TRECVID Evaluation. The goal of the 2014 Interactive SED evaluation track is to support the development of interactive (i.e., human in the loop) technologies to detect visual events (people engaged in particular activities) in a large collection of streaming video data. The interactive aspect of SED was added in 2012. The evaluation will be implemented using an 11 hour subset of the multi-camera airport surveillance domain evaluation data collected by the Home Office Scientific Development Branch (HOSDB).
The 2014 SED evaluation is the 6th evaluation of event detection technologies preceeded by the 2008, 2009, 2010, 2011, 2012 and 2013 evaluations. The interactive evaluation methodology is similar to previous evaluation, i.e., systems report the same information and are scored with the same metrics, however, unlike previous evaluations the data set size will be smaller and each search for an event by a searcher can take no more than 25 elapsed minutes (wall clock time), measured from the time the searcher is given the event to look for until the time the result set is considered final. The system may use an unlimited amount of computional resources prior to beginning the interactive search.
We have a mailing list for announcements/discussion -- contact firstname.lastname@example.org to subscribe.
The following resources are available:
Evaluation submissions will be scored using the Framework for Detection Evaluations (F4DE) toolkit downloadable from the MIG Tools Web Page. You need at least F4DE 3.1.1 (released July 3rd 2014) to use the SED14 SubmissionChecker tool.
The dry run period for the 2014 TRECVID Interactive Surveillance Event Detection Evaluation will run until July 16th 2014. Dry run submissions will be accepted at any time during that period. The procedure for participating is as follows:
Submissions for the formal evaluation are due July 23rd, 2014. The procedure for participating is as follows: