Media Forensics Challenge Evaluation Overview Talk on ARO Sponsored Workshop on Assured Autonomy

Abstract

The Media Forensics Challenge (MFC) Evaluation is a series of annual evaluations NIST administered to support the evaluation needs of the DARPA MediFor Program from 2016-2020. The objective of MFC is to support the development of media forensic technologies – technologies that determine the region and type of manipulations in imagery (image/video data) and the phylogenic process that modified the imagery. In this talk, we will discuss the MFC evaluation design that supported a diverse set of research directions as well as our approach to evaluating systems capable of determining when a system response should be given. We present MFC evaluation schema, which includes evaluation tasks, datasets, metrics and software, and infrastructure. We also present the evaluation results and provide NIST MFC resources. The MFC “Opt-In” evaluation paradigm that sought to not penalize systems for declining to respond may be a potential performance evaluation technique for assured autonomy.