Using Competitions to Advance the Development of Standard Test Methods for Response Robots
Adam S. Jacoff, Raymond Sheh, Ann Virts, Tetsuya Kimura, Johannes Pellenz, Soren Schwertfeger, Jackrit Suthakorn
Competitions are an effective aid to the development and dissemination of standard test methods, especially in rapidly developing, highly varied fields such as Urban Search and Rescue robotics. By exposing the development process to highly developmental systems that push the boundaries of current capabilities, it is possible to gain an insight into how the test methods will respond to the robots of the future. The competition setting also allows for the rapid iterative refinement of the test methods and apparatuses in response to new developments. For the research community, introducing the concepts behind the test methods at the research and development stage can also help to guide their work towards the operationally relevant requirements embodied by the test methods and apparatuses. This also aids in the dissemination of the test methods themselves as teams recreate them in their own laboratories and re-use them in work outside the competition. In this paper, we discuss how international competitions, and in particular the RoboCupRescue Robot League competition, have played a crucial role in the development of standard test metrics for response robots as part of the ASTM International Committee of Homeland Security Applications; Operational Equipment; Robots (E54.08.01). We will also discuss how the competition has helped to drive a vibrant developer community towards solutions that are relevant to first responders.
2012 Performance Metrics for Intelligent Systems Workshop
, Sheh, R.
, Virts, A.
, Kimura, T.
, Pellenz, J.
, Schwertfeger, S.
and Suthakorn, J.
Using Competitions to Advance the Development of Standard Test Methods for Response Robots, 2012 Performance Metrics for Intelligent Systems Workshop, College Park, MD, US, [online], https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=910662
(Accessed August 17, 2022)