Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.


Emergency responders risk life and limb interacting with known hazards to protect the public, rescue potential victims, and enhance the resilience of communities. Firefighters, bomb technicians, and urban search and rescue specialists typically wear only conventional personal protective equipment while dealing with a variety of extreme hazards for which remotely operated robots should be well suited. Examples include establishing situational awareness around large structure fires; disabling or dismantling improvised explosive devices (pipes, packages, vehicles); searching for survivors in collapsed or compromised structures; assessing large scale industrial or transportation accidents; or mitigating potential terror attacks using chemical, biological, or radiological sources. Responders say they want to “start remote and stay remote” when dealing with such hazards and need capable robotic systems that can be remotely operated from safe standoff distances. Many emergency response organizations already own robots but have had difficulty deploying them effectively. New ground, aerial, and aquatic robots are promising advanced capabilities but it is hard to sift through the marketing. Emergency responders need quantitative ways to measure whether a particular robot is capable and reliable enough to perform specific mission tasks. They also need ways to train and measure operator proficiency to improve very perishable operator skills.

This project will develop the measurement science and standards infrastructure necessary to evaluate the capabilities of remotely operated robotic systems, including autonomous functionalities and operator proficiency. The results of this research will help guide robot manufacturers toward developing advanced capabilities, quicken their evolution from innovation to deployment, and improve reliability. It will also support development of robot performance standards for federal, state, and local emergency responders, enabling safer and more effective tactics to strengthen the resilience of communities while reducing fatalities, injuries, and property loss.


Objective - Advance the capabilities of remotely operated emergency response robots by developing the measurement and standards infrastructure necessary to quantitatively evaluate system capabilities.

What is the new technical idea? Emergency responders need quantitative ways to measure whether a particular robot is capable and reliable enough to perform specific missions. These missions decompose into sets of elemental robot tasks that can be represented individually as standard test methods. Categories of elemental robot tasks include maneuvering, mobility, dexterity, sensing, endurance, radio communication, durability, reliability, autonomy, logistics, and safety. Each standard test method enables repeatable testing to establish statistically significant levels of reliability and confidence that the robot can perform the task. Standard test methods essentially define the test apparatuses, procedures, and performance metrics so they can be fabricated and practiced by robot manufacturers and user groups alike. They provide a tangible language to communicate responder requirements and demonstrate robot capabilities. The apparatuses typically include increasingly challenging settings to determine the maximum capability of the robot in a given task. Examples include stair inclines of 30°, 35°, 40°, 45°. Variables are controlled initially to measure baseline performance. They are then introduced incrementally to determine the effect. Examples of variables include lighted environments (<300 lux) versus dark environments (< 0.1 lux) or dry apparatuses versus wet apparatuses. A combination of 20-30 different standard test methods can be selected with appropriate apparatus settings and environmental variables to define a particular mission profile. Repeated testing within this set of standards establishes confidence that the robot can perform the overall mission. 

The project results will help inform purchasing and deployment decisions and enable specification of available combinations of robotic capabilities demonstrated to statistical significance within a set of standard test methods that represent a potential user's mission tasks. This will align emergency responder expectations prior to purchasing by fully informing them about what the robot can and cannot be expected to do in the field.  

The project deliverables will help foster innovation as robot manufacturers use the standard test methods to guide research, measure progress, and make engineering decisions regarding new sensors, coordinated manipulators, autonomous behaviors, etc. This will quicken the evolution of advanced robotic capabilities toward field deployment and improve reliability. Project deliverables will also help emergency responders improve and sustain very perishable operator skills. These standards provide inherent measures of operator proficiency that can focus training and track progress. When these standards are adopted widely for training, they can begin to isolate and measure deficiencies in equipment, including operator interfaces, and help identify technology gaps.

What is the research plan? The research plan includes three phases of development for particular mission tasks:

  1. Generate and validate a comprehensive suite of standard test methods to quantify component level and system level capabilities.
  2. Foster innovation in robotics research and manufacturing by using the standards to guide and measure advancements in robotic system capabilities using new sensors, coordinated manipulators, autonomous behaviors, etc.
  3. Apply the standards to support responder training and tactics with measures of operator proficiency.  

During the first phase, we generate and validate a series of elemental test methods to characterize robot capabilities for specific missions defined by responder organizations. Elemental test methods typically include some combination of maneuvering, mobility, dexterity, sensing, endurance, radio communication, durability, reliability, autonomy, logistics, and safety for remotely operated ground systems, small aquatic systems, and small aerial systems under 2 kg (4.4 lbs). For example, a set of standard test methods will be generated to perform missions around a burning structure to establish situational awareness, identify hazardous materials, and locate potential victims. Another set of standard test methods will include manipulator reach, strength, and dexterity to measure potential fire service robots' ability to open doors, turn valves, manipulate hose line connections, and retrieve objects. Many of the same standard test methods also apply to improvised explosive devices and other hazardous materials, and visa versa. Any given mission can provide the initial impetus to generate and validate a set of tests. The resulting standard test methods often apply to other responder missions as well.

These test methods are being standardized through the ASTM International Standards Committee on Homeland Security Applications; Operational Equipment; Robots (E54.08.01) which includes equal representation of robot manufacturers, emergency responders, and civilian/military test administrators. They are agreed upon ways to measure individual capabilities without identifying what level of performance is required by a particular user community for a given mission task. That is, they are not equipment standards that typically define the necessary outcomes of testing to meet a defined "standard." Any emergency responder organization can reference these standard test methods to specify their mission-oriented equipment or training objectives. They simply need to set thresholds within each applicable standard test method to clearly articulate their mission expectations. Operational mission descriptions specified using combinations of 20-30 standard test methods provide clear guidance to robot manufacturers. They also provide clear guidance to trainees by providing targeted levels of proficiency considered necessary by their responder organizations to be effective during deployments.

Major Accomplishments

Realized Impact

  • Adopted 30 standard and draft standard test methods by the Joint Program Office for Countering Improvised Explosive Devices (DHS, FBI, ATF, ARMY) at 4 training exercises across the U.S.
  • Adopted 8 standard and draft standard test methods by the DARPA Robotics Challenge for Disaster Response to evaluate robotic capabilities worldwide for their $100M development program focused on the Fukushima, Japan nuclear plant disaster.
  • Received Award of Merit from the ASTM International for "Outstanding Contributions and Leadership Resulting in Development of a Comprehensive Suite of Standard Test Methods to Evaluate and Improve Robot Capabilities for Homeland Security Applications.
  • Received Edward Bennett Rosa Award from the National Institute of Standards and Technology, U.S. Department of Commerce, for "Outstanding Contributions to the Development of ASTM International Standard Test Methods to Advance Emergency Response Robots."
Created April 28, 2014, Updated December 21, 2020