Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.


Emergency responders risk life and limb interacting with known hazards to protect the public and rescue potential victims during disasters. Bomb technicians, police, fire fighters, and urban search-and-rescue specialists all typically wear cumbersome, but necessary personal protective equipment while dealing with a variety of extremely hazardous scenarios. Example scenarios include:

  • Disabling improvised explosive devices (e.g., pipes, packages, vehicles)
  • Establishing situational awareness around large structure fires
  • Searching for survivors in collapsed or compromised structures
  • Assessing large scale industrial or transportation accidents
  • Mitigating potential terror attacks using chemical, biological, or radiological sources

Responders want to “start remote and stay remote” when dealing with such hazards and hence, they need capable robotic systems that can be operated from safe standoff distances. New ground robots, aerial drones, and aquatic vehicles are promising advanced capabilities, but it is difficult to understand and compare specifications. Many federal, state, and local public safety organizations also already own robots but have had difficulty deploying them effectively. Emergency responders need quantitative ways to measure whether a particular robotic system is capable and reliable enough to perform their specific mission tasks. They also need ways to train and measure the proficiency of the operators and pilots to ensure safe operations. This facilitates mutual aid at large-scale disasters because the incident commanders can recognize and trust certified levels of proficiency, like a driver license for a specialized vehicle. Standard test methods are essential to solving all these issues, and this project pioneered the standard test method development process for emergency response applications.

Test methods evaluate quickly deployable autonomous and remotely operated ground robots, aerial drones, and aquatic vehicles for complex and hazardous environments.
Test methods evaluate quickly deployable autonomous and remotely operated ground robots, aerial drones, and aquatic vehicles for complex and hazardous environments.

Since 2001, a series of Presidential Policy Directives and Homeland Security Directives have emphasized three national preparedness principles: 1) an all-hazards approach, 2) a focus on capabilities, and 3) outcomes with rigorous assessments to measure and track progress in building and sustaining capabilities over time. This project applies all three principles to ground robots, aerial drones, and aquatic vehicles.


To develop the measurement science necessary to quantitatively evaluate the capabilities of autonomous and remotely operated robotic systems along with the proficiency of remote operators and pilots for emergency response applications.

Technical Idea
Emergency responder missions can be represented by a set of elemental robot tasks and formalized into standard test methods. Categories of standard test methods include maneuvering, mobility, dexterity, sensing, endurance, radio communication, durability, reliability, autonomy, logistics, and safety.

The standard test methods that this project develops  are inexpensive to replicate and simple to conduct, so anyone can use them to evaluate their own robots and proficiency.  Each standard test method enables repeatable testing to measure statistically-significant levels of reliability and confidence that the robot can perform the task. Results are comparable no matter where or when the testing occurs.

These standard test methods provide a tangible language to communicate responder requirements to manufacturers, and to understand the effectiveness of emerging technologies as they become commercial products. The apparatuses typically include increasingly challenging settings to determine the maximum capability of the robot in a given task. Variables are controlled initially to measure baseline performance. They are then introduced incrementally to measure the effects on performance. Examples of variables can be apparatus difficulty settings, lighting, visibility, etc. Repeated trials in standard test methods establish confidence in what the robot can and cannot be expected to do in the field.  

The project’s focus is on quickly deployable ground robots weighing up to 500 kg (1,000 lbs), aerial drones weighing up to 25 kg (55 lbs), and aquatic vehicles weighing up to 100 kg (200 lbs). The project also seeks to: 

  • Inspire innovation in robot designs, operator interfaces, and autonomous behaviors. 
  • Increase the competitiveness of commercial robot manufacturers.
  • Improve the effectiveness and safety of emergency responders performing extremely hazardous tasks in complex environments.

Research Plan
The project pioneered an effective standards development process for autonomous and remotely operated robotic systems that includes the following: 

  • Requirements workshops to identify user gaps and priorities.
  •  Validation exercises to refine test apparatuses and procedures.
  • Robot evaluations to capture quantitative capabilities data.
  • Research competitions to inspire innovation, measure progress, and compare results.
  • Standards committees to publish and disseminate the resulting test methods.
Test method validation exercises and robot competitions
Project stakeholders extend across the U.S. and internationally and test method validation exercises and robot competitions are hosted as public events around the world.

The research plan includes three phases of test method development:

Generate and validate comprehensive suites of standard test methods to quantify key capabilities. This includes a series of elemental test methods to characterize robot capabilities for specific missions defined by responder organizations.

Quantitative results from different standard terrains
Quantitative results from different standard terrains evaluate overall system performance, design trade-offs for different conditions, and remote operator proficiency.

Foster innovation in robotics research and manufacturing by using the test methods to guide and measure advancements in robotic system capabilities. These activities center around international robotics competitions using tests developed by the project as challenge tasks. These events typically result in more than a hundred test trials in less than a week and help disseminate the tests to participating teams and interested audiences. 

Incremental Difficulty of Terrains and Obstacles
The Autonomous Quadruped Robot Challenge (AQRC) at the 2023 IEEE International Conference on Robotics and Automation is the newest robot competition focused on four-legged robots or “quadrupeds.”


2023 RoboCup Rescue International Championship
The 2023 RoboCup Rescue International Championship was the latest competition in a series that has been held in different countries each year since 2000, with several regional opens per year. These competitions use 20 standard ground robot test methods developed by this project as challenge tasks.

Disseminate the standards to support responder purchasing, training, and credentialing. The test methods are being standardized through the ASTM International Standards Committee on Homeland Security Applications; Response Robots (E54.09). The tests provide objective ways to measure individual robot capabilities without identifying what level of performance is required by a particular user community for a given mission task. Any emergency responder organization can use these tests to specify their mission-oriented equipment or training objectives. Mission descriptions specified using combinations of 10-20 standard test methods provide clear guidance to robot manufacturers about engineering trade-offs. Quantitative scores also guide trainees toward levels of proficiency considered necessary by their responder organizations to be effective during deployments. Over 500 people have participated in test method validation exercises across the U.S. to date. These include federal, state, and local public safety organizations, civilian and military test facilities, commercial manufacturers, and others. The project also collaborates with more than a dozen organizations around the world that use the tests. 

Major Accomplishments
To date, the project has developed more than 50 test methods for ground robots, aerial drones, and aquatic vehicles. More than 20 test methods have been standardized so far, the rest are being validated with manufacturers, emergency responders, and soldiers prior to balloting. Draft-standard test methods can also help coordinate innovation, commercialization, and deployment of quickly emerging technologies.

Published Standard Test Methods for Evaluating Ground Response Robots


E2521M-23 Standard Terminology for Evaluating Response Robot Capabilities


E3132M-17 System Configuration

E2592M-16 Packaging for Urban Search and Rescue Task Force Equipment Caches


E2829M-20 Sustained Speed

E2830M-20 Tasks with Grasped Sleds

E3310M-22 Align Ground Contacts with Parallel Rails

Mobility: Reproducible Terrains

E2826M-20 Continuous Pitch/Roll Ramps

E2827M-20 Crossing Pitch/Roll Ramps

E2828M-20 Symmetric Stepfields

E2991M-17 Gravel

E2992M-17 Sand

E3349M-22 K-Rails

Mobility: Confined Area Obstacles (variable)

E2801M-20 Variable Gap Lengths

E2802M-21 Variable Height Hurdles

E2803M-20 Variable Incline Planes

E2804M-20 Variable Stairs/Landings

E3311M-22 Variable Height Rail Obstacles


E3408M-23 Linear Inspection

Situational Awareness

E2853M-22 Search Tasks


E2566M-17 Visual Acuity


E3380M-23 Endurance Using Reproducible Terrains

Radio Communications

E2854M-21 Line-of-Sight Range

E2854M-21 Non-Line-of-Sight Range

Published Standard Test Methods for Evaluating Aerial Drones


E3426M-24 Endurance

Draft Test Methods for Evaluating Aerial Drones 
A series of tests for measuring 5 levels of system capability and remote pilot proficiency in increasingly difficult environments: Open, Obstructed, and Confined (indoors). These test methods enable remote pilot credentialing that can help ensure safety (e.g., when flying in the national airspace). The same test lanes used during dark conditions evaluate system performance and pilot proficiency for night operations.

Draft Test Methods for Evaluating Aquatic Vehicles
The project has developed a series of aquatic vehicle tests that can be floating on the surface, placed on the bottom, or omnidirectional. They are typically initially deployed in clear water tanks or pools to limit the variables before adding in each variable purposely to measure performance degradation. Examples include lighted/dark environments, clear/opaque water, placid/turbulent flow.

Created April 28, 2014, Updated May 6, 2024