It Takes a (Worldwide) Village to Improve Drone Technology for First Responders

Every day we take for granted that we carry a small computer with us everywhere, an appendage to our body, that helps us with tasks such as finding our way to a new address, communicating with doctors, and buying goods from around the world. Mobile devices are so critical to our personal lives that most people will return home if they leave their phone behind but many will not return home for their wallet. Our dependence on voice and data communications, and the related technologies, has grown at breakneck speeds from the days of the wired telephone. 

Today, not only can we use our wireless phones to make a call or watch high-bandwidth movies, but we can do it all on the same hand-held device! And now we can even put those same wireless voice and data services on a drone and fly it through a building looking for lost people. We’re starting to see the endless applications that are driving telecom technology, and it makes me wonder: What’s next? 

What Led Me to NIST

My career began in a computer lab as an engineering student at the University of Colorado (CU) Boulder, where writing “code” meant punching cards on a punch card machine and loading them one by one into a computer the size of a tiny home. The internet was limited to higher education, and dial-up internet access was not widely available. By the time I graduated college and took my first job at New York Telephone, I experienced the thrill of having a computer at my desk (even though it was a shared computer connected only to a mainframe). How we communicated was being transformed, and businesses were operating with near-instant information. 

I’ve been immersed in the wired and wireless industry for 35 years. Most of my career was spent at Sprint, and then Sprint Nextel, where I first worked with first responders and learned their operational needs. My job became more meaningful as I saw my technical skills contribute to their mission. First responders are the lifeline of our communities. They need different tools and communications devices to help them save property, the lives of their neighbors, and their own lives. Their work is more than a job.

After Sprint Nextel, I spent a few years working in CU’s graduate telecom program before joining NIST. The CU job allowed me to explore my interest in education and research while developing partnerships with industry. The NIST Public Safety Communications Research (PSCR) division was one of those partners.

NIST Implements Prize Challenges for Research

When I joined NIST, I followed my heart to (once again) focus on the needs of first responders. Using an interdisciplinary approach I learned at CU, I support the broadband communications research we perform within NIST’s PSCR division by asking probing questions and using open innovation techniques, such as prize challenges. Our goal with these challenges is to help industry and academia expand the marketplace for first responder communications. 

My specific work has focused on a passion I discovered after starting here: uncrewed aircraft systems (UAS) technology. Specifically, I investigate the unique use cases related to how first responders use UAS in their daily operations. The UAS Program and Open Innovation team has used prize challenges, or competitions, to expand our program and help us find new technology. When I first joined NIST, crowdsourcing solutions from around the world was a relatively new idea, but it is now a proven method in supporting our goals. Challenge participants are energized by receiving cash prizes, and, in turn, NIST receives scientific data, promotes economic development, and develops new research partnerships. 

What First Responders Need

To start a UAS program at a public safety agency requires piloting skills and money. It also needs to be a proven technology with high levels of accuracy and reliability to instill confidence and guarantees for the success of the mission. First responders can’t afford less. Today’s first responders mostly use commercial-off-the-shelf (COTS) drones with preloaded camera and video technology. They look for a low-cost drone that’s rich in features and durability, easy to operate, and secure to perform specialized tasks. 

However, COTS drones can be limiting. For example, first responders working in heavily forested areas need better tools for locating missing persons during a search and rescue (SAR) operation. Peering through thick foliage or other opaque materials can delay the time it takes to complete a rescue mission, so they need different types of cameras, data analytics, sensors or other adjunct devices to enhance the image detection. Confirming which of these technologies would work best became the goal of a recent prize challenge called the First Responder UAS Triple Challenge 3.1: FastFind.

2021 First Responder UAS Triple Challenge 3.1: FastFind Video

In Challenge 3.1, we asked competitors to build a low-cost UAS that could positively identify a person quickly. Thus, the UAS could provide the incident commander with GPS coordinates and avoid wasted time and effort searching in the wrong area, ultimately giving the response team a better chance of finding the lost person before harm occurs.

The top winners of Challenge 3.1 were academic teams. The first-place team was a group from the University of Maryland that used a combination of a COTS drone, both thermal and RGB (red, green, blue) cameras, and data analysis. The UAS livestreamed the target images to the drone operator, who processed the images in real time on the ground. The team successfully located multiple people using a thermal camera, processed the images using artificial intelligence (AI) and machine learning (ML) techniques, then returned the GPS coordinates to the operator for a positive identification of the lost person. 

The final head-to-head competition took place in a forest near Mississippi State University in Starkville, Mississippi, where teams flew their aircraft in hot and humid conditions. The weather conditions weren’t ideal for the competitors, who had difficulty with their UAS and camera equipment. However, it highlighted the fact that first responders operate SAR missions in tough conditions every day, and it was useful for NIST and competing teams to capture data outside of a comfortable research facility. One issue that impacted all the final teams that used thermal cameras was their ability to distinguish between items on the ground versus people, since the ambient temperature was similar to the ground temperature. The winning team overcame this obstacle by using both thermal and RGB cameras with its AI-trained UAS. 

The Research Continues Because the Need Continues 

After five years of studying UAS through prize challenges, we’ve been encouraged with the technical results and their impact on communications and situational awareness in the first responder community. For example, the second UAS prize challenge in 2020 attempted to improve flight time while carrying a payload. We saw times improve from 18 minutes to more than 112 minutes from the first UAS challenge in 2018. After the finals of Challenge 3.1, one of the attending first responders said they wished they could buy the winning solution because it was exactly what they needed for their toolbox. Similarly, in Challenge 3.2, an industry professional asked the winning contestant to join in future conversations and expound upon their solution. These are the outcomes we hope for.

Another important outcome from these challenges is increased research. The academic team members from UMD who finished first in Challenge 3.1 told us it was the first time they brought together this particular group of students to collaborate in an interdisciplinary fashion. As a result, each student has advanced in their individual research by combining techniques to create the design, build the prototype, and fly the drone in the challenge. As one competitor stated, “Without NIST’s investment, there wouldn’t be any incentive for researchers to focus on drone features that are unique to first responders. There’s no commercial incentive to do this for public safety today.”

Drones are already being used to deploy communications services, but what’s next? Where will our research take us in our mission to help first responders? It’s hard to predict, but, for now, I take comfort in knowing that we are innovating to help first responders. Our impact on drone technology may someday save me … or YOU!

For a description of all three UAS challenges in 2021-2022 and the winner profiles in the First Responder UAS Triple Challenge, visit us here.