Carnegie Mellon University
In April 2021, Carnegie Mellon University (CMU) was awarded $1.27M for the Public Safety Innovation Accelerator Program: Augmented Reality (AR) cooperative agreement.
Public safety operations involve intensive interactions among first responders, extreme conditions, equipment, and communication, as well as incident command posts. Telemetry is a critical component to provide emergency response teams with situational-awareness about the responders, victims, targets, and environment of any given situation. However, existing hardware configurations are not built for tough environments such as, dark, smoke-filled and noisy conditions with poor wireless connectivity.
That is why CMU is proposing a modular, wearable AR platform for public safety telemetry in extreme conditions. This project supports CMU’s prior work in AR headsets for location-based services (LBS) and expands their efforts in simultaneous localization and mapping (SLAM) through work in extreme environment overlays, gesture recognition, and thermal recognition.
CMU’s proposed work has two objectives. First, the team aims to develop expanded and ruggedized AR interfaces for real-time operations that adapt to extreme environments, enhance communication, and provide the “sixth sense” to first responders. This will include adaptive Head-Up Display (HUD), haptic interfaces, intelligent thermography, and thermography-based gesture control and localization.
Second, the team will develop virtual content for Mixed Reality training with real-time IoT data streams. Artificial Intelligence (AI) technologies will be applied to create Extreme Reality (EXR) scenarios. The AR training system will provide a feedback loop for assessing and improving performance and decision-making processes.
The team plans to integrate a number of innovations, including:
The team will work with Public Safety partners in Pittsburgh Bureaus of Fire and EMS, STAT Medevac, Pittsburgh SWAT, Allegheny County Fire and Police Academy, McCandless Township Fire Department, and CMU Police Department.
CMU’s work on developing an extreme reality telemetry interface has a number of potential impacts for first responders, including: