Meet the Team
Principle Investigator: Yang Cai
Carnegie Mellon University
This project focuses on the development of a smart helmet to assist first responders in emergency environments of fire, flood, shooting, cyber attack, and medical distress, where GPS, cellular and regular WiFi signals are not available. The hyper-reality technology is to superimpose the on-demand information to the objects in an actual scene image so that the user can see more in-depth information beyond reality. In contrast to many prevailing augmented reality technologies, our approach focuses on enhancing reality with minimal graphical and textual highlighting without obscuring the user’s view. The hyper-reality helmet contains a holographic screen that can mix the actual visible image with mapping and visualization information. The helmet contains a visible camera, a thermal imaging camera, an Android smartphone processor, voice command, and tactile interface. In order to be deployable to a broader spectrum of first responders, including firefighters, police, medical, and cyber professionals, it needs to be extremely affordable, reconfigurable and robust for different applications.
This project includes the following six tasks:
- rapid prototyping the holographic display and data processing helmet
- landmark recognition and tracking for correcting accumulated mapping errors and tagging landmarks on the map
- video simultaneous location and mapping (SLAM) and smartphone ad-hoc network (SPAN) based triangulation that map ego-motion and infrastructure components with other sensors for LBS
4. gesture and speech recognition for navigating and information retrieval
5. superimposing live thermal images to the visible views with articulated measurements, potential lasers, and case studies such as turning the invisible methane gas emission into visible objects
6. human-robot interaction interface development for 3D mapping of the infrastructure and the First-Person-View (FPV) from drone’s cameras
Benefits and Outcomes
A prototype of the hyper-reality helmet and measurable case studies in realistic environments such as an office building, airport, and paramedic emergency response situations. A system that can be integrated with available LBS technologies to enhance mapping and visualization of data on-demand. This approach can benefit not only individual responders, but also futuristic human-robot first responder forces.