Step into the factory of the future. Alicia, an operations manager, sits at her workstation viewing a digitally enhanced video feed of the facility using cameras installed in strategic locations. Wearing safety gear, a maintenance engineer named Bob checks his tablet for the next machine to fix. Equipped with a headset and controllers, Dave, a software engineer at HQ, serves as a virtual tour guide for Carrie, the company’s lead executive. Wearing an augmented reality (AR) headset, Carrie surveys her machines as she walks through the facility. With Dave’s guidance, she sees digital information, such as a machine’s status, appearing in her view.
Each able to experience a virtual overlay onto a physical environment that provides more context relevant for their jobs, these co-workers can realize their potential as a team through industrial extended reality (XR), an umbrella term that encompasses a spectrum of technologies from partially immersive AR to completely immersive virtual reality.
This factory might be hard to imagine, but each technology already exists. What’s missing are standard formats, protocols and guidelines for them to work seamlessly with one another. In other words, the communication channels among these technologies remain shut.
The National Institute of Standards and Technology (NIST) is helping U.S. factories realize this vision through advanced information technology. Vital to this transformation are standards. Standards enable efficient organization of data and create a common language for machines and people to communicate. The factory of the future will require people to work closely and collaboratively with complex machines, but many unanswered questions remain. How do we pin down the precise locations of people and machines? How do we present the right information at the right time about machines to people? And, how can we do it all safely?
Leveraging industrial XR, manufacturers can present data in a spatially relevant way that takes advantage of people’s senses and natural ability to instantly recognize objects. For example, our most fond childhood memories often start with a specific time and place. For instance, one of us (Bill) will never forget sitting on his father’s shoulders during an afternoon at Disney World watching his hero Mickey Mouse lead the parade. Similarly, we can easily describe the spatial context of our workplace.
To support the pragmatic use of industrial XR, data standards play a key role. Our Product Lifecycle Data Exploration and Visualization project explores new approaches to bridge the communication gap between human and machine. Our project uses XR as a platform for presenting standard data representations to workers on the floor. Using existing standards from both manufacturing and IT communities, we develop more precise, interoperable and efficient approaches for industrial XR experiences.
Currently, our research focuses on delivering contextual views of machines for human workers by correlating their location and performance data. In doing so, we have already achieved many aspects of the activities of the stars of our story: Alicia, Bob, Carrie and Dave.
We recently published an article that demonstrates the use of standard geospatial representations to calculate the best path for a worker to travel — just as Carrie is being led through the facility by Dave.
Central to our approach is the use of standards for digitally representing geospatial layouts. Our project builds on standards published by the Open Geospatial Consortium (OGC) that provide guidelines for location-based services. An OGC standard called IndoorGML makes it possible for multiple devices to access the same digital content relative to a shared spatial context. Our approach enables the integration of a variety of technologies, including hardware such as webcams, tablets and headsets, and software such as computer vision toolkits and game engines. Note that our approach requires up-front time to create a spatial representation of a facility. Much like blueprints to a construction project, well-defined plans are critical to robust communication.
Manufacturing is just one domain that benefits from XR technologies. At NIST, researchers are leveraging XR to help solve other important problems, including understanding the spread of wildfires, enabling AR for first responders using smart helmets, and easing the training of industrial robots. Those NISTers interested in XR belong to a recently publicly announced group, called the NIST XR Community of Interest, where we share our cross-disciplinary experiences and skills. We encourage you to check out our website to learn more.
We anticipate that XR technologies will soon become a staple in modern daily life. As evidence, smartphones are already incorporating embedded hardware to make AR ubiquitous. To support the efficient and safe use of such technologies, standards are critical. XR technologies, like AR, are inherently designed to augment people’s awareness of what’s around them. Fulfilling the technological potential requires proper placement of things (both virtual and physical) and efficient communication between them. The technology will let humans fulfill their potential to do more things better and more safely.
As a member of the American Industrial Hygiene Association (AIHA), we seek ways to collaborate with other stakeholders on the future of worker health. We believe that we can provide insight into the use of digital and sensor technology that will be important on how work gets done now and in the future. We see so many opportunities to share ideas and experience to improve profitability, production, and human performance. Feel free to contact me at: firstname.lastname@example.org.
Esta muy bueno me encanta todo lo referente al trabajo detectivesco en equipo ese tipo de trabajo me encantaría hacer
Please think also on how to control access to all the stuff. If this is not adressed properly, the standard may open doors to take over control of such future factories. UMA with Oauth would do a good job in such an environment, where you have to delegate instantly also to 3rd parties access to stuff. Just a thought. happy to contribute. br jp
Thank you for your comment. We certainly agree that making XR more secure is critical for its wide adoption. Recently, members of the National Cybersecurity Center of Excellence (NCCoE) have joined the NIST XR COI. They are focused on many IT security challenges, such as control override as you suggest. Also, NIST is a current member of the Augmented Reality for Enterprise Alliance (AREA). AREA is an industrial consortium of XR technology providers and lead users. Within AREA, there is a Security Committee that tackles such issues as well. We follow them closely. In short, the NIST XR COI is tracking the latest security challenges related to XR.
Very informative. Would like to know more about extended reality.