Skip to main content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

FirstSimVR: Evaluating Future Tools Using Today's VR

NEXTGEN INTERACTIONS LLC LOGO

NextGen Interactions


Next-generation first-responder tools and their interfaces have the potential to significantly enhance public safety. However, many such tools are still at an early experimental stage and are not yet ready to be used or fully tested. Even when the tools come to fruition, it can be difficult to evaluate and optimize their use in the context for which they will be deployed. To propel tool development, evaluation, and usage, we are leveraging virtual reality (VR) technologies to efficiently test early prototypes of those new tools in virtual environments that simulate the context in which they will be used. Whereas consumer VR systems can support scenarios that are quite visually and aurally realistic, most of today’s VR hardware is lacking when it comes to physical touch. This shortfall is especially critical when simulating real-world user interfaces and the real physical world first responders work in. For FirstSimVR, we focus on adding (and evaluating) realistic physical cues to VR interfaces and the environment the system is simulating. -July 2019


Quick Resources


 


Project Overview

We propose FirstSimVR, a versatile multimodal VR platform that simulates interfaces and environments to more effectively prototype, evaluate, and improve first-responder interfaces. We will target both Goal 1 (Development & Prototyping) & Goal 2 (Effectiveness & Transferability of AR/VR Simulations).

Objectives:

  1. Create a versatile framework to evaluate future first-responder interfaces by building upon SteamVR hardware/software with Unity and offering different configuration options for different scenarios.
  2. Simulate scenarios that are normally difficult to simulate by utilizing virtual reality multimodal cues such as visual effects, spatialized audio, and passive haptics.
  3. Improve simulations based on extensive feedback from first responders by hiring two first responders part time and obtaining feedback from a larger number of first-responder volunteers.
  4. Augment virtual interactions with real physical touch by using tracked and untracked physical objects that correspond to visual representations.
  5. Create an example next-generation tactile interface specific to first-responder needs by listening carefully to the needs of first responders and taking advantage of our body tracking and passive haptics.
  6. Minimize motion sickness and maximize safety by minimizing sensory conflict, maintaining high tracking accuracy, matching visual representations with physical objects, and not using cables.
  7. Integrate with existing systems for easy access to the public safety stakeholder community by having our base-level configuration only require the standard HTC Vive that is available at consumer prices.
  8. Evaluate how effectively simulating physical touch transfers to real-world first-responder tasks by collecting qualitative and quantitative data. Our final evaluations will consist of three user studies.
  9. Disseminate research findings by publishing a white paper, releasing anonymized data sets, presenting/demoing at conferences, and publishing papers.

Other Information:

Problem statement:

  • Experimental tools are not ready to be used or fully tested
  • Difficult to evaluate and optimize interfaces in the context which they will be deployed
  • Today’s best VR is designed for entertainment, not public safety
  • Current simulations are not able to effectively convey physical touch

Concept:

  • Configuration options range from a standard HTC Vive to an enhanced HTC Vive that integrates physical objects with the virtual
  • Work closely with first responders
  • Evaluation ranges from UX processes to formal user studies

Potential impact:

  • Improved evaluation of interfaces and processes
  • Increased safety through the use of better-designed tools
  • Decreased cognitive load as a result of tactile interaction
  • Better training transfer
  • Faster response to critical incidents

Deliverables:

  1. software
  2. 3D printed objects with embedded sensors
  3. documentation
  4. a white paper summarizing our findings
  5. anonymized datasets

 

Conceptual FirstSIM VR blueprint documentation - NextGen Interactions LLC
Credit: NextGen Interactions LLC

 

Touchscreen Interface - NextGen Interactions LLC
Credit: NextGen Interactions LLC
Figure 8. Since it can be important for responders to keep their hands free, we will build a touchscreen interface that goes on the non-dominant arm. A physical grid will be overlaid on top of a touchscreen so that the interface can be felt, and responders can better interact with options without necessarily having to take their eyes away from other critical details. Note the prototype we will build for this project will be a flat screen instead of a curved display as shown in this conceptual sketch, but future versions would likely contain a curved touchscreen.

<< Back to PSIAP - User Interface Page

 

Created June 20, 2018, Updated January 31, 2020