ACMD Seminar: Gesture-Centric Unimodal and Multimodal Interaction in Extended Reality

 

Abstract: Dr. Ortega has been researching gesture-centric unimodal and multimodal interaction in extended reality (XR), spanning augmented and virtual reality, across applications such as virtual reality (VR) sketching, VR sculpting, and augmented reality (AR) annotations, among other domains. Understanding how interaction works within these applications is critical to lowering the barrier to entry in XR. Dr. Ortega will discuss how his approach to unimodal and multimodal interaction can serve as a bridge across multiple domains, including complex environments such as shared immersive analytics experiences. While no single set of actions will solve the problems posed by different spatial interactive experiences, the talk focuses on how microgestures, gestures, speech, and other modalities work together. The talk closes by opening a discussion: how can these gesture-centric and multimodal foundations extend to collaborative immersive analytics, where multiple people explore 3D visualizations together in CAVEs, VR, and related environments, and what would it take to support that kind of shared spatial work?

 

Bio: Dr. Francisco R. Ortega is an Associate Professor at Colorado State University and Director of the Natural User Interaction Lab, which he has led since 2018. He earned his Ph.D. in Computer Science from Florida International University, with a focus on human-computer interaction and 3D user interfaces. His work centers on gesture-centric unimodal and multimodal interaction in XR, including microgestures and gesture elicitation. In addition, his work spans XR visual search cues and notifications, XR accessibility, VR sketching, and stress reduction through VR forest bathing. He has published over 170 peer-reviewed articles and brought in more than 5.5 million dollars in external funding, with over 4.5 million as principal investigator, including an NSF CAREER award and projects funded by ONR and DARPA. He recently received a CHI 2026 Honorable Mention in XR accessibility, recognizing the top 5% of submissions.

 

Host: Simon Su

 

Note: This talk will be recorded to provide access to NIST staff and associates who could not be present to the time of the seminar. The recording will be made available in the Math channel on NISTube, which is accessible only on the NIST internal network. This recording could be released to the public through a Freedom of Information Act (FOIA) request. Do not discuss or visually present any sensitive (CUI/PII/BII) material. Ensure that no inappropriate material or any minors are contained within the background of any recording. (To facilitate this, we request that cameras of attendees are muted except when asking questions.)