Skip to main content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Test Methods and Metrics for Effective HRI in Collaborative Human-Robot Teams

Human Awareness Evaluations

Megan Zimmerman from NIST performs metrology tests of a human-aware vision system for robot situational awareness.

The need for effective human-robot interaction (HRI) continues to present challenges for the field of robotics.  As new technologies are integrated into human-robot teams in a myriad of application domains, exposures to and expectations from robots are growing rapidly.  A key factor that limits the success of human-robot teams is the lack of principled metrics for assessing the effectiveness of HRI.

The necessity for validated test methods and metrics for human-robot teaming is driven by the desire for repeatable and consistent evaluations of HRI methodologies.  Such evaluations are critical for advancing underlying models of HRI, as well as establishing traceable mechanisms for vendors and consumers of HRI technologies to assess and assure functionality.

This full-day workshop will address the issues surrounding the development of test methods and metrics for evaluating the performance of human-robot teams across the multitude of human-centered application domains, including industrial, social, medical, field and service robotics.  This workshop will focus on establishing the diversity of approaches for addressing HRI metrology in different robotic domains, and identifying the underlying issues of traceability, objective repeatability, and transparency in HRI metrology.

The workshop organizers are seeking full paper submissions for presentation.  Papers should be limited to 6 pages in the IEEE Proceedings format.  Accepted papers will also be invited to submit their papers to a special journal issue to be published at a later date.  Extended abstracts and posters will also be accepted for consideration for presentation during a poster session.

Solicited paper topics include:

  • Test methods and metrics for evaluating human-robot teams
  • Best practices and real-world case studies in human-robot teaming
  • HRI data set generation, formatting, and dissemination for human-robot teams
  • Verification and validation of HRI studies involving human-robot teams
  • Benchmarking performance of human-robot teams
  • Evaluation of novel human-robot team designs and methods

Important Dates:

  • Paper submission deadline: Friday, 8 February, 2019
  • Notice of acceptance for presentation: Friday, 15 February, 2019
  • Submission of posters and extended abstracts for presentation: Friday, 8 February, 2019
  • Notice of acceptance of posters for presentation: Friday, 15 February, 2019
  • Workshop date: Monday, 11 March, 2019

Organizers:

  • jeremy.marvel [at] nist.gov (Dr. Jeremy A. Marvel), National Institute of Standards and Technology (NIST), USA
  • shelly.bagchi [at] nist.gov (Shelly Bagchi), National Institute of Standards and Technology (NIST), USA
  • megan.zimmerman [at] nist.gov (Megan Zimmerman), National Institute of Standards and Technology (NIST), USA
  • murat.aksu [at] nist.gov (Murat Aksu), National Institute of Standards and Technology (NIST), USA
  • brian.antonishek [at] nist.gov (Brian Antonishek), National Institute of Standards and Technology (NIST), USA
  • yue6 [at] g.clemson.edu (Dr. Yue Wang), Clemson University
  • ross [at] semio.ai (Dr. Ross Mead), Semio
  • terry.fong [at] nasa.gov (Dr. Terry Fong), National Aeronautics and Space Administration (NASA), USA
  • hbenamor [at] asu.edu (Dr. Heni Ben Amor), Arizona State University

Sponsors

Arizona State University Logo
Arizona State University

Clemson University
Clemson University

NASA Logo
NASA

Semio
Semio

Time

Topic

Details

0900 – 0915

Welcome and Introduction

Megan Zimmerman

0915 – 1000

Keynote

Dr. Laurel Riek

1000 – 1030

Contributor Presentations

Malte Jung

Kathleen Belhassein & Guilhem Buisan

1030 – 1050

Break

 

1050 – 1135

Invited Talk

Tathagata Chakraborti

1135 – 1230

Contributor Presentations

Raquel Oliveira

Dan Szafir

José Lopes

1230 – 1300

Recap and Discussion

All

1300 – 1400

Lunch Break

 

1400 – 1445

Panelist Presentations

Yun Kyung Kim

Chung Hyuk Park

Megan Zimmerman

Katrin Lohan

1445 - 1530

Metrics Panel Discussion

Moderator:  Murat Aksu

1530 – 1550

Break

 

1550 – 1605

Standardizations

Shelly Bagchi

1605 – 1650

Invited Talk

Dr. Seungbin Moon

1650 – 1730

Open Discussion, Recap, and Next Steps

All

1730

Close

 

 

Remote participants:

For those who wish to participate, but are unable to attend in person, a Google Meet has been scheduled to live-stream the workshop.

Link: https://meet.google.com/afc-nvxk-eof

‎Dial In: (‪US)‪+1 559-854-1451

PIN: ‪630 283 416#

 

Click here to add the stream to your calendar.

Created December 12, 2018, Updated March 8, 2019