Skip to main content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.


The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Advancing the State of Machine Learning for Manufacturing Robotics

Workshop at the Robotics: Science and Systems (RSS) Conference
July 12, 2020
Corvallis, Oregon, U.S.A.

Important dates 

  • Submission deadline: May 29, 2020
  • Notification of acceptance: June 12, 2020
  • Workshop date: July 12, 2020


Machine learning approaches are fostering impressive new capabilities for robots. The number of research projects and publications are growing quite rapidly, and ML-based product spending is increasing at a compound annual growth rate of 25%. It is an exciting time, but this rapid expansion is outpacing the definition of consensus and science-based methods of assessing approaches and best practices for applying these technologies. Supporting tools, such as datasets for training and benchmarking, are becoming widely available to assist in the development of ML-based systems, but there is a severe lack of such tools for manufacturing robotics applications. 

This workshop will focus on addressing the needs of this important application domain that is significantly under-representing in research publications and support infrastructure.  The goals of this workshop are:

  • Raise awareness of the need for ML metrics, evaluations, and benchmarks, especially for manufacturing-relevant parts, operations, and environments.
  • Convene stakeholders to define common language for discussing ML performance, characteristics, applicability and/or tools and measurement science necessary to advance the state of ML in manufacturing robotics and reduce the risk of adopting ML-based technologies and solutions.
  • Produce initial document articulating challenges and gaps, ideas for directions to go for defining metrics and other measurement science to bring more rigor to the field.
  • Form an ongoing community to develop, review, try out, mature, and contribute to the concepts and tools that can help mature the field and foster well-informed, successful adoption and implementation of ML-based manufacturing robotics capabilities.

workshop structure and invited speakers

The workshop will consist of a combination of invited talks that present user and developer perspectives, a panel discussion to bring out major themes or areas of need, a poster session, and a structured discussion with general participation intended to identify the priorities going forward for forming a community to define protocols, guidelines, metrics, test methods, datasets, and tools that will be useful for maturing the application of ML to manufacturing robotics.

Speakers represent different perspectives of this ecosystem: end users of ML-based solutions, developers of tools and implementors of solutions, researchers who seek ways to leverage existing resources and to present their results based on recognized benchmarks and metrics.

This workshop will feature invited presentations by:

  • Adam Norton, University of Massachusetts Lowell, New England Robotics Validation and Experimentation (NERVE) Center
  • Dragos Margineantu, Boeing Research & Technology
  • Berk Calli, Worchester Polytechnic Institute
  • Megan Zimmerman, National Institute of Standards and Technology (NIST)
  • Nathan Ratliff, NVIDIA

Topics speakers will be asked to address, from their relevant perspectives

  • Industry perspectives on requirements to assist in evaluation and matching of solutions to implementations
  • Available resources and lessons-learned
  • Existing tools to automate dataset curation
  • How to assess the quality, applicability, and trasferability of datasets or learned models
  • Discussion on how to convene the community and create consensus metrics, common datasets, benchmarks, and tools.

Submission Instructions

We invite submissions of extended abstracts (no more than three pages single-spaced) in the RSS conference template by May 29, 2020 on topics related to the workshop focus, including but not limited to:

  • Industry perspectives on requirements to assist in evaluation and matching of solutions to implementations
  • Available resources and lessons-learned that may be applicable to manufacturing robotics
  • Available tools to automate dataset collection and curation
  • How to assess the quality, applicability, and transferability of datasets or learned models

All extended abstracts will be reviewed by the members of the organizing committee and notification of acceptance will be provided by April 16, 2020.  All accepted contributions will be presented as posters during the interactive sessions.

It is the organizers’ intention to guest edit a special issue of a journal based on the output of this workshop. Contributors may be asked to submit an extended version of their submission for inclusion in the special issue.

All submissions should be sent in PDF format to the email:

Workshop Organizers

Elena Messina, National Institute of Standards and Technology,

Holly Yanco, University of Massachusetts, Lowell,

Megan Zimmerman, National Institute of Standards and Technology,

Craig Schlenoff, National Institute of Standards and Technology,

Dragos Margineantu, Boeing Research and Technology,

For further information please contact the organizing committee at

Created February 13, 2020, Updated April 2, 2020