Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Enhancing Robotic Unstructured Bin-Picking Performance by Enabling Remote Human Interventions in Challenging Perception Scenarios

Published

Author(s)

Krishnanand N. Kaipa, Akshaya S. Kankanhalli-Nagendra, Nithyananda B. Kumbla, Shaurya Shriyam, Srudeep Somnaath Thevendria-Karthic, Jeremy Marvel, Satyandra K. Gupta

Abstract

We present an approach that enables a robot to initiate a call to a remote human operator and ask help in resolving automated perception system failures during bin- picking operations. Our approach allows a robot to evaluate the quality of part recognition and pose estimation, based on a confidence-measure, and thereby determine whether to proceed with the task execution or to request assistance from a human in resolving the predicted perception failure. We present an automated perception algorithm that performs the joint task of part recognition and 6 degree-of-freedom pose estimation, and has built-in features to initiate the call to the human when needed. We also present the underlying mechanism for a rationalized basis for making the call to the human. If uncertainty in part detection leads to perception failure, then human intervention is invoked. We present a new user interface that enables remote human interventions when necessary. We report results from experiments with a dual-armed Baxter robot to validate our approach.
Proceedings Title
IEEE International Conference on Automation Science and Engineering
Conference Dates
August 21-24, 2016
Conference Location
Fort Worth, TX, US

Keywords

Robotic bin picking, Human-robot interaction, Smart manufacturing

Citation

Kaipa, K. , Kankanhalli-Nagendra, A. , Kumbla, N. , Shriyam, S. , Thevendria-Karthic, S. , Marvel, J. and Gupta, S. (2016), Enhancing Robotic Unstructured Bin-Picking Performance by Enabling Remote Human Interventions in Challenging Perception Scenarios, IEEE International Conference on Automation Science and Engineering, Fort Worth, TX, US, [online], https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=921083 (Accessed April 20, 2024)
Created August 23, 2016, Updated October 12, 2021