Enhancing Robotic Unstructured Bin-Picking Performance by Enabling Remote Human Interventions in Challenging Perception Scenarios

Published

August 24, 2016

Author(s)

Krishnanand N. Kaipa, Akshaya S. Kankanhalli-Nagendra, Nithyananda B. Kumbla, Shaurya Shriyam, Srudeep Somnaath Thevendria-Karthic, Jeremy Marvel, Satyandra K. Gupta

Abstract

We present an approach that enables a robot to initiate a call to a remote human operator and ask help in resolving automated perception system failures during bin- picking operations. Our approach allows a robot to evaluate the quality of part recognition and pose estimation, based on a confidence-measure, and thereby determine whether to proceed with the task execution or to request assistance from a human in resolving the predicted perception failure. We present an automated perception algorithm that performs the joint task of part recognition and 6 degree-of-freedom pose estimation, and has built-in features to initiate the call to the human when needed. We also present the underlying mechanism for a rationalized basis for making the call to the human. If uncertainty in part detection leads to perception failure, then human intervention is invoked. We present a new user interface that enables remote human interventions when necessary. We report results from experiments with a dual-armed Baxter robot to validate our approach.

Proceedings Title

IEEE International Conference on Automation Science and Engineering

Conference Dates

August 21-24, 2016

Conference Location

Fort Worth, TX, US

Pub Type

Conferences

Download Paper

Local Download

Keywords

Robotic bin picking, Human-robot interaction, Smart manufacturing

Citation

Kaipa, K. , Kankanhalli-Nagendra, A. , Kumbla, N. , Shriyam, S. , Thevendria-Karthic, S. , Marvel, J. and Gupta, S. (2016), Enhancing Robotic Unstructured Bin-Picking Performance by Enabling Remote Human Interventions in Challenging Perception Scenarios, IEEE International Conference on Automation Science and Engineering, Fort Worth, TX, US, [online], https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=921083 (Accessed October 11, 2025)