Preference-Based Assistance Prediction for Human-Robot Collaboration Tasks

Abstract

Human-Robot Collaboration (HRC) aims to develop robots that provide assistance to human workers while performing physical tasks. Such assistance comes in the form of supportive behaviors that are different from the actions part of the task, and that are meant to help a human worker more effectively accomplish the task. Learning how to provide useful behaviors that are tailored to a human peer represents a difficult challenge. This is due to the need of large amounts of training data in the form of real world observations that include information about such preferences. This data needs to encode not only the structure and progression of the task, but also the different workers' preferences with respect to when and what assistance the robot should provide. Our work separates the challenge of learning a model of the task (which requires a large amount of training data) from that of learning supportive behavior preferences for the interaction (which has obvious restrictions for the number of user-provided demonstrations to which we have access). We first learn a hidden Markov model (HMM) from a training set consisting of observed human workers performing the considered task in simulation. We then use this model to predict, while observing the human peer, what supportive behaviors a robot should offer throughout the task. Building upon the hidden state representation, our system is able to learn the supportive behaviors based on as few as five user-annotated demonstrations, learning a personalized supportive behavior model. We evaluate our system on a user study with 14 participants, and show results on par with human-level prediction for the task.