Optimization of criterion for objective evaluation of HRI performance that approximates subjective evaluation: a case study in robot competition

Abstract

The standard approach to evaluate the performance of human-robot interaction (HRI) is subjective evaluation, for example, using questionnaires. Because such subjective evaluation is time-consuming, an alternative evaluation method based on only objective factors (i.e. human reaction behavior) is required for autonomous learning by robots and for scoring in robot competitions, etc. To this end, we aimed to investigate the extent to which subjective evaluation results can be approximated using objective factors. We observed and stored HRI history data through a robot-competition task in which the robot was required to generate comprehensible and unambiguous natural language expressions and gestures to guide inexpert users in virtual everyday environments. In addition, to acquire subjective evaluation results, we asked third-parties to evaluate the HRI performance by reviewing the stored interaction histories. From the results of a case study of robot-competition, we demonstrate the necessity for an objective method for the evaluation of HRI performance and a determination process for an effective evaluation criterion. The results of a multiple linear regression analysis to estimate the subjective evaluation results reveal that the subjective evaluation of HRI can indeed be reasonably approximated base on objective factors.