Service skill improvement for home robots: Autonomous generation of action sequence based on reinforcement learning

Abstract

It still remains a challenge for robots to obtain knowledge automatically for performing home services. In the human learning process, natural languages act as an outline in guiding human beings complete tasks. From this point, a conditional generation method transforming textual manipulation instructions into action sequences is proposed, to provide home robots with knowledge automatically and improve the service skills finally. Due to the limited learning ability of the generation model on understanding complex semantic information, we present a two-phase conditional generation strategy in which the action space is reduced at the syntax level before generating action sequences semantically. For representing action sequences effectively, functional labels (FLs) are designed according to the requirements of performing home services, to identify six relationships about objects and actions. In action sequence generation, reinforcement learning is employed to guide the action sequence generation by introducing hierarchical rewards related to a priori knowledge, semantic similarity, and action logic. Based on statistic learning, a priori knowledge is constructed by modeling the relationship about object co-occurrence, action collaboration, and action–object correlation. The semantic similarity with Semantic Role Labeling enables the similarity evaluation between textual sentences (inputs) and produced sequences (outputs). And action logic, represented by the verb sequence in instructions, guides the production of action sequences logically. Experimental results demonstrate that the proposed method can produce competitive action sequences from textual instructions, and produced action sequences can be applied to robot for performing services.