Abstract
In the future, robots will be used more extensively as assistants in home scenarios and must be able to acquire expertise from trainers by learning through crossmodal interaction. One promising approach is interactive reinforcement learning (IRL) where an external trainer advises an apprentice on actions to speed up the learning process. In this paper we present an IRL approach for the domestic task of cleaning a table and compare three different learning methods using simulated robots: 1) reinforcement learning (RL); 2) RL with contextual affordances to avoid failed states; and 3) the previously trained robot serving as a trainer to a second apprentice robot. We then demonstrate that the use of IRL leads to different performance with various levels of interaction and consistency of feedback. Our results show that the simulated robot completes the task with RL, although working slowly and with a low rate of success. With RL and contextual affordances fewer actions are needed and can reach higher rates of success. For good performance with IRL it is essential to consider the level of consistency of feedback since inconsistencies can cause considerable delay in the learning process. In general, we demonstrate that interactive feedback provides an advantage for the robot in most of the learning cases.
Highlights
T HERE has been considerable progress in robotics in the last years allowing robots to be successful in diverse scenarios, from industrial environments where they are nowadays established to domestic environments where their presence is still limited [1]
We introduced contextual affordances and were able to reduce the needed episode numbers to obtain a satisfactory performance in terms of the performed actions for reaching the final state
A second agent was trained using interactive reinforcement learning (IRL) and receiving feedback from a previously trained robot that sometimes showed which action to choose in a specific state
Summary
T HERE has been considerable progress in robotics in the last years allowing robots to be successful in diverse scenarios, from industrial environments where they are nowadays established to domestic environments where their presence is still limited [1]. The simulated robot has no previous knowledge on how to perform the task and it can learn only through interaction with and reward from the environment. RL is a learning approach supported by behavioral psychology where an agent uses sequential decisions to interact with its environment trying to find an optimal policy to perform a particular task. The agent selects an action to be performed reaching a new state and obtains either a reward or a punishment. Our paper is organized as follows: first, we describe the main characteristics of the IRL paradigm and different strategies to combine the RL approach with the external trainer interaction. We define our robotic agent scenario for a domestic task and describe our experimental set-ups to speed up RL with both, interactive instructions and contextual affordances. We present our main conclusions and describe future research
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callMore From: IEEE Transactions on Cognitive and Developmental Systems
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
