A framework for tool cognition in robots without prior tool learning or observation

Abstract

Human tool use prowess distinguishes us from other animals. In many scenarios, a human is able to recognize objects seen for the first time as potential tools for a task and use them without requiring any learning. Here we propose a framework to enable similar abilities in robots. We first characterized human tools to identify a special category of tools that humans are able to use immediately through a process of skill transfer from their limbs, rather than tool learning. Motivated by the tool characterization and our neuroscientific studies on human tool use and embodiment, we then developed a tool cognition framework that enables a robot to recognize a previously unseen object as a tool for a task and plan how to grasp and wield the tool in the face of constraints and obstacles, before finally performing the task with the tool. Furthermore, the framework allows for flexibility in tool use, where the same tool can be adapted for different tasks, and different tools for the same task, all without any prior learning or observation of tool use. We demonstrate the possibilities offered by our tool cognition framework in several robot experiments with both toy and real objects as tools. Robots usually learn to use tools from direct experience or from observing the use of a tool. While knowledge can be transferred between similar tools, novel and creative use of tools is challenging. Tee and colleagues present a method where skill transfer does not come from experience of using other tools but from using the robot’s own limbs.