Curious Robot Research Articles

Expressive Cognitive Architecture for a Curious Social Robot

Artificial curiosity, based on developmental psychology concepts wherein an agent attempts to maximize its learning progress, has gained much attention in recent years. Similarly, social robots are slowly integrating into our daily lives, in schools, factories, and in our homes. In this contribution, we integrate recent advances in artificial curiosity and social robots into a single expressive cognitive architecture. It is composed of artificial curiosity and social expressivity modules and their unique link, i.e., the robot verbally and non-verbally communicates its internally estimated learning progress, or learnability, to its human companion. We implemented this architecture in an interaction where a fully autonomous robot took turns with a child trying to select and solve tangram puzzles on a tablet. During the curious robot’s turn, it selected its estimated most learnable tangram to play, communicated its selection to the child, and then attempted at solving it. We validated the implemented architecture and showed that the robot learned, estimated its learnability, and improved when its selection was based on its learnability estimation. Moreover, we ran a comparison study between curious and non-curious robots, and showed that the robot’s curiosity-based behavior influenced the child’s selections. Based on the artificial curiosity module of the robot, we have formulated an equation that estimates each child’s moment-by-moment curiosity based on their selections. This analysis revealed an overall significant decrease in estimated curiosity during the interaction. However, this drop in estimated curiosity was significantly larger with the non-curious robot, compared to the curious one. These results suggest that the new architecture is a promising new approach to integrate state-of-the-art curiosity-based algorithms to the growing field of social robots.

Experiments in Artificial Sociality

In this article, I explore how experiments with social robots enact and reconfigure more-than-human forms of sociality. I combine recent anthropological discussions of nonhuman sociality with Andy Pickering’s work on dances of agency (1993, 1995) and John Law’s method assemblages (2004) to show how human-robot interaction experiments enact open-ended and decentred configurations of entangling relations between humans and robots. I propose the concept of artificial sociality to capture both the ongoing enactments and multiple results of such experimental reconfigurations. Using these conceptual tools, I unpack the “curious robot experiment” from my ethnographic fieldwork in a Japanese robotics laboratory and compare the kinds of sociality produced in the two experimental conditions. I argue that the curious robot exemplifies what Pickering calls technologies of engagement (2018) by manifesting a form of artificial sociality that augments the unpredictability of dances of agency enacted in (re)configurations of entangling relations.

Curiosity Did Not Kill the Robot

Learning from human interaction data is a promising approach for developing robot interaction logic, but behaviors learned only from offline data simply represent the most frequent interaction patterns in the training data, without any adaptation for individual differences. We developed a robot that incorporates both data-driven and interactive learning. Our robot first learns high-level dialog and spatial behavior patterns from offline examples of human--human interaction. Then, during live interactions, it chooses among appropriate actions according to its curiosity about the customer's expected behavior, continually updating its predictive model to learn and adapt to each individual. In a user study, we found that participants thought the curious robot was significantly more humanlike with respect to repetitiveness and diversity of behavior, more interesting, and better overall in comparison to a non-curious robot.

Intrinsically motivated model learning for developing curious robots

Reinforcement Learning (RL) agents are typically deployed to learn a specific, concrete task based on a pre-defined reward function. However, in some cases an agent may be able to gain experience in the domain prior to being given a task. In such cases, intrinsic motivation can be used to enable the agent to learn a useful model of the environment that is likely to help it learn its eventual tasks more efficiently. This paradigm fits robots particularly well, as they need to learn about their own dynamics and affordances which can be applied to many different tasks. This article presents the texplore with Variance-And-Novelty-Intrinsic-Rewards algorithm (texplore-vanir), an intrinsically motivated model-based RL algorithm. The algorithm learns models of the transition dynamics of a domain using random forests. It calculates two different intrinsic motivations from this model: one to explore where the model is uncertain, and one to acquire novel experiences that the model has not yet been trained on. This article presents experiments demonstrating that the combination of these two intrinsic rewards enables the algorithm to learn an accurate model of a domain with no external rewards and that the learned model can be used afterward to perform tasks in the domain. While learning the model, the agent explores the domain in a developing and curious way, progressively learning more complex skills. In addition, the experiments show that combining the agent's intrinsic rewards with external task rewards enables the agent to learn faster than using external rewards alone. We also present results demonstrating the applicability of this approach to learning on robots.

A Curious Robot Vision System

Event Abstract Back to Event A Curious Robot Vision System Pramod Chandrashekhariah1 and Jochen Triesch1* 1 Frankfurt Institute for Advanced Studies (FIAS), Germany The ability of biological organisms to learn autonomously is a challenge for future robots assisting humans in their homes, work places, or other changing environments [1-3]. Autonomous learning implies that the system can choose what activities it should engage in and hence what it can learn about. Here we introduce an autonomously learning robot vision system implemented on the iCub robot that explores a scene and learns visual object representations without any human assistance. The system integrates a number of visual competences including attention, stereoscopic vision, segmentation, tracking, model learning, and recognition. Our system is driven by an attention mechanism that endows it with a simple form of curiosity: the system will preferentially attend to objects, for which it estimates that it can still learn something new. The basic mode of operation of the system is as follows. An attention mechanism combines bottom-up (saliency based) attention with top-down information about object familiarity and previously visited locations. It drives the system to look at interesting points in the scene. At each location, an object candidate is segmented from the background using stereo information and actively tracked if it is moving. A new object model, containing information about the spatial arrangement of local image features, is created if the object is novel. Otherwise the existing model for the object is updated. The system continues to look at and learn about an object as long as the object model can still be improved by acquiring additional features from the object. When the learning progress for the current object drops below a threshold, attention is diverted to a new location, where the process repeats. Our experiments comparing our “curious” attention system to several alternatives show greater learning speed and higher recognition accuracy when learning is focused on objects and locations where the learning progress is expected to be high. Acknowledgements Supported by the BMBF through the Bernstein Focus: Neurotechnology Frankfurt.

The Curious Robot as a Case Study for Comparing Dialogue Systems

Modeling interaction with robots raises new and different challenges for dialogue modeling than traditional dialogue modeling with less‐embodied machines. We present four case studies of implementing a typical human‐robot interaction (HRI) scenario with different state‐of‐the‐art dialogue frameworks in order to identify challenges and pitfalls specific to HRI and potential solutions. The results are discussed with a special focus on the interplay between dialogue and task modeling on robots.

Developmental robotics, optimal artificial curiosity, creativity, music, and the fine arts

Even in the absence of external reward, babies and scientists and others explore their world. Using some sort of adaptive predictive world model, they improve their ability to answer questions such as what happens if I do this or that? They lose interest in both the predictable things and those predicted to remain unpredictable despite some effort. One can design curious robots that do the same. The author’s basic idea (1990, 1991) for doing so is a reinforcement learning (RL) controller is rewarded for action sequences that improve the predictor. Here, this idea is revisited in the context of recent results on optimal predictors and optimal RL machines. Several new variants of the basic principle are proposed. Finally, it is pointed out how the fine arts can be formally understood as a consequence of the principle: given some subjective observer, great works of art and music yield observation histories exhibiting more novel, previously unknown compressibility/regularity/predictability (with respect to the observer’s particular learning algorithm) than lesser works, thus deepening the observer’s understanding of the world and what is possible in it.