Robot Behavior Research Articles

Modifying RL Policies with Imagined Actions: How Predictable Policies Can Enable Users to Perform Novel Tasks

It is crucial that users are empowered to use the functionalities of a robot to creatively solve problems on the fly. A user who has access to a Reinforcement Learning (RL) based robot may want to use the robot's autonomy and their knowledge of its behavior to complete new tasks. One way is for the user to take control of some of the robot's action space through teleoperation while the RL policy simultaneously controls the rest. However, an out-of-the-box RL policy may not readily facilitate this. For example, a user's control may bring the robot into a failure state from the policy's perspective, causing it to act in a way the user is not familiar with, hindering the success of the user's desired task. In this work, we formalize this problem and present Imaginary Out-of-Distribution Actions, IODA, an initial algorithm for addressing that problem and empowering user's to leverage their expectation of a robot's behavior to accomplish new tasks.

Design Optimization of a Hybrid-Driven Soft Surgical Robot with Biomimetic Constraints.

The current study investigated the geometry optimization of a hybrid-driven (based on the combination of air pressure and tendon tension) soft robot for use in robot-assisted intra-bronchial intervention. Soft robots, made from compliant materials, have gained popularity for use in surgical interventions due to their dexterity and safety. The current study aimed to design a catheter-like soft robot with an improved performance by minimizing radial expansion during inflation and increasing the force exerted on targeted tissues through geometry optimization. To do so, a finite element analysis (FEA) was employed to optimize the soft robot's geometry, considering a multi-objective goal function that incorporated factors such as chamber pressures, tendon tensions, and the cross-sectional area. To accomplish this, a cylindrical soft robot with three air chambers, three tendons, and a central working channel was considered. Then, the dimensions of the soft robot, including the length of the air chambers, the diameter of the air chambers, and the offsets of the air chambers and tendon routes, were optimized to minimize the goal function in an in-plane bending scenario. To accurately simulate the behavior of the soft robot, Ecoflex 00-50 samples were tested based on ISO 7743, and a hyperplastic model was fitted on the compression test data. The FEA simulations were performed using the response surface optimization (RSO) module in ANSYS software, which iteratively explored the design space based on defined objectives and constraints. Using RSO, 45 points of experiments were generated based on the geometrical and loading constraints. During the simulations, tendon force was applied to the tip of the soft robot, while simultaneously, air pressure was applied inside the chamber. Following the optimization of the geometry, a prototype of the soft robot with the optimized values was fabricated and tested in a phantom model, mimicking simulated surgical conditions. The decreased actuation effort and radial expansion of the soft robot resulting from the optimization process have the potential to increase the performance of the manipulator. This advancement led to improved control over the soft robot while additionally minimizing unnecessary cross-sectional expansion. The study demonstrates the effectiveness of the optimization methodology for refining the soft robot's design and highlights its potential for enhancing surgical interventions.

Tactile‐Based Negotiation of Unknown Objects during Navigation in Unstructured Environments with Movable Obstacles

Traditional robot navigation passively plans/replans to avoid any contact with obstacles in the scene. This limits the obtained solutions to the collision‐free space and leads to failures if the path to the goal is obstructed. In contrast, humans actively modify their environment by repositioning objects if it assists locomotion. This article aims to bring robots closer to such abilities by providing a framework to detect and clear movable obstacles to continue navigation. The approach leverages a multimodal robot skin that provides both local proximity and tactile feedback regarding physical interactions with the surroundings. This multimodal contact feedback is employed to adapt the robot's behavior when interacting with object surfaces and regulating applied forces. This enables the robot to remove bulky obstacles from its path and solves otherwise infeasible navigation problems. The system's ability is demonstrated in simulation and real‐world scenarios involving movable and nonmovable obstacles.

A heterogeneous attention fusion mechanism for the cross-environment scene classification of the home service robot

There have been many methods to improve the capacity of scene classification of service robots. However, most of them are proposed from a technical standpoint but without reference to any cognitive principle of the brain, and furthermore, from design to evaluation, the particularity of the robot task is still not fully considered, such as cross-environment generalization, explicit semantic preservation and interpretation. Thus, the scene cognitive behavior of robots is far from humans, and their environmental adaptability is still poor. It is difficult to complete learning place concepts from discrete fragments and then continuously perceiving them with a limited view in unvisited spaces. Inspired by the recent findings from neuroscience, an attention-based global and object attribute fusion mechanism (AGOFM for short) constructed by three parts is proposed to overcome these deficiencies. In the global attribute part, a global feature extractor and a sequence context extractor are used to generate the holistic feature. The involved context integrates limited views to form an overall impression of a scene for guiding attention. In the object attribute part, a novel object vector is proposed. It simultaneously involves the detected object quantity, category and confidence information, which are all related to the vector index and high-level semantics. In the attention generation part, two sorted top-X characteristics deriving from the above two parts are fed into a fully connected (FC) network with batch normalization to generate effective attention. The attention weights are then applied to the batch normalized global and object vectors respectively, and subsequently, the two heterogeneous information are directly fused by another FC network to achieve scene classification. The policies for multi-learner fusion and frame rejection are also provided. Finally, a novel evaluation paradigm is proposed that the model is trained on a discrete prior dataset, and then the inference is tested on a traditional dataset and two robot view datasets. This simulates the cross-environment situation. Under such severe conditions, the results demonstrate that the proposed method outperforms several popular methods.

Robots’ Social Behaviors for Language Learning: A Systematic Review and Meta-Analysis

Although robots’ social behaviors are known for their capacity to facilitate learner–robot interaction for language learning, their application and effect have not been adequately explored. This study reviewed 59 empirical articles to examine the contexts and application of various social behaviors of robots for language learning, and conducted a meta-analysis of 18 study samples to evaluate the effect of robots’ social supportive behaviors on language learning achievement. Results indicate that robots’ social behaviors have mostly been applied in the studies with K–12 students, for learning vocabulary in English, including small sample sizes of below 80 participants, and lasting for one session. Second, various verbal and non-verbal behaviors of robots have been identified and applied, showing mixed results on language learning achievement. Third, robots’ social supportive behaviors have produced a positive effect on language learning achievement compared to neutral behaviors (g = 0.269). Finally, detailed suggestions for future research are discussed.

Incremental learning of humanoid robot behavior from natural interaction and large language models.

Natural-language dialog is key for an intuitive human-robot interaction. It can be used not only to express humans' intents but also to communicate instructions for improvement if a robot does not understand a command correctly. It is of great importance to let robots learn from such interaction experiences in an incremental way to allow them to improve their behaviors or avoid mistakes in the future. In this paper, we propose a system to achieve such incremental learning of complex high-level behavior from natural interaction and demonstrate its implementation on a humanoid robot. Our system deploys large language models (LLMs) for high-level orchestration of the robot's behavior based on the idea of enabling the LLM to generate Python statements in an interactive console to invoke both robot perception and action. Human instructions, environment observations, and execution results are fed back to the LLM, thus informing the generation of the next statement. Since an LLM can misunderstand (potentially ambiguous) user instructions, we introduce incremental learning from the interaction, which enables the system to learn from its mistakes. For that purpose, the LLM can call another LLM responsible for code-level improvements in the current interaction based on human feedback. Subsequently, we store the improved interaction in the robot's memory so that it can later be retrieved on semantically similar requests. We integrate the system in the robot cognitive architecture of the humanoid robot ARMAR-6 and evaluate our methods both quantitatively (in simulation) and qualitatively (in simulation and real-world) by demonstrating generalized incrementally learned knowledge.

Kinematic Analysis of Multi-Section Opposite Magnetic Catheter Robots With Solution Multiplicity

Magnetic catheter robots have great potential for application in minimally invasive surgical interventions that require submillimeter soft catheters to be inserted into the body. Compared to other medical continuum robots, the kinematic characteristics of the magnetic continuum robot are required to be improved, including operating dexterity. The opposite magnetic continuum robot (OMCR), which has oppositely-magnetized magnets on the robot, is capable of deforming in large and high-order curvatures and shows improved dexterity. However, unstable kinematic behaviors emerge owing to solution multiplicity and affect the actual operation. For example, snapping, which indicates the sudden jump from one state to another, induces danger for medical operations. In addition, the motion sequence of the external mobile magnet affects the final state of OMCR. Therefore, in this study, we first analyze the reason for these emerging behaviors by understanding the solution multiplicity of an OMCR. We then modified the OMCR kinematic model with instability analysis and additional constraints, to predict the critical point for snapping and avoid other unexpected behavior for path planning. Herein, our study aims to analyze the stability of the OMCR with relatively unstable configurations in their actuation space to ensure stable and dexterous motions for medical applications. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —The motivation of this work is to analyze, predict, and avoid the unexpected behaviors of the medical magnetic catheter robots, and ensure stable and dexterous operation. Submillimeter magnetic catheter robots have received considerable attention in the past ten years owing to their small size and dexterous motion. They have great potential for minimally invasive surgical interventions that require the navigation of “tighter corners” inside the human body. However, there may exist multiple local solutions under an external field, especially when the robot deforms in high-order curvature for dexterous operation. This situation induces unexpected behaviors and danger for medical use, such as a sudden jump from one state to another. Herein, we analyze the solution evolution as the external field varies and predict the unexpected behaviors using a modified kinematic model with instability analysis. Preliminary experiments demonstrated the effectiveness of the modified model, which can predict unexpected behaviors, and evaluate and enhance the stability during path planning. This method can be implemented for the dexterous and stable operation of magnetic catheter robots, which is significant for medical applications.

Investigation into Effect of Magnetic Moments to Swimming Behavior and Performance of the Soft Milli-Film Robots

Investigation into Effect of Magnetic Moments to Swimming Behavior and Performance of the Soft Milli-Film Robots

Automatic Temperature Parameter Tuning for Reinforcement Learning Using Path Integral Policy Improvement.

In this article, we propose a novel variant of path integral policy improvement with covariance matrix adaptation ( [Formula: see text] - [Formula: see text] ), which is a reinforcement learning (RL) algorithm that aims to optimize a parameterized policy for the continuous behavior of robots. [Formula: see text] - [Formula: see text] has a hyperparameter called the temperature parameter, and its value is critical for performance; however, little research has been conducted on it and the existing method still contains a tunable parameter, which can be critical to performance. Therefore, tuning by trial and error is necessary in the existing method. Moreover, we show that there is a problem setting that cannot be learned by the existing method. The proposed method solves both problems by automatically adjusting the temperature parameter for each update. We confirmed the effectiveness of the proposed method using numerical tests.

Simulation of robot interaction algorithms

The paper presents a study of robot interaction algorithms. The study of algorithms that control the behavior of robots is relevant due to the development of robotic means and their wide application in various fields: in manufacturing, medicine, military. The paper considers the application of nanorobot swarms in medicine. The study is performed using simulation methods. The simulation system NetLogo is used as a toolkit.

Neural dynamics of robust legged robots.

Legged robot control has improved in recent years with the rise of deep reinforcement learning, however, much of the underlying neural mechanisms remain difficult to interpret. Our aim is to leverage bio-inspired methods from computational neuroscience to better understand the neural activity of robust robot locomotion controllers. Similar to past work, we observe that terrain-based curriculum learning improves agent stability. We study the biomechanical responses and neural activity within our neural network controller by simultaneously pairing physical disturbances with targeted neural ablations. We identify an agile hip reflex that enables the robot to regain its balance and recover from lateral perturbations. Model gradients are employed to quantify the relative degree that various sensory feedback channels drive this reflexive behavior. We also find recurrent dynamics are implicated in robust behavior, and utilize sampling-based ablation methods to identify these key neurons. Our framework combines model-based and sampling-based methods for drawing causal relationships between neural network activity and robust embodied robot behavior.

The Effect of Expressive Robot Behavior on Users’ Mental Effort: A Pupillometry Study

The Effect of Expressive Robot Behavior on Users’ Mental Effort: A Pupillometry Study

Adaptive Robot Behavior Based on Human Comfort Using Reinforcement Learning

Adaptive Robot Behavior Based on Human Comfort Using Reinforcement Learning

The Art of Navigation: Pure Pursuit Controller Strategies for Four-Wheeled Mobile Robots

The Pure Pursuit Algorithm (PPA) is used in this paper to explain how a car with four wheels moves. The MATLAB environment has extensive simulation capabilities that can accurately represent complex robotic behaviors. It was these that were deployed for an extended analysis of the robot’s operational dynamics. In the MATLABSimulink framework, waypoints obtained from different algorithms define robot trajectory. An odometer sensor helped to localize the robot thus giving accurate real-time information on its position. After critically evaluating several performance indices, it became clear just how well this control algorithm worked because it smoothly moved the robot from its initial state to its target with almost no oscillations at all. The findings of the simulation confirmed that if an appropriate lookahead distance is selected then the robot can effectively track waypoints and maintain optimal path along a trajectory up until reaching the target point at last

ENHANCING HUMAN-ROBOT INTERACTION THROUGH GROUP EMOTION RECOGNITION

Abstract - This article explores within the field of Human-Robot Interaction (HRI), focusing on the complicated relationship between emotions, decision-making, and robot behaviors. Emotions are essential to effective communication and interaction, requiring the development of emotion recognition systems in robots. The article explores both individual and group emotion recognition, including microexpressions and macroexpressions. Group emotion dynamics, encompassing phenomena like emotional contagion, convergence, and social influence, are separated to understand how emotions combine within collective settings. A concept, Group Emotion Recognition (GER), is introduced, providing a framework for recognizing emotions within groups. GER involves proximity metrics, emotion classification, and entropy-based analysis to quantify emotion diversity. The article also outlines how GER can enhance user engagement, personalize interactions, improve group dynamics, and foster social acceptance in various human-robot interaction scenarios. Decision-making based on GER, driven by positive or negative emotion labels, is discussed, highlighting the adaptability and sensitivity required for effective human-robot interactions. Ethical considerations regarding the use of emotion recognition technology are addressed throughout the article, emphasizing responsible implementation. Overall, this work lays a solid foundation for advancing the field of HRI by integrating emotion recognition and decision-making to create emotionally intelligent and socially aware robots.

A complete mathematical modelling of robots actuated by epicyclic-gear

The industrial robots often use planetary gear system to have high joint torques; therefore, the influence of the rotary inertia of the number of the equally spaced planet-gears on the dynamical behavior of the robot is very important. The main objective of this paper is to develop the dynamic modeling of robot actuated by (n) equally spaced planet-gears in the case where the planet-carrier is fixed, no closed solution has been reported for this dynamic modeling, and to compare between the dynamic behavior of robot actuated by (n+1) and (n) equally spaced planet-gears for a same trajectory planning. The authors derive the explicit dynamic model for an elbow down of 2-R manipulator holding an external mass. Finally, the obtained simulation results of the dynamic modeling are verified by modeling the same robot and using an advanced simulation via SolidWorks.

Robot-Mediated Nudges for Workplace Health: Not a One-Size-Fits-All Modeling Problem

Prolonged sedentary behavior in the vast population of office and remote workers leads to increased cardiovascular and musculoskeletal health challenges, and existing solutions for encouraging breaks are either costly health coaches or notification systems that are easily ignored. A socially assistive robot (SAR) for promoting healthy workplace practices could provide the physical presence of a health coach along with the scalability of a notification system. To investigate the impact of such a system, we implemented a SAR as an alternative break-taking support solution and examined its impact on individual users’ break-taking habits over relatively long-term deployments. We conducted an initial two-month-long study (N = 7) to begin to understand the robot’s influence beyond the point of novelty, and we followed up with a week-long data collection (N = 14) to augment the dataset size. The resulting data was used to inform a robot behavior model and formulate possible methods of personalizing robot behaviors. We found that uninterrupted sitting time tended to decrease with our SAR intervention. During model formulation, we found participant responsiveness to the break-taking prompts could be classified into three archetypes and that archetype-specific adjustments to the general model led to improved system success. These results indicate that break-taking prompts are not a one-size-fits-all problem, and that even a small dataset can support model personalization for improving the success of assistive robotic systems.

What should a robot disclose about me? A study about privacy-appropriate behaviors for social robots.

For robots to become integrated into our daily environment, they must be designed to gain sufficient trust of both users and bystanders. This is in particular important for social robots including those that assume the role of a mediator, working towards positively shaping relationships and interactions between individuals. One crucial factor influencing trust is the appropriate handling of personal information. Previous research on privacy has focused on data collection, secure storage, and abstract third-party disclosure risks. However, robot mediators may face situations where the disclosure of private information about one person to another specific person appears necessary. It is not clear if, how, and to what extent robots should share private information between people. This study presents an online investigation into appropriate robotic disclosure strategies. Using a vignette design, participants were presented with written descriptions of situations where a social robot reveals personal information about its owner to support pro-social human-human interaction. Participants were asked to choose the most appropriate robot behaviors, which differed in the level of information disclosure. We aimed to explore the effects of disclosure context, such as the relationship to the other person and the information content. The findings indicate that both the information content and relationship configurations significantly influence the perception of appropriate behavior but are not the sole determinants of disclosure-adequacy perception. The results also suggest that expected benefits of disclosure and individual general privacy attitudes serve as additional influential factors. These insights can inform the design of future mediating robots, enabling them to make more privacy-appropriate decisions which could foster trust and acceptance.

Deep CNN-Based Static Modeling of Soft Robots Utilizing Absolute Nodal Coordinate Formulation.

Soft continuum robots, inspired by the adaptability and agility of natural soft-bodied organisms like octopuses and elephant trunks, present a frontier in robotics research. However, exploiting their full potential necessitates precise modeling and control for specific motion and manipulation tasks. This study introduces an innovative approach using Deep Convolutional Neural Networks (CNN) for the inverse quasi-static modeling of these robots within the Absolute Nodal Coordinate Formulation (ANCF) framework. The ANCF effectively represents the complex non-linear behavior of soft continuum robots, while the CNN-based models are optimized for computational efficiency and precision. This combination is crucial for addressing the complex inverse statics problems associated with ANCF-modeled robots. Extensive numerical experiments were conducted to assess the performance of these Deep CNN-based models, demonstrating their suitability for real-time simulation and control in statics modeling. Additionally, this study includes a detailed cross-validation experiment to identify the most effective model architecture, taking into account factors such as the number of layers, activation functions, and unit configurations. The results highlight the significant benefits of integrating Deep CNN with ANCF models, paving the way for advanced statics modeling in soft continuum robotics.

Detection of tactile-based error-related potentials (ErrPs) in human-robot interaction.

Robot learning based on implicitly extracted error detections (e.g., EEG-based error detections) has been well-investigated in human-robot interaction (HRI). In particular, the use of error-related potential (ErrP) evoked when recognizing errors is advantageous for robot learning when evaluation criteria cannot be explicitly defined, e.g., due to the complex behavior of robots. In most studies, erroneous behavior of robots were recognized visually. In some studies, visuo-tactile stimuli were used to evoke ErrPs or a tactile cue was used to indicate upcoming errors. To our knowledge, there are no studies in which ErrPs are evoked when recognizing errors only via the tactile channel. Hence, we investigated ErrPs evoked by tactile recognition of errors during HRI. In our scenario, subjects recognized errors caused by incorrect behavior of an orthosis during the execution of arm movements tactilely. EEG data from eight subjects was recorded. Subjects were asked to give a motor response to ensure error detection. Latency between the occurrence of errors and the response to errors was expected to be short. We assumed that the motor related brain activity is timely correlated with the ErrP and might be used from the classifier. To better interpret and test our results, we therefore tested ErrP detections in two additional scenarios, i.e., without motor response and with delayed motor response. In addition, we transferred three scenarios (motor response, no motor response, delayed motor response). Response times to error was short. However, high ErrP-classification performance was found for all subjects in case of motor response and no motor response condition. Further, ErrP classification performance was reduced for the transfer between motor response and delayed motor response, but not for the transfer between motor response and no motor response. We have shown that tactilely induced errors can be detected with high accuracy from brain activity. Our preliminary results suggest that also in tactile ErrPs the brain response is clear enough such that motor response is not relevant for classification. However, in future work, we will more systematically investigate tactile-based ErrP classification.