Robotic Tasks Research Articles

GAFAR revisited—Exploring the limits of point cloud registration on sparse subsets

Robust estimation of a rigid transformation aligning two point clouds is a fundamental building block in many tasks in computer vision and robotics, such as mapping, (re-)localization, SLAM or 3D scanning, and many more. Deep-Learning based registration methods have proven superior in terms of robustness to outliers and bad initializations, yet their resource needs often render them unsuitable for applications where space, energy and real-time constraints come into play. Since - provided a good initialization - conventional registration algorithms like ICP prove to be fast and accurate, we may relax the requirements of absolute precision of a learning-based matcher and instead focus our attention on efficiency, robustness and generalization.In prior work, we introduced our small, fast and light-weight registration algorithm GAFAR, which works on sparse subsets of point clouds and exhibits a small enough footprint to be deployed on edge devices in mobile applications, while still providing accurate results and promising generalization ability. Based thereon, we develop this idea further towards applicability on real world data with improvements to the algorithm as well as further evaluations and analyses. We showcase this by applying it to Kitti Odometry Benchmark and 3DMatch dataset as well as demonstrating its usability on edge devices. The code and trained weights are published in https://github.com/mordecaimalignatius/GAFAR/.

Mixing Corrupted Preferences for Robust and Feedback-Efficient Preference-Based Reinforcement Learning

Preference-based reinforcement learning (RL) trains agents using non-expert feedback without the need for detailed reward design. Human teacher guides the agent by comparing two behavior trajectories and labeling the preference. Although recent studies improved feedback efficiency through methods like unsupervised exploration and self- or semi-supervised learning, they often assume flawless human annotation. In practice, human teachers might make mistakes or have conflicting opinions on trajectory preferences, posing a challenge in capturing user intent. To address this, we introduce mixing corrupted preferences (MCP) for robust and feedback-efficient preference-based RL. Inspired by the robustness of mixup against corrupted labels, MCP offers three key advantages. Firstly, by component-wise mixing of two labeled preferences, MCP mitigates the impact of corrupted feedback, enhancing robustness. Secondly, MCP improves feedback efficiency by generating unlimited new data even with limited labeled feedback. Lastly, MCP helps regulate overconfidence in preference predictor, moderating excessive reward divergence between two trajectories. We evaluate our method on three locomotion and six robotic manipulation tasks in B-Pref benchmark, in contexts with both perfectly rational and imperfect teachers (including actual human teachers). Our results show that MCP significantly outperforms PEBBLE, requiring fewer feedback instances and a shorter training period, highlighting its superior feedback efficiency. Our code is available at https://github.com/JongKook-Heo/MCP.

An Enhanced Deep Q Network Algorithm for Localized Obstacle Avoidance in Indoor Robot Path Planning

Path planning is a key task in mobile robots, and the application of Deep Q Network (DQN) algorithm for mobile robot path planning has become a hotspot and challenge in current research. In order to solve the obstacle avoidance limitations faced by the DQN algorithm in indoor robot path planning, this paper proposes a solution based on an improved DQN algorithm. In view of the low learning efficiency of the DQN algorithm, the Duel DQN structure is introduced to enhance the performance and combined with a Prioritized Experience Replay (PER) mechanism to ensure the stability of the robot during the learning process. In addition, the idea of Munchausen Deep Q Network (M-DQN) is incorporated to guide the robot to learn the optimal policy more effectively. Based on the above improvements, the PER-D2MQN algorithm is proposed in this paper. In order to validate the effectiveness of the proposed algorithm, we conducted multidimensional simulation comparison experiments of the PER-D2MQN algorithm with DQN, Duel DQN, and the existing methodology PMR-DQN in the Gazebo simulation environment and examined the cumulative and average rewards for reaching the goal point, the number of convergent execution steps, and the time consumed by the robot in reaching the goal point. The simulation results show that the PER-D2MQN algorithm obtains the highest reward in both static and complex environments, exhibits the best convergence, and finds the goal point with the lowest average number of steps and the shortest elapsed time.

Autonomous Robot Goal Seeking and Collision Avoidance in the Physical World: An Automated Learning and Evaluation Framework Based on the PPO Method

Mobile robot navigation is a critical aspect of robotics, with applications spanning from service robots to industrial automation. However, navigating in complex and dynamic environments poses many challenges, such as avoiding obstacles, making decisions in real-time, and adapting to new situations. Reinforcement Learning (RL) has emerged as a promising approach to enable robots to learn navigation policies from their interactions with the environment. However, application of RL methods to real-world tasks such as mobile robot navigation, and evaluating their performance under various training–testing settings has not been sufficiently researched. In this paper, we have designed an evaluation framework that investigates the RL algorithm’s generalization capability in regard to unseen scenarios in terms of learning convergence and success rates by transferring learned policies in simulation to physical environments. To achieve this, we designed a simulated environment in Gazebo for training the robot over a high number of episodes. The training environment closely mimics the typical indoor scenarios that a mobile robot can encounter, replicating real-world challenges. For evaluation, we designed physical environments with and without unforeseen indoor scenarios. This evaluation framework outputs statistical metrics, which we then use to conduct an extensive study on a deep RL method, namely the proximal policy optimization (PPO). The results provide valuable insights into the strengths and limitations of the method for mobile robot navigation. Our experiments demonstrate that the trained model from simulations can be deployed to the previously unseen physical world with a success rate of over 88%. The insights gained from our study can assist practitioners and researchers in selecting suitable RL approaches and training–testing settings for their specific robotic navigation tasks.

OLM-SLAM: online lifelong memory system for simultaneous localization and mapping

Abstract Simultaneous Localization and Mapping is a fundamental task for robots in unknown environments. However, the poor generalization ability of learning-based algorithms in unknown environments hinders their widespread adoption. Additionally, artificial neural networks are subject to catastrophic forgetting. We propose a lifelong SLAM framework called OLM-SLAM that effectively solves the neural network catastrophic forgetting problem. To ensure the generalization of the neural network, this paper proposes a method for the sensitivity analysis of the network weight parameters. Meanwhile, inspired by human memory storage mechanisms, we design a dual memory storages mechanism that retains dynamic memory and static memory. A novel memory filtering mechanism is proposed to maximize image diversity within a fixed-size memory storage area addressing the problem of limited storage capacity of embedded devices in real-world situations. We have extensively evaluated the model on a variety of real-world datasets. Compared with CL-SLAM, the overall translation error of the test sequence is improved by 44.9%. The translation and rotation errors of Retention Ability were improved by 111.6% and 66.7%, respectively. The results demonstrate that OLM-SLAM can outperform previous methods of the same type, and OLM-SLAM has high retention ability when facing different sequences of the same type of environment.

Towards zero-shot cross-agent transfer learning via latent-space universal notice network

Despite numerous improvements regarding the sample-efficiency of Reinforcement Learning (RL) methods, learning from scratch still requires millions (even dozens of millions) of interactions with the environment to converge to a high-reward policy. This is usually because the agent has no prior information about the task and its own physical embodiment. One way to address and mitigate this data-hungriness is to use Transfer Learning (TL). In this paper, we explore TL in the context of RL with the specific purpose of transferring policies from one agent to another, even in the presence of morphology discrepancies or different state–action spaces. We propose a process to leverage past knowledge from one agent (source) to speed up or even bypass the learning phase for a different agent (target) tackling the same task. Our proposed method first leverages Variational Auto-Encoders (VAE) to learn an agent-agnostic latent space from paired, time-aligned trajectories collected on a set of agents. Then, we train a policy embedded inside the created agent-invariant latent space to solve a given task, yielding a task-module reusable by any of the agents sharing this common feature space. Through several robotic tasks and heterogeneous hardware platforms, both in simulation and on physical robots, we show the benefits of our approach in terms of improved sample-efficiency. More specifically we report zero-shot generalization in some instances, where performances after transfer are recovered instantly. In worst case scenarios, performances are retrieved after fine-tuning on the target robot for a fraction of the training cost required to train a policy with similar performances from scratch.

Towards Open-Set NLP-Based Multi-Level Planning for Robotic Tasks

This paper outlines a conceptual design for a multi-level natural language-based planning system and describes a demonstrator. The main goal of the demonstrator is to serve as a proof-of-concept by accomplishing end-to-end execution in a real-world environment, and showing a novel way of interfacing an LLM-based planner with open-set semantic maps. The target use-case is executing sequences of tabletop pick-and-place operations using an industrial robot arm and RGB-D camera. The demonstrator processes unstructured user prompts, produces high-level action plans, queries a map for object positions and grasp poses using open-set semantics, then uses the resulting outputs to parametrize and execute a sequence of action primitives. In this paper, the overall system structure, high-level planning using language models, low-level planning through action and motion primitives, as well as the implementation of two different environment modeling schemes—2.5 or fully 3-dimensional—are described in detail. The impacts of quantizing image embeddings on object recall are assessed and high-level planner performance is evaluated using a small reference scene data set. We observe that, for the simple constrained test command data set, the high-level planner is able to achieve a total success rate of 96.40%, while the semantic maps exhibit maximum recall rates of 94.69% and 92.29% for the 2.5d and 3d versions, respectively.

SCORE: Skill-Conditioned Online Reinforcement Learning

Solving complex long-horizon tasks through Reinforcement Learning (RL) from scratch presents challenges related to efficient exploration. Two common approaches to reduce complexity and enhance exploration efficiency are (i) integrating learning-from-demonstration techniques with online RL, where the prior knowledge acquired from demonstrations is used to guide exploration, refine representations, or tailor reward functions, and (ii) using representation learning to facilitate state abstraction. In this study, we present Skill-Conditioned Online REinforcement Learning (SCORE), a novel approach that leverages these two strategies and utilizes skills acquired from an unstructured demonstrations dataset in a policy gradient RL algorithm. This integration enriches the algorithm with informative input representations, improving downstream task learning and exploration efficiency. We evaluate our method on long-horizon robotic and navigation tasks and game environments, demonstrating enhancements in online RL performance compared to the baselines. Furthermore, we show our approach’s generalization capabilities and analyze its effectiveness through an ablation study.

EchoPT: A Pretrained Transformer Architecture That Predicts 2D In-Air Sonar Images for Mobile Robotics.

The predictive brain hypothesis suggests that perception can be interpreted as the process of minimizing the error between predicted perception tokens generated via an internal world model and actual sensory input tokens. When implementing working examples of this hypothesis in the context of in-air sonar, significant difficulties arise due to the sparse nature of the reflection model that governs ultrasonic sensing. Despite these challenges, creating consistent world models using sonar data is crucial for implementing predictive processing of ultrasound data in robotics. In an effort to enable robust robot behavior using ultrasound as the sole exteroceptive sensor modality, this paper introduces EchoPT (Echo-Predicting Pretrained Transformer), a pretrained transformer architecture designed to predict 2D sonar images from previous sensory data and robot ego-motion information. We detail the transformer architecture that drives EchoPT and compare the performance of our model to several state-of-the-art techniques. In addition to presenting and evaluating our EchoPT model, we demonstrate the effectiveness of this predictive perception approach in two robotic tasks.

Hydrogen-Bonded Supramolecular Network Enabled Gentle Reprogramming of Liquid Crystal Elastomer toward Evolutionary Robot.

In nature, many organisms augment chances of survival by reprogramming their structures to evolving environment, among which sea squirts being a prime example. Such reprogramming has been demonstrated in liquid crystal elastomer (LCE) actuator assembled with heat assistance. However, the required temperature being higher than the actuation temperature limits its application. Here, we reported a hydrogen-bonded supramolecular network LCE to construct soft modular and reprogrammable actuator by assembling with a gentle heat treatment. Leveraging the Michael addition reaction, we introduced hydrogen bonding to the LCE matrix with functionalized pyridine monomers. Experimental and molecular dynamics modeling proved the efficient dynamic hydrogen bond exchange at 60 °C, significantly lower than the actuating temperature of the LCE. This gave rise to the reversible and robust adhesion of the same collection of LCE modules capable of being built into different bilayers and performing various morphing upon a short thermal stimulation. Therefore, we demonstrated that these comparatively weak cross-links enabled reconfiguration of the LCE actuator. With the developed hydrogen-bonding LCEs, we built proof-of-concept modular reprogrammable robot, performing crawling, sailing, and microcircuit repair tasks. This bioinspired and efficient method for evolutionary LCE robot offers a viable path for further development of intelligent actuators sustainable in complex environments.

Hydrogen‐Bonded Supramolecular Network Enabled Gentle Reprogramming of Liquid Crystal Elastomer toward Evolutionary Robot

AbstractIn nature, many organisms augment chances of survival by reprogramming their structures to evolving environment, among which sea squirts being a prime example. Such reprogramming has been demonstrated in liquid crystal elastomer (LCE) actuator assembled with heat assistance. However, the required temperature being higher than the actuation temperature limits its application. Here, we reported a hydrogen‐bonded supramolecular network LCE to construct soft modular and reprogrammable actuator by assembling with a gentle heat treatment. Leveraging the Michael addition reaction, we introduced hydrogen bonding to the LCE matrix with functionalized pyridine monomers. Experimental and molecular dynamics modeling proved the efficient dynamic hydrogen bond exchange at 60 °C, significantly lower than the actuating temperature of the LCE. This gave rise to the reversible and robust adhesion of the same collection of LCE modules capable of being built into different bilayers and performing various morphing upon a short thermal stimulation. Therefore, we demonstrated that these comparatively weak cross‐links enabled reconfiguration of the LCE actuator. With the developed hydrogen‐bonding LCEs, we built proof‐of‐concept modular reprogrammable robot, performing crawling, sailing, and microcircuit repair tasks. This bioinspired and efficient method for evolutionary LCE robot offers a viable path for further development of intelligent actuators sustainable in complex environments.

Gender Stereotyping in Robot Service Failures

AI agents, such as service robots, could encompass gender cues. However, little is known regarding whether and how customers apply gender stereotyping to service failures in gendered service tasks performed by robots. Drawing on gender stereotype theory, we investigate the joint effects of robot gender (feminine vs. masculine) and task type (female-dominated vs. male-dominated) on customer dissatisfaction following service failures. Study 1 reveals that feminine service robots are perceived as more communal but as equally agentic as their masculine counterparts. Study 2 demonstrates that feminine (vs. masculine) service robots generate lower customer dissatisfaction when failing a female-dominated task. However, this discrepancy diminishes when failures occur in a male-dominated task. Perceived communion and tolerance serially mediate such robot gender effects. Our findings suggest that using feminine robots across all service categories may be a cost-effective strategy for hospitality organizations, eliminating the need to vary robot gender by task type.

Scene Representations for Robotic Spatial Perception

The ability of a robot to build a persistent, accurate, and actionable model of its surroundings through sensor data in a timely manner is crucial for autonomous operation. While representing the world as a point cloud might be sufficient for localization, denser scene representations are required for obstacle avoidance. On the other hand, higher-level semantic information is often crucial for breaking down the necessary steps to autonomously complete a complex task, such as cooking. So the looming question is, What is a suitable scene representation for the robotic task at hand? This survey provides a comprehensive review of key approaches and frameworks driving progress in the field of robotic spatial perception, with a particular focus on the historical evolution and current trends in representation. By categorizing scene modeling techniques into three main types—metric, metric–semantic, and metric–semantic–topological—we discuss how spatial perception frameworks are transitioning from building purely geometric models of the world to more advanced data structures incorporating higher-level concepts, such as the notion of object instances and places. Special emphasis is placed on approaches for real-time simultaneous localization and mapping, their integration with deep learning for enhanced robustness and scene understanding, and their ability to handle scene dynamicity as some of the hottest topics of interest driving robotics research today. We conclude with a discussion of ongoing challenges and future research directions in the quest to develop robust and scalable spatial perception systems suitable for long-term autonomy.

A Systematic Literature Review on Multi-Robot Task Allocation

Muti-Robot system is gaining attention and is one of the critical areas of research when it comes to robotics. Coordination among multiple robots and how different tasks are allocated to different system agents are being studied. The objective of this Systematic Literature Review (SLR) is to provide insights on the recent advancement in Multi-Robot Task Allocation (MRTA) problems emphasizing promising approaches for task allocation. In this study, we collected scientific papers from five different databases for MRTA. We outline the different approaches for task allocation algorithms, classifying them according to the methods, and emphasizing recent advances. In addition, we discuss the function of uncertainty in task allocation and typical coordination techniques utilized in task allocation to identify gaps in the literature and suggest the most promising ones.

Robotic Warehousing Operations: A Learn-Then-Optimize Approach to Large-Scale Neighborhood Search

The rapid deployment of robotics technologies requires dedicated optimization algorithms to manage large fleets of autonomous agents. This paper supports robotic parts-to-picker operations in warehousing by optimizing order–workstation assignments, item–pod assignments, and the schedule of order fulfillment at workstations. The model maximizes throughput, managing human workload at the workstations and congestion in the facility. We solve it via large-scale neighborhood search with a novel learn-then-optimize approach to subproblem generation. The algorithm relies on an off-line machine learning procedure to predict objective improvements based on subproblem features and an online optimization model to generate a new subproblem at each iteration. In collaboration with Amazon Robotics, we show that our model and algorithm generate much stronger solutions for practical problems than state-of-the-art approaches. In particular, our solution enhances the utilization of robotic fleets by coordinating robotic tasks for human operators to pick multiple items at once and by coordinating robotic routes to avoid congestion in the facility. Funding: This research was partially funded by Amazon.com Services LLC under award number 2D-12552417. Supplemental Material: The online appendix is available at https://doi.org/10.1287/ijoo.2024.0033 .

Preferences and Expectations for Home Robot Tasks: Comparison According to Age and Household Type in Republic of Korea.

Most studies of consumer preferences and expectations for home robots focus on either older adults or single-person households (SPHs). However, with the rise in voluntary SPHs among young adults and seniors, it is critical to compare both age and household types in robot research. This study explored perceptions of home robots and willingness to use their features based on age and household type, in the context of the expanding home robot market in Republic of Korea. An online survey of 400 individuals was conducted, targeting young SPHs and multi-person households (MPHs) in their 20s and 30s as well as older SPHs and MPHs in their 50s and 60s. The survey covered four robot task categories, with 40 items derived from previous research: household chores (20 items), personal care (seven items), leisure/companion (nine items), and health (four items). The results helped predict the main target groups for each in-home robot task by identifying items that showed differences in responses between groups and interpreting these based on age, household type, and their combination. The study provides valuable data on consumer expectations, highlighting differences in responses according to both age and household type, offering insights for the robotics industry to effectively target their products.

The Role of Embodiment in Learning Representations: Discovering Space Topology via Sensory-Motor Prediction

This paper explores the crucial role of embodiment in learning representations for space topology in robotics. Embodiment, the ability of an agent to interact with its environment and receive sensory feedback, is fundamental to developing accurate and efficient representations. In this work, we investigate this by applying an action-conditional prediction algorithm to data collected from a simulated environment, aiming to learn the topology of the environment through sequences of random interactions. Using a simple mobile-robot-like scenario, by leveraging sensory-motor interactions we demonstrate how the agent can discover the topology of its environment. Our results demonstrate the importance of embodiment in the development of representations and potential applicability in robotic tasks, and a simple but effective method of integrating actions into a learning loop. We suggest that building abstract representations through the use of action-conditional prediction is a step towards unification of the representations used in robotics.

Evaluation of robotic process automation tasks in large-scale environments: success rates for performance and Z scoring for non-conformity

Abstract Automation software is utilized to enhance efficiency and precision in various industries. Nonetheless, the performance and consistency of these software programs require careful monitoring through measurements. At the Finnish Government Shared Services Centre for Finance and HR (Palkeet), robotic process automation logging systems and experiments have been conducted to evaluate these measures. Success rate calculations are based on the logging system’s information and the findings (100 107 robot runs and 69 657 084 log levels during 2021-2022 and 2023/1-6). Through control charts and Z scoring, the experiments assess the proportion of non-conformity purchase invoices (987253 invoices, and 263822 defectives during 2020-2022) that the software cannot handle due to certain reasons, and the findings support an implementation of the Z scoring to observe out-of-control robot runs and analyze the causes. These results can be used to adapt and control the automation software’s processing capabilities.

Cooperative Object Transport Via Non-Contact Prehensile Pushing by Magnetic Forces

Abstract Cooperative robot systems are an essential candidate for object transportation solutions. They offer cost-efficient and flexible operation for various types of robotic tasks. The benefits of cooperative robot systems have triggered the improvement of the object transportation field. In this study, a new way of transporting objects by cooperative robots is presented. The proposed method is performed by the pushing action of the magnetic forces of the robots. The permanent magnets mounted on the mobile robots and the cart create this repelling force. The rectangular object carrier cart equipped with passive caster wheels can be manipulated on flat terrains easily and be assigned to carry different shapes of objects. Using a carrier cart has the advantage of eliminating the vertical loads on the robots. Controlling a non-contact pushing method offers a low computational burden since simple velocity and position updates are adequate for operation management. Compared with the other methods of object transportation systems, the non-contact pushing method provides a faster operation with less sensitivity to control errors. Both simulations and real-world experiments are conducted and the performances are given comparatively with a generalized frictional contact object-pushing method. The results show that the proposed method provides 10.48% faster and 20.03% more accurate object transportation compared to the frictional contact method. It is envisioned that the presented method can be a promising candidate for object transportation tasks in the industry.

Occlusion‐robust markerless surgical instrument pose estimation

AbstractThe estimation of the pose of surgical instruments is important in Robot‐assisted Minimally Invasive Surgery (RMIS) to assist surgical navigation and enable autonomous robotic task execution. The performance of current instrument pose estimation methods deteriorates significantly in the presence of partial tool visibility, occlusions, and changes in the surgical scene. In this work, a vision‐based framework is proposed for markerless estimation of the 6DoF pose of surgical instruments. To deal with partial instrument visibility, a keypoint object representation is used and stable and accurate instrument poses are computed using a PnP solver. To boost the learning process of the model under occlusion, a new mask‐based data augmentation approach has been proposed. To validate the model, a dataset for instrument pose estimation with highly accurate ground truth data has been generated using different surgical robotic instruments. The proposed network can achieve submillimeter accuracy and the experimental results verify its generalisability to different shapes of occlusion.