Robotics Challenge Research Articles

Integrating Historical Learning and Multi-View Attention with Hierarchical Feature Fusion for Robotic Manipulation.

Humans typically make decisions based on past experiences and observations, while in the field of robotic manipulation, the robot's action prediction often relies solely on current observations, which tends to make robots overlook environmental changes or become ineffective when current observations are suboptimal. To address this pivotal challenge in robotics, inspired by human cognitive processes, we propose our method which integrates historical learning and multi-view attention to improve the performance of robotic manipulation. Based on a spatio-temporal attention mechanism, our method not only combines observations from current and past steps but also integrates historical actions to better perceive changes in robots' behaviours and their impacts on the environment. We also employ a mutual information-based multi-view attention module to automatically focus on valuable perspectives, thereby incorporating more effective information for decision-making. Furthermore, inspired by human visual system which processes both global context and local texture details, we have devised a method that merges semantic and texture features, aiding robots in understanding the task and enhancing their capability to handle fine-grained tasks. Extensive experiments in RLBench and real-world scenarios demonstrate that our method effectively handles various tasks and exhibits notable robustness and adaptability.

A Comprehensive Review of Mobile Robot Navigation Using Deep Reinforcement Learning Algorithms in Crowded Environments

Navigation is a crucial challenge for mobile robots. Currently, deep reinforcement learning has attracted considerable attention and has witnessed substantial development owing to its robust performance and learning capabilities in real-world scenarios. Scientists leverage the advantages of deep neural networks, such as long short-term memory, recurrent neural networks, and convolutional neural networks, to integrate them into mobile robot navigation based on deep reinforcement learning. This integration aims to enhance the robot's motion control performance in both static and dynamic environments. This paper illustrates a comprehensive survey of deep reinforcement learning methods applied to mobile robot navigation systems in crowded environments, exploring various navigation frameworks based on deep reinforcement learning and their benefits over traditional simultaneous localization and mapping-based frameworks. Subsequently, we comprehensively compare and analyze the relationships and differences among three types of navigation: autonomous-based navigation, navigation based on simultaneous localization and mapping, and planning-based navigation. Moreover, the crowded environment includes static, dynamic, and a combination of obstacles in different typical application scenarios. Finally, we offer insights into the evolution of navigation based on deep reinforcement learning, addressing the problems and providing potential solutions associated with this emerging field.

Robotics in Disaster Management: AI-Driven Rescue, Aerial Mapping, and Emerging IoT Applications for Effective Response

This review paper explores the growing role of robotics in disaster management, focusing on the integration of artificial intelligence (AI), Internet of Things (IoT) applications, and advanced simulation environments. Highlighting lessons from events like the Fukushima nuclear disaster and Wenchuan earthquake, it examines the deployment of aerial, ground, and underwater robots in tasks such as data collection, search and rescue, and hazard monitoring. AIoT frameworks enable these robotic systems to classify objects and make decisions on-site, supporting timely and efficient responses in hazardous conditions. Virtual platforms like the DARPA Virtual Robotics Challenge, using simulators such as Gazebo and CloudSim, facilitate realistic testing of robotic capabilities in disaster scenarios, advancing operational readiness through virtual training. This review synthesizes recent progress in AI-driven disaster robotics, identifies challenges, and discusses potential improvements for future applications.

Adaptive FPGA-Based Accelerators for Human-Robot Interaction in Indoor Environments.

This study addresses the challenges of human-robot interactions in real-time environments with adaptive field-programmable gate array (FPGA)-based accelerators. Predicting human posture in indoor environments in confined areas is a significant challenge for service robots. The proposed approach works on two levels: the estimation of human location and the robot's intention to serve based on the human's location at static and adaptive positions. This paper presents three methodologies to address these challenges: binary classification to analyze static and adaptive postures for human localization in indoor environments using the sensor fusion method, adaptive Simultaneous Localization and Mapping (SLAM) for the robot to deliver the task, and human-robot implicit communication. VLSI hardware schemes are developed for the proposed method. Initially, the control unit processes real-time sensor data through PIR sensors and multiple ultrasonic sensors to analyze the human posture. Subsequently, static and adaptive human posture data are communicated to the robot via Wi-Fi. Finally, the robot performs services for humans using an adaptive SLAM-based triangulation navigation method. The experimental validation was conducted in a hospital environment. The proposed algorithms were coded in Verilog HDL, simulated, and synthesized using VIVADO 2017.3. A Zed-board-based FPGA Xilinx board was used for experimental validation.

Towards a Unified Framework for Software-Hardware Integration in Evolutionary Robotics

The discrepancy between simulated and hardware experiments, the reality gap, is a challenge in evolutionary robotics. While strategies have been proposed to address this gap in fixed-body robots, they are not viable when dealing with populations and generations where the body is in constant change. The continual evolution of body designs necessitates the manufacturing of new robotic structures, a process that can be time-consuming if carried out manually. Moreover, the increased manufacturing time not only prolongs hardware experimental durations but also disrupts the synergy between hardware and simulated experiments. Failure to effectively manage these challenges could impede the implementation of evolutionary robotics in real-life environments. The Autonomous Robot Evolution project presents a framework to tackle these challenges through a case study. This paper describes the main three contributions of this work: Firstly, it analyses the different reality gap experienced by each different robot or the heterogenous reality gap. Secondly, it emphasizes the importance of automation in robot manufacturing. And thirdly, it highlights the necessity of a framework to orchestrate the synergy between simulated and hardware experiments. In the long term, integrating these contributions into evolutionary robotics is envisioned to enable the continuous production of robots in real-world environments.

Advancements in amphibious robot navigation through wheeled odometer uncertainty extension and distributed information fusion

The advancement and safeguarding of the water-land interface region is of paramount importance, and amphibious robots with the capacity for autonomous operation can play a pivotal role in this domain. However, the inability of the majority of reliable navigation sensors to adapt to the water-land interface environment presents a significant challenge for amphibious robots, as obtaining positional information is crucial for autonomous operation. To address this issue, we have proposed a positioning and navigation framework, designated as NAWR (Navigation Algorithm for Amphibious Wheeled Robots), with the objective of enhancing the navigation capabilities of amphibious robots. Firstly, a method for representing the odometer's confidence based on a simplified wheel-terrain interaction model has been developed. This method quantitatively assesses the reliability of each odometer by estimating the slip rate. Secondly, we have introduced an improved split covariance intersection filter (I-SCIF), which maximizes the utilization of navigation information sources to enhance the accuracy of positional estimation. Finally, we will integrate these two methods to form the NAWR framework and validate the effectiveness of the proposed methods through multiple robot field trials. The results from both field trials and ablation tests collectively demonstrate that the modules and overall approach within the NAWR framework effectively enhance the navigation capabilities of amphibious robots.

Trajectory optimization and obstacle avoidance of autonomous robot using Robust and Efficient Rapidly Exploring Random Tree.

One of the key challenges in robotics is the motion planning problem. This paper presents a local trajectory planning and obstacle avoidance strategy based on a novel sampling-based path-finding algorithm designed for autonomous vehicles navigating complex environments. Although sampling-based algorithms have been extensively employed for motion planning, they have notable limitations, such as sluggish convergence rate, significant search time volatility, a vast, dense sample space, and unsmooth search routes. To overcome the limitations, including slow convergence, high computational complexity, and unnecessary search while sampling the whole space, we have proposed the RE-RRT* (Robust and Efficient RRT*) algorithm. This algorithm adapts a new sampling-based path-finding algorithm based on sampling along the displacement from the initial point to the goal point. The sample space is constrained during each stage of the random tree's growth, reducing the number of redundant searches. The RE-RRT* algorithm can converge to a shorter path with fewer iterations. Furthermore, the Choose Parent and Rewire processes are used by RE-RRT* to improve the path in succeeding cycles continuously. Extensive experiments under diverse obstacle settings are performed to validate the effectiveness of the proposed approach. The results demonstrate that the proposed approach outperforms existing methods in terms of computational time, sampling space efficiency, speed, and stability.

Learning to Play Guitar with Robotic Hands

AbstractPlaying the guitar is a dexterous human skill that poses significant challenges in computer graphics and robotics due to the precision required in finger positioning and coordination between hands. Current methods often rely on motion capture data to replicate specific guitar playing segments, which restricts the range of performances and demands intricate post‐processing. In this paper, we introduce a novel reinforcement learning model that can play the guitar using robotic hands, without the need for motion capture datasets, from input tablatures. To achieve this, we divide the simulation task for playing guitar into three stages. (a): for an input tablature, we first generate corresponding fingerings that align with human habits. (b): based on the generated fingerings as the guidance, we train a neural network for controlling the fingers of the left hand using deep reinforcement learning, and (c): we generate plucking movements for the right hand based on inverse kinematics according to the tablature. We evaluate our method by employing precision, recall, and F1 scores as quantitative metrics to thoroughly assess its performance in playing musical notes. In addition, we conduct qualitative analysis through user studies to evaluate the visual and auditory effects of guitar performance. The results demonstrate that our model excels in playing most moderately difficult and easier musical pieces, accurately playing nearly all notes.

Addressing cybersecurity challenges in robotics: A comprehensive overview

As robotics technology becomes increasingly integrated into various sectors, ensuring the cybersecurity of robotic systems is paramount. This article provides an in-depth exploration of the cybersecurity challenges confronting robotics and offers strategies to address these concerns. With the growing connectivity and networking capabilities of robots, vulnerabilities such as unauthorized access, data breaches, and network attacks are significant threats [1]. Protecting sensitive data collected and processed by robots is crucial to preserving privacy and trust. Remote access features, while enhancing operational flexibility, also pose security risks if not adequately secured. Weak authentication mechanisms and insecure interfaces could allow malicious actors to compromise robot functionality. Furthermore, robots are susceptible to malware and cyber-attacks, including viruses, worms, and ransomware. To mitigate these risks, a comprehensive approach is necessary, incorporating secure design principles, robust authentication mechanisms, encryption techniques, and cybersecurity training. Collaboration among industry stakeholders, researchers, policymakers, and cybersecurity experts is essential to develop resilient robotic systems capable of withstanding evolving cyber threats. This article underscores the importance of addressing cybersecurity challenges in robotics to ensure the safety and security of robotic deployments across diverse domains. As robotics technology evolves and becomes integral across various sectors, prioritizing cybersecurity [2] is crucial to protect these systems from unauthorized access, data breaches, and network attacks. The interconnected nature and remote access features of robots pose significant vulnerabilities. Comprehensive measures, including secure design, encryption, and cybersecurity training, are essential. Collaboration among industry stakeholders, researchers, policymakers, and cybersecurity experts is vital for developing resilient robotic systems. This article highlights the urgent need to address cybersecurity challenges to ensure the safety and integrity of robotic deployments.

RoboMan: An Adult-Sized Humanoid Robot with Enhanced Performance, Inherent Stability, and Two-Stage Balance Control to Facilitate Research on Humanoids

Creating an adult-sized humanoid robot with stable walking capabilities is a major challenge in robotics. While many renowned research groups focus on robots for perilous work environments and precision tasks, our approach simplifies balance control, making it accessible to robotics research groups and educational institutes. This facilitates the development of complex functionalities such as vision and object manipulation for adult-sized humanoids. This research article introduces RoboMan II, an advanced version of RoboMan I, which won the most prestigious award in all humanoid robot leagues at RoboCup 2016 due to its exceptional performance in walking and playing soccer. RoboMan II features significant improvements in performance, inherent stability, recovery after falls, and balance control. To facilitate its development, RoboMan II is lighter and incorporates a modified foot and parallel structure for its leg to boost its inherent stability, along with a two-stage balance control system for Immediate Response and Gradual Adaptation, enhancing its adaptability in various environments. Our simulation results demonstrate that RoboMan II’s walking stability on flat surfaces improved significantly in the face of minor perturbations, with the number of steps within the stable region increasing from 24%, with only the immediate controller to 58% when both controllers were used. Similar improvements were observed on inclined surfaces. Additionally, the 3D CAD files for all of the robot parts are released as open source in conjunction with this paper to facilitate reproduction and further innovation. The forthcoming RoboMan III will incorporate custom servo motors for increased speed, torque, and enhanced fall recovery, preventing disengagement of the gear box after a fall. It promises to be an invaluable asset for research and practical applications in humanoid robotics.

Challenges in contemporary spinal robotics: encouraging spine surgeons to drive transformative changes in the development of future robotic platforms.

The field of spinal robotics has witnessed considerable advances, which have primarily focused on enhancing pedicle screw placement. This article critically evaluates the current direction of spinal robotics development, raising concerns about the disproportionate emphasis on pedicle screw placement when existing techniques already yield commendable results. Discussions on various parameters, including quality, cost-effectiveness, and accessibility, highlight the need for a broader perspective in the development of robotics for spinal surgery. Comparative analyses reveal that navigation systems offer cost-effective and time-efficient alternatives to robotics, with similar accuracy levels. Patient demand for robotic interventions is influenced by perceived superiority, warranting careful consideration of public sentiment. This article also underscores the need for future spine surgeons to maintain proficiency in traditional techniques. The influence of industry and key opinion leaders in steering the focus toward pedicle screw placement is discussed, emphasizing the need for a more holistic approach. Accessibility issues and legal considerations in the evolving field of spinal robotics are addressed, and the potential for robotics to enhance various aspects of surgical procedures beyond pedicle screw placement is explored. In conclusion, we advocate for a shift in focus in spinal robotics, emphasizing the untapped potential to streamline common surgical procedures (such as discectomy, laminectomy, and endoscopy), enhance precision, and improve patient outcomes in areas beyond pedicle screw placement. Future advances in spinal robotics have the potential to transform the surgical landscape, benefitting all stakeholders, including patients, surgeons, and hospitals.

Motion Planning for a Legged Robot with Dynamic Characteristics.

Legged soccer robots present a significant challenge in robotics owing to the need for seamless integration of perception, manipulation, and dynamic movement. While existing models often depend on external perception or static techniques, our study aims to develop a robot with dynamic and untethered capabilities. We have introduced a motion planner that allows the robot to excel in dynamic shooting and dribbling. Initially, it identifies and predicts the position of the ball using a rolling model. The robot then pursues the ball, using a novel optimization-based cycle planner, continuously adjusting its gait cycle. This enables the robot to kick without stopping its forward motion near the ball. Each leg is assigned a specific role (stance, swing, pre-kick, or kick), as determined by a gait scheduler. Different leg controllers were used for tailored tiptoe trajectory planning and control. We validated our approach using real-world penalty shot experiments (5 out of 12 successful), cycle adjustment tests (11 out of 12 successful), and dynamic dribbling assessments. The results demonstrate that legged robots can overcome onboard capability limitations and achieve dynamic mobility and manipulation.

Ground4Act: Leveraging visual-language model for collaborative pushing and grasping in clutter

The challenge in robotics is to enable robots to transition from visual perception and language understanding to performing tasks such as grasp and assembling objects, bridging the gap between “seeing” and “hearing” to “doing”. In this work, we propose Ground4Act, a two-stage approach for collaborative pushing and grasping in clutter using a visual-language model. In the grounding stage, Ground4Act extracts target features from multi-modal data via visual grounding. In the action stage, it embeds a collaborative pushing and grasping framework to generate the action's position and direction. Specifically, we propose a DQN-based reinforcement learning pushing policy that uses RGBD images as the state space to determine the push action's pixel-level coordinates and direction. Additionally, a least squares-based linear fitting grasping policy takes the target mask from the grounding stage as input to achieve efficient grasp. Simulations and real-world experiments demonstrate Ground4Act's superior performance. The simulation suite, source code, and trained models will be made publicly available.

Application of AI integrated semi-autonomous robotics in spine surgery: A review

Spine surgery is a high-risk surgery associated with high precision and accuracy. Artificial intelligence (AI) and semi-autonomous robotics can benefit the surgery. This paper explores the impact of AI and robotics on improving surgical precision, accuracy, and safety with details on preoperative planning, optimal screw placement, and intraoperative real-time monitoring. In addition, the potential of AI-driven simulation spaces for training and skill assessment was evaluated. Various AI models and robotic systems have improved surgical outcomes by providing precise guidance and reducing human error. This review aims to summarize the recent developments and highlight the challenges of AI and semi-autonomous robotics in spine surgery.

A Multifunctional Tactile Sensory System for Robotic Intelligent Identification and Manipulation Perception.

Humans recognize and manipulate objects relying on the multidimensional force features captured by the tactile sense of skin during the manipulation. Since the current sensors integrated in robots cannot support the robots to sense the multiple interaction states between manipulator and objects, achieving human-like perception and analytical capabilities remains a major challenge for service robots. Prompted by the tactile perception involved in robots performing complex tasks, a multimodal tactile sensory system is presented to provide in situ simultaneous sensing for robots when approaching, touching, and manipulating objects. The system comprises a capacitive sensor owning the high sensitivity of 1.11E-2pFmm-1, a triboelectricity nanogenerator with the fast response speed of 30ms, and a pressure sensor array capable of 3D force detection. By Combining transfer learning models, which fuses multimodal tactile information to achieve high-precision (up to 95%) recognition of the multi-featured targets such as random hardness and texture information under random sampling conditions, including random grasp force and velocity. This sensory system is expected to enhance the intelligent recognition and behavior-planning capabilities of autonomous robots when performing complex tasks in undefined surrounding environments.

Programming gel automata shapes using DNA instructions

The ability to transform matter between numerous physical states or shapes without wires or external devices is a major challenge for robotics and materials design. Organisms can transform their shapes using biomolecules carrying specific information and localize at sites where transitions occur. Here, we introduce gel automata, which likewise can transform between a large number of prescribed shapes in response to a combinatorial library of biomolecular instructions. Gel automata are centimeter-scale materials consisting of multiple micro-segments. A library of DNA activator sequences can each reversibly grow or shrink different micro-segments by polymerizing or depolymerizing within them. We develop DNA activator designs that maximize the extent of growth and shrinking, and a photolithography process for precisely fabricating gel automata with elaborate segmentation patterns. Guided by simulations of shape change and neural networks that evaluate gel automata designs, we create gel automata that reversibly transform between multiple, wholly distinct shapes: four different letters and every even or every odd numeral. The sequential and repeated metamorphosis of gel automata demonstrates how soft materials and robots can be digitally programmed and reprogrammed with information-bearing chemical signals.

KISS-Keep It Static SLAMMOT-The Cost of Integrating Moving Object Tracking into an EKF-SLAM Algorithm.

The treatment of moving objects in simultaneous localization and mapping (SLAM) is a key challenge in contemporary robotics. In this paper, we propose an extension of the EKF-SLAM algorithm that incorporates moving objects into the estimation process, which we term KISS. We have extended the robotic vision toolbox to analyze the influence of moving objects in simulations. Two linear and one nonlinear motion models are used to represent the moving objects. The observation model remains the same for all objects. The proposed model is evaluated against an implementation of the state-of-the-art formulation for moving object tracking, DATMO. We investigate increasing numbers of static landmarks and dynamic objects to demonstrate the impact on the algorithm and compare it with cases where a moving object is mistakenly integrated as a static landmark (false negative) and a static landmark as a moving object (false positive). In practice, distances to dynamic objects are important, and we propose the safety-distance-error metric to evaluate the difference between the true and estimated distances to a dynamic object. The results show that false positives have a negligible impact on map distortion and ATE with increasing static landmarks, while false negatives significantly distort maps and degrade performance metrics. Explicitly modeling dynamic objects not only performs comparably in terms of map distortion and ATE but also enables more accurate tracking of dynamic objects with a lower safety-distance-error than DATMO. We recommend that researchers model objects with uncertain motion using a simple constant position model, hence we name our contribution Keep it Static SLAMMOT. We hope this work will provide valuable data points and insights for future research into integrating moving objects into SLAM algorithms.

Planning under uncertainty for safe robot exploration using Gaussian process prediction

The exploration of new environments is a crucial challenge for mobile robots. This task becomes even more complex with the added requirement of ensuring safety. Here, safety refers to the robot staying in regions where the values of certain environmental conditions (such as terrain steepness or radiation levels) are within a predefined threshold. We consider two types of safe exploration problems. First, the robot has a map of its workspace, but the values of the environmental features relevant to safety are unknown beforehand and must be explored. Second, both the map and the environmental features are unknown, and the robot must build a map whilst remaining safe. Our proposed framework uses a Gaussian process to predict the value of the environmental features in unvisited regions. We then build a Markov decision process that integrates the Gaussian process predictions with the transition probabilities of the environmental model. The Markov decision process is then incorporated into an exploration algorithm that decides which new region of the environment to explore based on information value, predicted safety, and distance from the current position of the robot. We empirically evaluate the effectiveness of our framework through simulations and its application on a physical robot in an underground environment.

Preventing isolation of people with dementia: Effectiveness and challenges of communication robots

Preventing isolation of people with dementia: Effectiveness and challenges of communication robots

A multi-strategy improved sparrow search algorithm for mobile robots path planning

Abstract Path planning for mobile robots plays a vital role in task execution, given the constraints imposed by environments and energy resources. It poses a significant challenge for mobile robots, requiring them to find a feasible path between the start point and target point that is obstacle-free and as short as possible. To address the challenge of path planning, a multi-strategy improved sparrow search algorithm with crazy operator (CMISSA) is proposed. Firstly, Tent chaos mapping and reverse learning are introduced into the population initialization of sparrow search algorithm (SSA) to enhance the uniformity and effectiveness of the initial population distribution. Secondly, adaptive parameters are applied in SSA to maintain a balance between exploitation and exploration. Thirdly, to prevent SSA from getting trapped in local optima, the crazy operator is used to perturb the population position. Finally, a novel adaptive boundary control strategy is introduced to handle the location of individuals that have crossed the boundary. In addition, the experimental results on 15 classical benchmark functions show that CMISSA has better optimization performance than other 10 comparison algorithms. Furthermore, in the path planning experimental results, the results of comparing CMISSA with 5 comparison algorithms on 5 different environments reveal CMISSA's average path shortening rates were 34.90%, 20.11%, 29.01%, 51.97%, 37.42%, respectively. It is further demonstrated that CMISSA has superior availability for solving mobile robots path planning.