Robots In Unstructured Environments Research Articles

Fall prediction, control, and recovery of quadruped robots

When legged robots perform complex tasks in unstructured environments, falls are inevitable due to unknown external disturbances. However, current research mainly focuses on the locomotion control of legged robots without falling. This paper proposes a comprehensive decision-making and control framework to address the falling over of quadruped robots. First, a capturability-based fall prediction algorithm is derived for planar single-contact and 3D multi-contact locomotion with a predefined gait sequence. For safe fall control, a novel contact-implicit trajectory optimization method is proposed to generate both state and input trajectories and contact mode sequences. Specifically, incorporating uncertainty into the system and terrain models enables mitigating the non-smoothness of contact dynamics while improving the robustness of the resulting trajectories. Furthermore, a model-free deep reinforcement learning-based approach is presented to achieve fall recovery after the robot completes a fall. Experimental results demonstrate that the proposed fall prediction algorithm accurately predicts robot falls with up to 95% accuracy approximately 395ms in advance. Compared to classical locomotion controllers, which often struggle to maintain balance under significant pushes or terrain perturbations, the presented framework can autonomously switch to the fall controller approximately 0.06s after the perturbation, effectively preventing falls or achieving recovery with a threefold reduction in touchdown impact velocity. These findings highlight the effectiveness of the proposed framework in enhancing the stability and safety of legged robots in unstructured environments.

OA-Pose: Occlusion-aware monocular 6-DoF object pose estimation under geometry alignment for robot manipulation

Object pose estimation is the fundamental technology of robot manipulation systems. Recently, various learning-based monocular pose estimation methods have achieved outstanding performance by establishing sparse/dense 2D-3D correspondences. However, in cluttered environments, occlusion has been a challenging problem for pose estimation due to limited information provided by visible parts. In this work, we propose an efficient occlusion-aware monocular pose estimation method, called OA-Pose, to learn geometric feature information of occluded objects from cluttered scenes. Our framework takes RGB images as input and generates 2D-3D correspondences of visible and invisible parts based on the proposed Occlusion-aware Geometry Alignment Module. Extensive experiments show that our method is superior and competitive with state-of-the-art on multiple public datasets. We also conduct grasping experiments with different degrees of object occlusion, demonstrating the usability of our algorithm to deploy on robots in unstructured environments.

A RRT based path planning scheme for multi-DOF robots in unstructured environments

Path planning for robots in complex orchard environments is crucial to ensure efficient and safe completion of harvesting tasks. The use of flexible multi-degree-of-freedom (multi-DOF) robots further adds to the complexity of the task. Therefore, based on a rapidly exploring random tree (RRT) algorithm, a bidirectional path planning method adjusting the expansion direction and dynamic step (EDDS-bi-RRT) according to the scene information is proposed to improve the planning efficiency of multi-DOF robots in the configuration space (C-space). Two improvements are introduced to the algorithm: (1) The algorithm begins by designating the search’s starting point as the tree’s root node. Subsequently, it generates an extended tree by continuously adding leaf nodes through sampling. The expansion direction is set from the nearest node to the target point, and the cosine and sine factors are introduced to adjust the direction when the sampling point collides with obstacles, and the extended trees will be exchanged for growth from the target point to the nearest node accordingly. (2) The minimum distance between the robot links and obstacles is the maximum safe distance d with which the robot links can move in the workspace. Then, d is converted to the expansion step in the high-dimensional C-space by introducing the matrix operator norm. This way, the explore efficiency is improved significantly in complex environments while the computational time and the path length decrease. Simulations in different scenarios are performed based on a 17-DOF humanoid dual-arm apple harvesting robot developed in our lab, and the results show that the proposed algorithm presents a higher efficiency and shorter planning time and path length compared with previous similar algorithms. In addition, experiments in the lab orchard environments are also performed to indicate that the EDDS-bi-RRT algorithm can be applied to generate collision-free path points for multi-DOF robots in unstructured environments safely and effectively.

Adaptive Online Learning and Robust 3-D Shape Servoing of Continuum and Soft Robots in Unstructured Environments.

In this article, we present a novel and generic data-driven method to servo-control the 3-D shape of continuum and soft robots based on proprioceptive sensing feedback. Developments of 3-D shape perception and control technologies are crucial for continuum and soft robots to perform tasks autonomously in surgical interventions. However, owing to the nonlinear properties of continuum robots, one main difficulty lies in the modeling of them, especially for soft robots with variable stiffness. To address this problem, we propose a versatile learning-based adaptive shape controller by leveraging proprioception of 3-D configuration from fiber Bragg grating (FBG) sensors, which can online estimate the unknown model of continuum robot against unexpected disturbances and exhibit an adaptive behavior to the unmodeled system without priori data exploration. Based on a new composite adaptation algorithm, the asymptotic convergences of the closed-loop system with learning parameters have been proven by Lyapunov theory. To validate the proposed method, we present a comprehensive experimental study using two continuum and soft robots both integrated with multicore FBGs, including a robotic-assisted colonoscope and multisection extensible soft manipulators. The results demonstrate the feasibility, adaptability, and superiority of our controller in various unstructured environments, as well as phantom experiments.

A Closed-Loop Shared Control Framework for Legged Robots

Shared control, as a combination of human and robot intelligence, has been deemed as a promising direction toward complementing the perception and learning capabilities of legged robots. However, previous works on human–robot control for legged robots are often limited to simple tasks, such as controlling movement direction, posture, or single-leg motion, yet extensive training of the operator is required. To facilitate the transfer of human intelligence to legged robots in unstructured environments, this article presents a user-friendly closed-loop shared control framework. The main novelty is that the operator only needs to make decisions based on the recommendations of the autonomous algorithm, without having to worry about operations or consider contact planning issues. Specifically, a rough navigation path from the operator is smoothed and optimized to generate a path with reduced traversing cost. The traversability of the generated path is assessed using fast Monte Carlo tree search, which is subsequently fed back through an intuitive image interface and force feedback to help the operator make decisions quickly, forming a closed-loop shared control. The simulation and hardware experiments on a hexapod robot show that the proposed framework gives full play to the advantages of human–machine collaboration and improves the performance in terms of learning time from the operator, mission completion time, and success rate than comparison methods.

Research on system integration and control methods of an apple-picking robot in unstructured environment

PurposeThe purpose of this article is to solve the issue that apple-picking robots are easily interfered by branches or other apples near the target apple in an unstructured environment, leading to grasping failure and apple damage.Design/methodology/approachThis study introduces the system units of the apple-picking robot prototype, proposes a method to determine the apple-picking direction via 3D point cloud data and optimizes the path planning method according to the calculated picking direction.FindingsAfter the field experiments, the average deviation of the calculated picking direction from the desired angle was 11.81°, the apple picking success rate was 82% and the picking cycle was 11.1 s.Originality/valueThis paper describes a picking control method for an apple-picking robot that can improve the success and reliability of picking in an unstructured environment and provides a basis for automated and mechanized picking in the future.

LIF-M: A Manifold-Based Approach for 3D Robot Localization in Unstructured Environments

Accurate localization of robots in unstructured environments poses challenges due to low localization accuracy and local trajectory oscillation caused by complex feature points when using Euclidean-based filtering methods. In this study, we propose a novel 3D robot localization method named LIF-M that leverages a manifold-based approach in conjunction with an unscented Kalman filter (UKF-M). Additionally, a relocalization algorithm is designed to ensure localization stability. The proposed method addresses the limitations of conventional Euclidean-based filtering methods by incorporating manifold-based techniques, providing a more comprehensive representation of the complex geometric features. We introduce the manifold concept, where the relevant definition is defined and utilized within the LIF-M framework. By combining left and right invariants, we effectively reduce noise uncertainty, resulting in improved localization accuracy. Moreover, we employ sigma points as a matrix representation of the state points’ space in order to seamlessly transition between the matrix space and the vector representation of the tangent space. Experimental tests and error calculations were conducted to evaluate the performance of various algorithm frameworks, and the results demonstrated the importance of the manifold-based approach for accurate attitude estimation. Compared to the standard UKF, the manifold space equips LIF-M with better robustness and stability in unstructured environments.

Optimization of fuzzy rules using neural network to control mobile robot in non-structured environment

It is still a challenge for all authors to control an autonomous mobile robot in an unstructured environment. The purpose of this paper is to propose a new control method for mobile robots in unstructured environments using neuro fuzzy technique. The proposed algorithm reduces the processing time of the fuzzy logic controller (FLC) inference engine. The neural network (NN) will therefore select the optimum rule(s) directly from the inference engine. This means that all of the rules of the inference engine do not need to be processed. As a result, the inference engine process speed will decrease, and the fuzzy logic response will increase. An actual mobile robot with three distance sensors and one virtual orientation angle is used to test the proposed algorithm. Based on the results, the mobile robot is capable of avoiding all obstacles and reaching the target point accurately.

Self-adaptive rolling motion for snake robots in unstructured environments based on torque control

Snake robots have great potential for exploring and operating in challenging unstructured environments, such as rubble, caves, and narrow pipelines. However, due to the complexity and unpredictability of unstructured environments, designing a controller that can achieve adaptive motion is crucial. This paper proposes a self-adaptive torque-based rolling controller for snake robots, enabling compliant motion in unstructured environments. First, a controller is designed to modify the torque of each motor by focusing on the different motion states of the rolling gait. Second, an experimental platform is established for snake robots to verify the effectiveness of the controller. Finally, a series of rolling experiments are conducted using the torque-based rolling controller. In conclusion, the self-adaptive torque-based rolling controller enhances snake robot adaptability and mobility.

DeforMoBot: A Bio-Inspired Deformable Mobile Robot for Navigation Among Obstacles

Many animals can move in cluttered environments by conforming their body shape to geometric constraints in their surroundings such as narrow gaps. Most robots are rigid structures and do not possess these capabilities. Navigation around movable or compliant obstacles results in a loss of efficiency—and possible mission failure—compared to progression through them. In this letter, we propose the novel design of a deformable mobile robot; it can adopt a wider stance for greater stability (and possible higher payload capacity), or a narrower stance to become capable of fitting through small gaps and progressing through flexible obstacles. We use a whisker-based feedback control approach to match the amount of the robot's deformation with the compliance level of the obstacle. We present a real-time algorithm which uses whisker feedback and performs shape adjustment in uncalibrated environments. The developed robot was tested navigating among obstacles with varying physical properties from different approach angles. Our results highlight the importance of co-development of environment perception and physical reaction capabilities for improved performance of mobile robots in unstructured environments.

Kinematic modeling and motion characterization of an articulated mobile robot

In response to the outstanding problems of poor motion flexibility and poor terrain adaptability of conventional mobile robots in unstructured environments, a multi-vehicle modular articulated mobile robot capable of adapting to complex and variable obstacle terrain was developed by designing and introducing a flexible articulation device. By constructing a kinematic model of the single body module of the robot, integrating the motion constraint relations among the modules, and establishing the overall robot attitude equations, we propose a robot soft motion control method. Based on the analysis of robot motion characteristics under special obstacle terrain, the relationship model between obstacle parameters and robot pose is constructed, and the correction strategy under unilateral unstable motion state is proposed through the analysis of robot motion state under unilateral unstable condition. Theoretical analysis and experimental results show that the robot has good motion flexibility and complex terrain adaptability.

A unified modeling and trajectory planning method based on system manipulability for the operation process of the legged locomotion manipulation system

AbstractFlexibility is one of the most significant advantages of legged robots in unstructured environments. However, quadruped robots cannot interact with environments to complete some manipulation tasks. One effective way is to load a manipulation arm. In this paper, we exhibit a quadruped locomotion manipulation system (LMS) named HITPhanT. This system comprises a quadruped locomotion platform and a six-degree-of-freedom manipulation arm. Besides, when the LMS moves to a designated position for operation, it is necessary to constrain the foot contact points to avoid sliding. Therefore, the foot contact point is regarded as a spherical hinge. So the locomotion platform can be considered as a parallel mechanism. A hybrid kinematics model is established by considering the serial robotic arms connecting this parallel mechanism. Besides, the trajectory planning method, which improves the system’s manipulability in evaluating the system balance, is also proposed. Finally, corresponding experiments verify the overall system’s stabilization and algorithm’s effectiveness.

Cone–hole docking mechanism for a modular reconfigurable mobile robot and its characteristic analysis

Purpose The purpose of this paper is that the modular mobile robots reformed the multimachine joint mode to achieve obstacle-crossing, climbing and other multifunctional inspection in unstructured environment under the connection of the cone–hole docking mechanism. Design/methodology/approach An arc-shaped docking cone head with a posture-maintaining spring and two arc-shaped connecting rods that formed a ring round hole were designed to achieve large tolerance docking. Before active locking, the coordination between structures was used to achieve passive locking, which mitigated the docking impact of modular robots in unstructured environment. Using the locking ring composed of the two arc-shaped connecting rods, open-loop and closed-loop motion characteristics were obtained through the mutual motion of the connecting rod and the sliding block to achieve active locking, which not only ensured high precision docking, but also achieved super docking stability. Findings The cone–hole docking mechanism had the docking tolerance performance of position deviation of 6mm and pitch deviation of 8° to achieve docking of six degrees of freedom (6-DOF), which had a load capacity of 230 N to achieve super docking stability. Under the connection of the cone–hole docking mechanism, the modular mobile robots reformed the multimachine joint mode to achieve obstacle-crossing, climbing and other multifunctional inspection in unstructured environment. Originality/value Based on mechanical analysis of universal models, a cone–hole docking mechanism combining active and passive functions, six-dimensional constraints could be implemented, was proposed in this paper. The characteristics of the posture-maintaining spring in the cone docking head and the compression spring at the two ends of two arc-shaped connecting rods were used to achieve docking with large tolerance. Passive locking and active locking modules were designed, mitigating impact load and the locking did not require power to maintain, which not only ensured high precision docking, but also achieved super docking stability.

Multitarget Flexible Grasping Detection Method for Robots in Unstructured Environments

Multitarget Flexible Grasping Detection Method for Robots in Unstructured Environments

Terrain-aware communication coverage prediction for cooperative networked robots in unstructured environments

Networked robots will play an important role in lunar exploration. Communication is key to enable cooperation among robots for information sharing, and to remotely control robots with lower degree of autonomy from a lander or habitat. Operators and scientists must be able to make sound decisions on communication availability before or during sending robots to regions of interest for exploration. In this work we have a closer look at the communication coverage prediction for lunar exploration. We present an interdisciplinary and modular framework, which exploits terrain information to predict the data rate for exploring robots. Additionally, we create intuitively usable coverage maps for operators and scientists, and show how connectivity can be improved in unstructured environments by using a relay rover. This paper provides an overview of this framework, details on individual framework components, and simulation results for two exemplary exploration scenarios.

In-field pose estimation of grape clusters with combined point cloud segmentation and geometric analysis

In-field object detection and pose estimation are challenging tasks in industrial harvesting scenarios. This study investigates a method for object detection and 6D pose estimation using a depth camera to prevent harvesting robots from collisions while harvesting grape clusters. First, the Mask Region Convolutional Neural Network (Mask R-CNN) is deployed to segment 2D images and output binary mask images of grape clusters. Second, the grape cluster point cloud is segmented based on the binary mask image and the mapping relationship between the image and point cloud to provide a high-quality point cloud through pre-processing. Third, the optimal cutting point is located by constructing the grape peduncle region of interest (RoI). Finally, the peduncle surface is fit based on the locally weighted scatterplot smoothing (LOWESS) algorithm, and the pose of the peduncle is estimated using geometric methods. The detection results from 238 test images show that the mean Precision (mP) is 86.0%, the mean Recall (mR) is 79.9%, the F1-score is 0.828, and mean Intersection Over Union (mIOU) for instance segmentation is 87.9%. The pose estimation results from 172 grape peduncles yield an error angle of 22.22 degrees ± 17.96 degrees in pose estimation. The detection and pose estimations for each grape cluster require approximately 1.786 s. The demonstrated performance of the proposed method indicates it can be applied to collision-free grape harvesting using robots in unstructured environments.

Division-merge based inverse kinematics for multi-DOFs humanoid robots in unstructured environments

A Two-step Division-Merge (TDM) based inverse kinematics (IK) mechanism is proposed for robots with multiple degrees of freedom (multi-DOFs) in unstructured environments. For any robot with n-DOFs (n>6), a division point is selected first to divide it into a lower part subsystem from its base to the division point, together with an upper part one from the point to its end-effector. Once a collision-free configuration is found for the upper part subsystem to match with the target, the lower part subsystem is then planned to merge with the upper part subsystem at the division point. Specifically, the division point is selected such that the lower part subsystem has 6-DOFs, while the upper part subsystem has (n-6)-DOFs. Furthermore, a Reverse & Random Rotation (RRR) algorithm is also proposed to solve the IK of the upper part subsystem in unstructured environments, while the IK of the lower part subsystem is solved analytically. The proposed TDM based mechanism is verified via experiments using a lab-customized humanoid apple harvesting robot with 11- and 13-DOFs, respectively. Under the various scenarios with different target and obstacle configurations, errors in position coordinates and orientation angles between the target pose and those of the end-effector obtained with forward kinematics (FK) are calculated. Results show that the errors in position and orientation are less than 0.005 mm and 0.007°, respectively, and the average computational time is less than 2 s for the IK solution of the entire robot. Such results indicate that the proposed mechanism is reliable and capable of reducing the complexity of IK for multi-DOFs humanoid robots. With the proposed algorithm, the apple harvesting success rate of a lab-customized 11-DOFs robot reaches 89.2%, which can be further increased up to 93.3% with error compensation, while the average time starting from the robot action to the end-effector reaching the target pose of the apple is around 12.3 s.

Twisting for soft intelligent autonomous robot in unstructured environments

Soft robots that can harvest energy from environmental resources for autonomous locomotion is highly desired; however, few are capable of adaptive navigation without human interventions. Here, we report twisting soft robots with embodied physical intelligence for adaptive, intelligent autonomous locomotion in various unstructured environments, without on-board or external controls and human interventions. The soft robots are constructed of twisted thermal-responsive liquid crystal elastomer ribbons with a straight centerline. They can harvest thermal energy from environments to roll on outdoor hard surfaces and challenging granular substrates without slip, including ascending loose sandy slopes, crossing sand ripples, escaping from burying sand, and crossing rocks with additional camouflaging features. The twisting body provides anchoring functionality by burrowing into loose sand. When encountering obstacles, they can either self-turn or self-snap for obstacle negotiation and avoidance. Theoretical models and finite element simulation reveal that such physical intelligence is achieved by spontaneously snapping-through its soft body upon active and adaptive soft body-obstacle interactions. Utilizing this strategy, they can intelligently escape from confined spaces and maze-like obstacle courses without any human intervention. This work presents a de novo design of embodied physical intelligence by harnessing the twisting geometry and snap-through instability for adaptive soft robot-environment interactions.

Spatio-Temporal Encoding Improves Neuromorphic Tactile Texture Classification

With the increase in interest in the deployment of robots in unstructured environments to work alongside humans, the development of human-like sense of touch for robots becomes important. In this work, we implement a multi-channel neuromorphic tactile system that encodes contact episodes as discrete spike events that mimic the behavior of slow adapting mechanoreceptors. We study the impact of information pooling across artificial mechanoreceptors on classification performance of spatially non-uniform naturalistic textures. We encoded the spatio-temporal activation patterns of mechanoreceptors through gray-level co-occurrence matrix computed from time-varying mean spiking rate-based tactile response volume. We found that this approach greatly improved texture classification in comparison to the use of individual mechanoreceptor response alone. In addition, the performance was also more robust to changes in sliding velocity. The importance of exploiting precise spatial and temporal correlations between sensory channels is evident from the fact that on either removal of precise temporal information or altering of spatial structure of response pattern, a significant performance drop was observed. This study thus demonstrates the superiority of population coding approaches that can exploit the precise spatio-temporal information encoded in activation patterns of mechanoreceptor populations. It, therefore, makes an advance in the direction of development of bio-inspired tactile systems required for realistic touch applications in robotics and prostheses.

Autonomous system to control a mobile robot

This paper presents an ongoing effort to control a mobile robot in unstructured environment. Obstacle avoidance is an important task in the field of robotics, since the goal of autonomous robot is to reach the destination without collision. Several algorithms have been proposed for obstacle avoidance, having drawbacks and benefits. In this paper, the fuzzy controller is used to tackle the problem of mobile robot autonomous navigation in unstructured environment. The objective is to make the robot move along a collision free trajectory until it reaches its target. The proposed approach uses the fuzzified, adaptive inference engine and defuzzification engine. Also number of linguistic labels is optimized for the input of the mobile robot in order to reduce computational time for real-time applications. The proposed fuzzy controller is evaluated subjectively and objectively with other approaches and also the processing time is taken in consideration.