Unstructured Scenarios Research Articles

Automated parking trajectory planning: a hybrid curve search and nonlinear optimisation approach

ABSTRACT This paper focuses on the problem of automatic parking trajectory planning in unstructured scenarios. The proposed algorithm adopts a combination of graph search and optimization strategy. First, A* search is utilized to obtain paths connecting the global start and end points. Second, the path is divided into three segments and subpaths are searched using the hybrid A* algorithm and the proposed A*-iBspline algorithm. Finally, the three subpaths are merged and a valid initial guess is generated for the OCP. Additionally, an iterative framework is used to improve the quality of the OCP solution. Simulation results show that the proposed algorithm has a significant advantage over existing methods in terms of time and applicability.

Dense Feature Matching for Hazard Detection and Avoidance Using Machine Learning in Complex Unstructured Scenarios

Exploring the Moon and Mars are crucial steps in advancing space exploration. Numerous missions aim to land and research in various lunar locations, some of which possess challenging surfaces with unchanging features. Some of these areas are cataloged as lunar light plains. Their main characteristics are that they are almost featureless and reflect more light than other lunar surfaces. This poses a challenge during navigation and landing. This paper compares traditional feature matching techniques, specifically scale-invariant feature transform and the oriented FAST and rotated BRIEF, and novel machine learning approaches for dense feature matching in challenging, unstructured scenarios, focusing on lunar light plains. Traditional feature detection methods often need help in environments characterized by uniform terrain and unique lighting conditions, where unique, distinguishable features are rare. Our study addresses these challenges and underscores the robustness of machine learning. The methodology involves an experimental analysis using images that mimic lunar-like landscapes, representing these light plains, to generate and compare feature maps derived from traditional and learning-based methods. These maps are evaluated based on their density and accuracy, which are critical for effective structure-from-motion reconstruction commonly utilized in navigation for landing. The results demonstrate that machine learning techniques enhance feature detection and matching, providing more intricate representations of environments with sparse features. This improvement indicates a significant potential for machine learning to boost hazard detection and avoidance in space exploration and other complex applications.

Research on multitask model of object detection and road segmentation in unstructured road scenes

With the rapid development of artificial intelligence and computer vision technology, autonomous driving technology has become a hot area of concern. The driving scenarios of autonomous vehicles can be divided into structured scenarios and unstructured scenarios. Compared with structured scenes, unstructured road scenes lack the constraints of lane lines and traffic rules, and the safety awareness of traffic participants is weaker. Therefore, there are new and higher requirements for the environment perception tasks of autonomous vehicles in unstructured road scenes. The current research rarely integrates the target detection and road segmentation to achieve the simultaneous processing of target detection and road segmentation of autonomous vehicle in unstructured road scenes. Aiming at the above issues, a multitask model for object detection and road segmentation in unstructured road scenes is proposed. Through the sharing and fusion of the object detection model and road segmentation model, multitask model can complete the tasks of multi-object detection and road segmentation in unstructured road scenes while inputting a picture. Firstly, MobileNetV2 is used to replace the backbone network of YOLOv5, and multi-scale feature fusion is used to realize the information exchange layer between different features. Subsequently, a road segmentation model was designed based on the DeepLabV3+ algorithm. Its main feature is that it uses MobileNetV2 as the backbone network and combines the binary classification focus loss function for network optimization. Then, we fused the object detection algorithm and road segmentation algorithm based on the shared MobileNetV2 network to obtain a multitask model and trained it on both the public dataset and the self-built dataset NJFU. The training results demonstrate that the multitask model significantly enhances the algorithm’s execution speed by approximately 10 frames per scond while maintaining the accuracy of object detection and road segmentation. Finally, we conducted validation of the multitask model on an actual vehicle.

Trajectory Planning for Autonomous Driving in Unstructured Scenarios Based on Deep Learning and Quadratic Optimization

Trajectory Planning for Autonomous Driving in Unstructured Scenarios Based on Deep Learning and Quadratic Optimization

Co-Optimization of Speed Planning and Power Management for Automated HEVs in Unstructured Road Scenarios

With the development of intelligent and connected vehicle technology, the simultaneous optimization of powertrain performance and vehicle motions provides an unprecedented perspective to further explore the energy-saving potential of hybrid electric vehicles (HEV). However, it is still very challenging because of the complex driving environment and large computational burdens, and hence, the co-optimization strategy under unstructured scenarios has rarely been studied. For unstructured road driving scenarios, this paper proposes a hierarchical optimal control system combing speed planning and energy management for an autonomous HEV. In the first layer, a vehicle stability constraints system is designed with consideration of both external road characteristics and vehicle dynamics to limit the longitudinal speed, preventing the vehicle from excessive yaw, sideslip, and rollover during a complex, unstructured road trip. Then, the improved gray wolf optimizer is designed to efficiently generate optimal speed and SOC trajectories to guide the vehicle motion behavior and power split while ensuring driver safety. In the second layer, a mechanical implementation system is applied to fulfill the control system framework. Finally, the performance of proposed co-optimization is comprehensively verified in terms of fuel economy, vehicle maneuverability, computational effectiveness, and adaptability. Compared with the sequential optimization, the improvements by co-optimization range from 9.38% to 13.03% in fuel economy and from 0.31% to 3.15% in vehicle maneuverability under different test cycles, while ensuring real-time capability.

In-Depth Evaluation of Automated Fruit Harvesting in Unstructured Environment for Improved Robot Design

Using modern machines like robots comes with its set of challenges when encountered with unstructured scenarios like occlusion, shadows, poor illumination, and other environmental factors. Hence, it is essential to consider these factors while designing harvesting robots. Fruit harvesting robots are modern automatic machines that have the ability to improve productivity and replace labor for repetitive and laborious harvesting tasks. Therefore, the aim of this paper is to design an improved orange-harvesting robot for a real-time unstructured environment of orchards, mainly focusing on improved efficiency in occlusion and varying illumination. The article distinguishes itself with not only an efficient structural design but also the use of an enhanced convolutional neural network, methodologically designed and fine-tuned on a dataset tailored for oranges integrated with position visual servoing control system. Enhanced motion planning uses an improved rapidly exploring random tree star algorithm that ensures the optimized path for every robot activity. Moreover, the proposed machine design is rigorously tested to validate the performance of the fruit harvesting robot. The unique aspect of this paper is the in-depth evaluation of robots to test five areas of performance that include not only the accurate detection of the fruit, time of fruit picking, and success rate of fruit picking, but also the damage rate of fruit picked as well as the consistency rate of the robot picking in varying illumination and occlusion. The results are then analyzed and compared with the performance of a previous design of fruit harvesting robot. The study ensures improved results in most aspects of the design for performance in an unstructured environment.

A Survey of Trajectory Planning Methods for Autonomous Driving — Part I: Unstructured Scenarios

A Survey of Trajectory Planning Methods for Autonomous Driving — Part I: Unstructured Scenarios

Regularized Maximum Correntropy Criterion Kalman Filter for Uncalibrated Visual Servoing in the Presence of Non-Gaussian Feature Tracking Noise.

Some advantages of using cameras as sensor devices on feedback systems are the flexibility of the data it represents, the possibility to extract real-time information, and the fact that it does not require contact to operate. However, in unstructured scenarios, Image-Based Visual Servoing (IBVS) robot tasks are challenging. Camera calibration and robot kinematics can approximate a jacobian that maps the image features space to the robot actuation space, but they can become error-prone or require online changes. Uncalibrated visual servoing (UVS) aims at executing visual servoing tasks without previous camera calibration or through camera model uncertainties. One way to accomplish that is through jacobian identification using environment information in an estimator, such as the Kalman filter. The Kalman filter is optimal with Gaussian noise, but unstructured environments may present target occlusion, reflection, and other characteristics that confuse feature extraction algorithms, generating outliers. This work proposes RMCKF, a correntropy-induced estimator based on the Kalman Filter and the Maximum Correntropy Criterion that can handle non-Gaussian feature extraction noise. Unlike other approaches, we designed RMCKF for particularities in UVS, to deal with independent features, the IBVS control action, and simulated annealing. We designed Monte Carlo experiments to test RMCKF with non-Gaussian Kalman Filter-based techniques. The results showed that the proposed technique could outperform its relatives, especially in impulsive noise scenarios and various starting configurations.

A Safety-Assured Semantic Map for an Unstructured Terrain Environment towards Autonomous Engineering Vehicles

Accurate obstacle detection plays a crucial role in the creation of high-precision maps within unstructured terrain environments, as it supplies vital decision-making information for unmanned engineering vehicles. Existing works primarily focus on the semantic segmentation of terrain environments, overlooking the safety aspect of vehicle driving. This paper presents a hazardous obstacle detection framework in addition to driving safety-assured semantic information in the generated high-precision map of unstructured scenarios. The framework encompasses the following key steps. Firstly, a continuous terrain point cloud model is obtained, and a pre-processing algorithm is designed to filter noise and fill holes in the point cloud dataset. The Sobel-G operator is then utilized to establish a digital gradient model, facilitating the labeling of hazardous obstacles. Secondly, a bidirectional long short-term memory (Bi-LSTM) neural network is trained on obstacle categories. Finally, by considering the geometric driving state of the vehicle, obstacles that pose safety risks to the vehicle are accurately extracted. The proposed algorithm is validated through experiments conducted on existing datasets as well as real, unstructured terrain point clouds reconstructed by drones. The experimental results affirm the accuracy and feasibility of the proposed algorithm for obstacle information extraction in unstructured scenes.

Perspectives on Computation in Plants.

Plants thrive in virtually all natural and human-adapted environments and are becoming popular models for developing robotics systems because of their strategies of morphological and behavioral adaptation. Such adaptation and high plasticity offer new approaches for designing, modeling, and controlling artificial systems acting in unstructured scenarios. At the same time, the development of artifacts based on their working principles reveals how plants promote innovative approaches for preservation and management plans and opens new applications for engineering-driven plant science. Environmentally mediated growth patterns (e.g., tropisms) are clear examples of adaptive behaviors displayed through morphological phenotyping. Plants also create networks with other plants through subterranean roots-fungi symbiosis and use these networks to exchange resources or warning signals. This article discusses the functional behaviors of plants and shows the close similarities with a perceptron-like model that could act as a behavior-based control model in plants. We begin by analyzing communication rules and growth behaviors of plants; we then show how we translated plant behaviors into algorithmic solutions for bioinspired robot controllers; and finally, we discuss how those solutions can be extended to embrace original approaches to networking and robotics control architectures.

Real-time detection of a camouflaged object in unstructured scenarios based on hierarchical aggregated attention lightweight network

Accurate and real-time grasping of household items is a challenge for home service robots in complex indoor environments. In this paper, using transparent cups as the recognition object, an accurate, reliable, and real-time visual algorithm is proposed for the elderly who needs nursing care to solve the recognition problems of overlapping, blurred, and small cups in complex unstructured indoor scenes. In order to meet the lightweight deployment of home service robots while considering both high-scale and low-scale semantic features, this paper designs a multi-scale fusion lightweight backbone network based on the Split Shuffle Block (SSB) and Group Shuffle Block (GSB) feature encoding units. Among them, depthwise separable convolution (DwConv) uses to reduce the amount of coding unit parameters, and feature shuffle uses to promote information exchange and the ability of information expression. In order to accurately identify the transparent cup camouflaged in complex backgrounds, this paper proposed a lightweight feature enhancement module that combines multi-scale hierarchical aggregation attention and multi-branch parallel convolution structure. The proposed module used an adaptive weighting strategy and a channel normalization weighting strategy to highlight the active regions of boundary features in each branch feature map, enhance the exchange of boundary information, and reduce the loss of detailed information. The experimental results on the IndoorCup and MobileCup datasets show that the detection accuracy of the proposed method is 93.6% and 92.6%, respectively, and the model calculation amount is only 0.91 M, which can lightweight deploy on indoor mobile robots for real-time detection. From the results of qualitative comparisons, the proposed method has strong robustness. It can effectively suppress false and missed detection caused by background interference. Likewise, it also effectively identifies camouflage objects and small objects in complex backgrounds.

Autonomous Following for Unmanned Ground Vehicles on Unstructured Scenario: Risk and Performance Assessment

Military unmanned ground vehicles are often used in wild natural environment, autonomous following and accompanying security are typical functions. In order to make a quantitative assessment about the autonomous following performance of unmanned ground vehicles on unstructured scenarios, a set of evaluation index system construction method is proposed based on the combination of risk analysis and key performance indexes. First, the kinematics model of autonomous vehicle following in straight and curve based on meta-trajectory is established. Second, the key indexes affecting the following performance are analyzed and obtained from collision risk and lost risk. Third, the evaluation index system of autonomous following performance is constructed from five dimensions: average following speed, collision risk, collision hazard degree, following stability and lost risk. Finally, two unmanned vehicles are tested, the results show that the conclusion based on the constructed evaluation index system can significantly reflect the difference of autonomous following performance, which is basically consistent with the subjective evaluation of the test personnel.

Bioinspired Soft Wrist Based on Multicable Jamming With Hybrid Motion and Stiffness Control for Dexterous Manipulation

Dexterous manipulation is important for modern logistics automation. Industrial robots remain inferior to human labor in this area, especially in handling unknown objects in unstructured scenarios. It is observed that the human wrist contributes significantly to the dexterous manipulation capability by agilely changing the hand orientation to approach the target object and adjusting wrist stiffness to adapt to dynamic interaction. Inspired by the human wrist, we propose a novel soft wrist to achieve hybrid motion/stiffness control in a compact and lightweight structure. The dexterous motion and stiffness adjustment are simultaneously enabled by a novel multicable jamming mechanism, which is achieved by a precalibration of the relationship between stiffness with robot length and bending angle to adapt to both compliant and forceful tasks. Dedicated experiments were performed to validate its motion dexterity and variable stiffness property. A soft grasping system was developed to demonstrate the manipulation capability of the soft wrist by tasking it with automatic packaging, which is challenging for robots. Overall, the proposed soft wrist demonstrates a promising solution to enhance robotic manipulation capability to a human comparable level.

A Bin-Picking Benchmark for Systematic Evaluation of Robotic-Assisted Food Handling for Line Production

Robotic manipulation and automation have gained increasing popularity in the food manufacturing industry due to their potential benefits for enhancing hygiene standards, enforcing quality consistency, promoting product traceability, and reducing labor costs. As a majority of robotic manipulation, the pick-and-place operation plays a crucial role in food handling applications. However, the reproducibility and comparability of results have put a dilemma that hinders further advancement in this field, especially for those unstructured scenarios. To tackle such thorny issues, this article proposes a benchmarking framework for system-level evaluation of robotic-assisted food handling under the line production environment. A typical food handling scenario, including a pick-and-place operation and a packing operation, is presented as the benchmark task, where food items are supposed to be picked from the tray and placed in the serving dish. A robotic system incorporating a high-speed Delta robot, vision system, conveyor belt, and end-effector is developed as the testbed for the benchmarking implementation. Finally, five variants of the robotic system with different end-effectors are evaluated using the proposed benchmarking framework. Comparative studies illustrate the performance of various benchmarked systems and validate the applicability of the benchmarking strategy for the food handling context. Videos of our experiments are available at <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://youtu.be/SBAOoswnjWM</i> .

A semantic robotic grasping framework based on multi-task learning in stacking scenes

Autonomous robotic grasping is an essential skill for service robots to perform specified tasks in unstructured scenarios. Previous work focus on simple pick-and-place tasks, and it is not satisfactory for real-world scenes that have requirements for manipulation. In this paper, we present a modular intelligent robot architecture via multi-task convolutional neural network which can be used for specific object grasping and manipulation in a stacked and cluttered environment. Firstly, an end-to-end, multi-task semantic grasping convolutional neural network (MSG-ConvNet) that simultaneously outputs the results of grasp detection and semantic segmentation is proposed to recognize the affiliations between objects and grasps in cluttered scenarios. Secondly, we propose a post-processing method which allows the robot to select an optimal grasping area in an active perception way through simply reasoning on the multi-modal information output by the proposed model. The proposed multi-task network has a great improvement in both recognition accuracy and detection speed on the public multi-object dataset GraspNet-1Billion compared with the benchmark. The proposed grasp detection method also yields state-of-the-art performance with accuracies of 95.06% and 98.6% on the public single-object Jacquard Dataset and Cornell Dataset, respectively. In addition, the experiments in a real-world scene demonstrate that our proposed method has stronger robustness and adaptability than the simple direct grasping strategy in the environment with higher mutual occlusion.

A method to benchmark the balance resilience of robots.

Robots that work in unstructured scenarios are often subjected to collisions with the environment or external agents. Accordingly, recently, researchers focused on designing robust and resilient systems. This work presents a framework that quantitatively assesses the balancing resilience of self-stabilizing robots subjected to external perturbations. Our proposed framework consists of a set of novel Performance Indicators (PIs), experimental protocols for the reliable and repeatable measurement of the PIs, and a novel testbed to execute the protocols. The design of the testbed, the control structure, the post-processing software, and all the documentation related to the performance indicators and protocols are provided as open-source material so that other institutions can replicate the system. As an example of the application of our method, we report a set of experimental tests on a two-wheeled humanoid robot, with an experimental campaign of more than 1100 tests. The investigation demonstrates high repeatability and efficacy in executing reliable and precise perturbations.

High-Performance Pixel-Level Grasp Detection Based on Adaptive Grasping and Grasp-Aware Network

Machine vision-based planar grasping detection is challenging due to uncertainty about object shape, pose, size, etc. Previous methods mostly focus on predicting discrete gripper configurations, and may miss some ground-truth grasp postures. In this article, a pixel-level grasp detection method is proposed, which uses deep neural network to predict pixel-level gripper configurations on RGB images. First, a novel oriented arrow representation model (OAR-model) is introduced to represent the gripper configuration of parallel-jaw and three-fingered gripper, which can partly improve the applicability to different grippers. Then, the adaptive grasping attribute model is proposed to adaptively represent the grasping attribute of objects, for resolving angle conflicts in training and simplifying pixel-level labeling. Lastly, the adaptive feature fusion and grasp-aware network (AFFGA-Net) is proposed to predict pixel-level OAR-models on RGB images. AFFGA-Net improves the robustness in unstructured scenarios by using hybrid atrous spatial pyramid and adaptive decoder connected in sequence. On the public Cornell dataset and actual objects, our structure achieves 99.09% and 98.0% grasp detection accuracy, respectively. In over 2400 robotic grasp trials, our structure achieves an average success rate of 98.77% in single-object scenarios and 93.69% in cluttered scenarios. Moreover, AFFGA-Net completes a grasp detection pipeline within 15 ms.

Decentralized Control of a Heterogeneous Human–Robot Team for Exploration and Patrolling

We present a decentralized connectivity-maintenance control framework for a heterogeneous human–robot team. The algorithm is able to manage a team composed of an arbitrary number of mobile robots (drones and ground robots in our case) and humans, for collaboratively achieving exploration and patrolling tasks. Differently from other works on the subject, here the human user physically becomes part of the team, moving in the same environment of the robots and receiving information about the team connectivity through wearable haptics or audio feedback. Although human explores the environment, robots move so as to keep the team connected via a connectivity-maintenance algorithm; at the same time, each robot can also be assigned with a specific target to visit. We carried out three human subject experiments, both in virtual and real environments. Results show that the proposed approach is effective in a wide range of scenarios. Moreover, providing either haptic or audio feedback for conveying information about the team connectivity significantly improves the performance of the considered tasks, although users significantly preferred receiving haptic stimuli w.r.t. the audio ones. Note to Practitioners—Exploration, patrolling, and search-and-rescue are highly dynamic and unstructured scenarios. When considering the operative conditions of such environments, the benefits of multirobot systems are evident. Most tasks can be carried out faster and more robustly by a team of robots with respect to a single unit. There are also situations explicitly requiring the presence of a multirobot team, e.g., using one drone for surveillance of the ground team and one ground mobile robot for carrying supplies. Of course, if the operator(s) in charge of the operation could share the same environment of the robots (i.e., be together with the robots in the field), they would be provided with a level of situational awareness that no teleoperation technology can match as of today. This work presents a framework for controlling heterogeneous teams composed of one human operator and an arbitrary number of aerial and ground mobile robots. The operator moves together with the robotic team and, at the same time, he or she receives meaningful information about the status of the formation. The algorithm only uses the relative position of the drones and humans with respect to each other, and all computations are designed in a decentralized fashion. Decentralization avoids relying on any absolute positioning system (e.g., GPS) or centralized command centers. These features make the proposed framework ready for deployment in different high-impact applications, such as in surveillance, search-and-rescue, and disaster response scenarios.

3D SLAM based on NDT matching and ground constraints for ground robots in complex environments

PurposeIn large-scale environments and unstructured scenarios, the accuracy and robustness of traditional light detection and ranging (LiDAR) simultaneous localization and mapping (SLAM) algorithms are reduced, and the algorithms might even be completely ineffective. To overcome these problems, this study aims to propose a 3D LiDAR SLAM method for ground-based mobile robots, which uses a 3D LiDAR fusion inertial measurement unit (IMU) to establish an environment map and realize real-time localization.Design/methodology/approachFirst, we use a normal distributions transform (NDT) algorithm based on a local map with a corresponding motion prediction model for point cloud registration in the front-end. Next, point cloud features are tightly coupled with IMU angle constraints, ground constraints and gravity constraints for graph-based optimization in the back-end. Subsequently, the cumulative error is reduced by adding loop closure detection.FindingsThe algorithm is tested using a public data set containing indoor and outdoor scenarios. The results confirm that the proposed algorithm has high accuracy and robustness.Originality/valueTo improve the accuracy and robustness of SLAM, this method proposed in the paper introduced the NDT algorithm in the front-end and designed ground constraints and gravity constraints in the back-end. The proposed method has a satisfactory performance when applied to ground-based mobile robots in complex environments experiments.

Hybrid Navigation System Based Autonomous Positioning and Path Planning for Mobile Robots

Positioning and navigation technology is a new trend of research in mobile robot area. Existing researches focus on the indoor industrial problems, while many application fields are in the outdoor environment, which put forward higher requirements for sensor selection and navigation scheme. In this paper, a complete hybrid navigation system for a class of mobile robots with load tasks and docking tasks is presented. The work can realize large-range autonomous positioning and path planning for mobile robots in unstructured scenarios. The autonomous positioning is achieved by adopting suitable guidance methods to meet different application requirements and accuracy requirements in conditions of different distances. Based on the Bezier curve, a path planning scheme is proposed and a motion controller is designed to make the mobile robot follow the target path. The Kalman filter is established to process the guidance signals and control outputs of the motion controller. Finally, the autonomous positioning and docking experiment are carried out. The results of the research verify the effectiveness of the hybrid navigation, which can be used in autonomous warehousing logistics and multi-mobile robot system.