Autonomous Land Vehicle Research Articles

LTI-SAM: Lidar-Template Matching-Inertial Odometry via Smoothing and Mapping

Robust localization is an essential requirement for autonomous land vehicles. In similar scenes such as corridors and tunnels or open scenes such as airports and mines, the performance of the traditional algorithm of laser odometry will be seriously deteriorated or even become completely invalid due to the similarity of observation data. Based on the LIO-SAM algorithm, LTI-SAM is proposed in this article, that is to use the downward-facing camera and inertial measurement unit (IMU) to perform dead reckoning through template matching algorithm and use template matching visual odometry (VO) to judge themotion state and decidewhether to save the keyframe if the laser odometer fails and the keyframe is not saved. If the keyframe is kept, theVOis added between two keyframes instead of the laser odometry to optimize through the factor graph, which improves the accuracy of localization and mapping. The proposed algorithm with loose coupling between template matching VO and laser inertial navigation is implemented under the robot operating system (ROS) platform, and the algorithm is verified by indoor and outdoor experiments. Experimental results show that the proposed algorithm can bring significant performance improvement in both similar indoor scenes and outdoor degraded scenes.

Autonomous land vehicle path planning algorithm based on improved heuristic function of A-Star

The path planning of autonomous land vehicle has become a research hotspot in recent years. In this article, we present a novel path planning algorithm for an autonomous land vehicle. According to the characteristics of autonomous movement towards the autonomous land vehicle, an improved A-Star path planning algorithm is designed. The disadvantages of using the A-Star algorithm for path planning are that the path planned by the A-Star algorithm contains many unnecessary turning points and is not smooth enough. Autonomous land vehicle needs to adjust its posture at each turning point, which will greatly waste time and also will not be conducive to the motion control of autonomous land vehicle. In view of these shortcomings, this article proposes a new heuristic function combined with the artificial potential field method, which contains both distance information and obstacle information. Our proposed algorithm shows excellent performance in improving the execution efficiency and reducing the number of turning points. The simulation results show that the proposed algorithm, compared with the traditional A-Star algorithm, makes the path smoother and makes the autonomous land vehicle easier to control.

LiDAR‐based robust localization for field autonomous vehicles in off‐road environments

AbstractRobust localization is an essential capability for autonomous land vehicles. While a lot of work focused on structured environments, this article focuses on navigation in off‐road environments. In the off‐road environment, due to the lack of salient features, scan matching algorithms tend to degenerate. Therefore, the first contribution of this paper is to propose a reliable degeneracy indicator which can evaluate the scan matching performance. The evaluated degeneracy indicator is then integrated into the factor graph optimization framework which is used in both the offline mapping system and the online localization system. Moreover, a complete navigation system that can handle the incomplete and partly outdated LiDAR maps is developed. Extensive tests on real‐world data sets show that the proposed system outperforms state‐of‐the‐art approaches, especially in degenerate scenarios.

Design of Agricultural Autonomous Land Vehicles for Monitoring and Control

The world population is growing and it is necessary to take advantage of agricultural natural resources. According to FAO studies, undernourishment is increasing in almost all sub-regions of Africa, as well as in South America. The situation worsens in countries whose agricultural systems are extremely sensitive to rainfall, temperature and droughts. On the other hand, contamination by pesticides and herbicides generates a series of environmental problems. In addition, the insects and parasites themselves develop more resistant species, which forces the use of higher doses or more effective products. Due to the rains and flooding of the fields, the chemicals are transmitted to the rivers, contaminating the water, vegetation, livestock and people. There are unmanned aerial vehicles known as drones to monitor fields, but they are expensive and require qualified personnel. In this work, a low-cost and reduced-size autonomous land vehicle (car) is designed to monitor fields. An electronic nose is mounted with 7 low consumption metal oxide semiconductor (MOS) gas sensors and batteries. The size and scope of the vehicle can be changed by modifying the structure of the car using a 3D printer, changing batteries and motors. Solar panels can also be added for extended use. The electronic nose weighs only 50 grams, has small dimensions, is low cost, has an embedded system and proprietary software. The software stores the data in memory, using artificial intelligence, different odors can be recognized in a field. In this way, a map of the monitored terrain is assembled and it can be fumigated in a controlled manner, selecting different doses as necessary. This system is expected to reduce environmental pollution and improve agricultural production.Key Words: Autonomous Vehicle, Agriculture, Electronic Nose, Data Analysis.

Double Deep Q-Learning and Faster R-CNN-Based Autonomous Vehicle Navigation and Obstacle Avoidance in Dynamic Environment.

Autonomous vehicle navigation in an unknown dynamic environment is crucial for both supervised- and Reinforcement Learning-based autonomous maneuvering. The cooperative fusion of these two learning approaches has the potential to be an effective mechanism to tackle indefinite environmental dynamics. Most of the state-of-the-art autonomous vehicle navigation systems are trained on a specific mapped model with familiar environmental dynamics. However, this research focuses on the cooperative fusion of supervised and Reinforcement Learning technologies for autonomous navigation of land vehicles in a dynamic and unknown environment. The Faster R-CNN, a supervised learning approach, identifies the ambient environmental obstacles for untroubled maneuver of the autonomous vehicle. Whereas, the training policies of Double Deep Q-Learning, a Reinforcement Learning approach, enable the autonomous agent to learn effective navigation decisions form the dynamic environment. The proposed model is primarily tested in a gaming environment similar to the real-world. It exhibits the overall efficiency and effectiveness in the maneuver of autonomous land vehicles.

A novel robust event‐triggered fault tolerant automatic steering control approach of autonomous land vehicles under in‐vehicle network delay

AbstractThis paper proposes a novel robust event‐triggered fault tolerant automatic steering control strategy for autonomous land vehicles to achieve path tracking and vehicle lateral motion control under in‐vehicle network delay. From the practical point of view, the parameter uncertainties, time delay, and actuator fault are simultaneously introduced to make the designed controller robust to more extensive and challenging driving conditions. A novel polytope reduction and norm‐bounded uncertainty reduction method is used to effectively handle the time‐varying velocity and tire cornering stiffness uncertainties. Then, the uncertain vehicle model can be reconstructed as an uncertain network control system model with time delay and actuator fault. Due to the inevitable time delay and actuator fault, a new cubic absolute‐value Lyapunov function is developed to guarantee the asymptotical stability of the closed control system with the H∞ performance, and the modified project‐based adaptive law is applied to strengthen the fault tolerant ability. Furthermore, a progressive event‐triggered mechanism is proposed for the collaborative design of robust fault tolerant automatic steering controller, which can signally improve the communication resource utilization of the limited bandwidth in‐vehicle network. Finally, the performance comparisons of different controllers are presented. These results prove that the proposed controller can simultaneously guarantee the path tacking performance and lateral dynamics stability, and save the communication resource of the in‐vehicle network.

Road Detection Method based on Online Learning

Road detection is always the key problem of re-searches on areas of unmanned ground vehicle and computer vision. A road detection method is proposed based on online learning and multi-sensor fusion. First of all, the Lidar point clouds are projected onto the images via the joint calibration of these two kinds of sensors. Then Simple Linear Iterative Clustering is used to segment images into many superpixles. Based on that, a multilayer online learning method is proposed, in which 2 Support Vector Machines are trained to detect the road. To be specific, the superpixel layer Support Vector Machine is used to detect road roughly, and the pixel layer Support Vector Machine is then trained to classify the edge pixels of the road areas, which is classified by the upper-layer Support Vector Machine. These 2 Support Vector Machines are updated online at each frame to be adapted to the changing environment. At last, some experiments are carried out on KITTI RAW dataset and an autonomous land vehicle, and the results show the effectiveness of proposed method. The main contributions of this work lie on as follows: 1) a multilayer learning model is proposed to detect road more robustly and accurately; 2) an online learning method is proposed which can be adapted to the changing environment.

Deep learning for privacy preservation in autonomous moving platforms enhanced 5G heterogeneous networks

5G heterogeneous networks have become a promising platform to connect a growing number of Internet-of-Things (IoT) devices and accommodate a wide variety of vertical services. IoT has not been limited to traditional sensing systems since the introduction of 5G, but also includes a range of autonomous moving platforms, e.g., autonomous flying vehicles, autonomous underwater vehicles, autonomous surface vehicles as well as autonomous land vehicles. These platforms can be used as an effective means to connect air, space, ground, and sea mobile networks for providing a wider diversity of Internet services. Deep learning has been widely used to extract useful information from network big data for enhancing network quality-of-service and user quality-of-experience. Privacy preservation for user and network data is a burning concern in 5G heterogeneous networks due to various attacks in this environment. In this paper, we conduct an in-depth investigation on how deep learning can cope with privacy preservation issues in 5G heterogeneous networks, in terms of heterogeneous radio access networks (RANs), beyond-RAN networks, and end-to-end network slices, followed by a set of key research challenges and open issues that aim to guide future research.

An improved A-Star based path planning algorithm for autonomous land vehicles

This article presents a novel path planning algorithm for autonomous land vehicles. There are four main contributions: Firstly, an evaluation standard is introduced to measure the performance of different algorithms and to select appropriate parameters for the proposed algorithm. Secondly, a guideline generated by human or global planning is employed to develop the heuristic function to overcome the shortcoming of traditional A-Star algorithms. Thirdly, for improving the obstacle avoidance performance, key points around the obstacle are employed, which would guide the planning path to avoid the obstacle much earlier than the traditional one. Fourth, a novel variable-step based A-Star algorithm is also introduced to reduce the computing time of the proposed algorithm. Compared with the state-of-the-art techniques, experimental results show that the performance of the proposed algorithm is robust and stable.

Integrated Artificial Intelligence Systems

From Shakey the Robot to self‐driving cars, from the personal computer to personal assistants on our phones, the Defense Advanced Research Projects Agency (DARPA) has led the development of integrated artificial intelligence (AI) systems for more than half a century. From the earliest days of AI, it was apparent that a robust, generally intelligent system should include a complete set of capabilities: perception, memory, reasoning, learning, planning, and action; and when DARPA initiated AI research in the 1960s, ambitious projects such as Shakey the Robot went after the complete package. As DARPA realized the challenges, they backed away from the ultimate goal of integrated AI and tried to make progress on the individual problems of image understanding, speech and language understanding, knowledge representation and reasoning, planning and decision aids, machine learning, and robotic manipulation. Yet, even as researchers struggled to make progress in these subdisciplines, DARPA periodically resurrected the challenge of integrated intelligent systems and pushed the community to try again. In the 1980s, DARPA's Strategic Computing Initiative took on challenges of integrated AI projects such as the Autonomous Land Vehicle and the Pilot's Associate. These did not succeed, but instead set the stage for the several decades of more siloed research that followed, until it was time to try again. In the 2000s, DARPA took on the integrated AI problem again with its Grand Challenges, which led to the first self‐driving cars, and projects such as the Personalized Assistant that Learns, which produced Apple's Siri. These efforts created complex, richly‐integrated systems that represented quantum leaps ahead in machine intelligence. The integration of sophisticated capabilities in a fundamental way is the key to general intelligence. This is the story of DARPA's persistent long‐term support for this essential premise of AI.

LIDAR Scan Matching in Off-Road Environments

Accurately matching the LIDAR scans is a critical step for an Autonomous Land Vehicle (ALV). Whilst most previous works have focused on the urban environment, this paper focuses on the off-road environment. Due to the lack of a publicly available dataset for algorithm comparison, a dataset containing LIDAR pairs with varying amounts of offsets in off-road environments is firstly constructed. Several popular scan matching approaches are then evaluated using this dataset. Results indicate that global approaches, such as Correlative Scan Matching (CSM), perform best on large offset datasets, whilst local scan matching approaches perform better on small offset datasets. To combine the merits of both approaches, a two-stage fusion algorithm is designed. In the first stage, several transformation candidates are sampled from the score map of the CSM algorithm. Local scan matching approaches then start from these transformation candidates to obtain the final results. Four performance indicators are also designed to select the best transformation. Experiments on a real-world dataset prove the effectiveness of the proposed approach.

Novel Mechatronic Sensors for Irregular Movements of Autonomous Land Vehicles

In this paper, novel sensors using physical phenomena deduced from ultrasound waves are proposed to overcome problems of cost, availability and adaptability in the measure of position, velocity and tilt of Autonomous Land Vehicles. The true challenge for the Autonomous Land Vehicles concerns essentially the accurate perception of hazardous environments to receive faithful information about reality. Current sensors used nowadays are viewed very expensive, complicated and unable to give accurate measures without time delay. For that, our contribution is located in this context. At first, fundamental principle of ultrasonic waves and their specific physical properties are described to introduce their use in the proposed sensors. Mechatronic concepts of sensors are then detailed with the corresponding mathematical models. Indeed, online implementation and real time experiences are conducted to verify the effectiveness of measurements.

A Novel Multi-Level Integrated Navigation System for Challenging GNSS Environments

Global Navigation Satellite Systems (GNSS) is utilized to provide route guidance information to land and autonomous vehicles. The GNSS-based positioning and navigation (POS/NAV) usually suffer from satellite signal blockage, interference, and multipath in urban areas. Autonomous land vehicles are equipped with cameras, radars, and laser ranging devices. The availability of these systems provides an attractive opportunity to increase the POS/NAV system accuracy. This research focuses on the development of an integrated multi-sensor POS/NAV system capable of offering seamless positioning for autonomous land vehicles. A new multi-sensor POS/NAV module integrating both adaptive cruise control frequency modulated continuous wave (ACC-FMCW) radar (RAD), and magnetometer measurements with the reduced inertial sensor system (RISS) was designed to update the navigation system during GNSS outages. Augmenting RAD/RISS system with the magnetometer measurements produces a robust solution. The designed system is further improved by utilizing fast orthogonal search (FOS) to provide nonlinear error modeling of the residual errors associated with the RAD/RISS positioning solution in order to reduce the error growth overextended and frequent GNSS outages.The proposed systems were evaluated on several real road test trajectories involving different types of land vehicles experiencing different motion dynamics. GNSS outages of up to 10 minutes were intentionally introduced to examine the performance. The results show that the proposed methods have resulted in a significant performance improvement in the positioning accuracy that can reach more than 80% if compared to the present techniques that rely only on integrating the inertial sensor technology with GNSS.

Rao‐Blackwellised particle filtering for low‐cost encoder/INS/GNSS integrated vehicle navigation with wheel slipping

This study presents a Rao-Blackwellised particle filter (RBPF)-based encoder/inertial navigation system (INS)/global navigation satellite system (GNSS) integration method for improving the navigation performance of an autonomous land vehicle (ALV) with wheel slipping. In contrast to traditional integration methods, the proposed integration method introduces an overall wheel slip consideration for the ALV, which greatly improves the accuracy of the velocity estimation, especially when the inertial sensor is low cost. Additionally, the proposed integrated system uses double-difference pseudorange measurements instead of single point positioning results provided by low-cost GNSS receivers, which greatly improves the accuracy of the position estimation. To verify the navigation performance of the proposed integrated system, comparisons between the states estimated by the proposed system, the EKF-based integrated system and the joint wheel-slip and motion-estimation system are provided. The results of the experiment show that the proposed integrated system has the highest accuracy in both the position estimation and the velocity estimation among the three compared systems, and can improve the navigation performance during GNSS signals outages.

A multimedia stereo calibration algorithm based on rectangular pyramidal method used to aid visual navigation of ALVs under low illumination

In order to measure and reconstruct accurate three-dimension (3D) data for visual aided navigation of autonomous land vehicles (ALVs), a multimedia stereo calibration algorithm which is suitable for normal scene and especially for low illumination scene is proposed. Firstly, an expression of object-point re-projection errors is derived by the collinear equation model, and the non-linear least square algorithm (NLS) is introduced to iteratively optimize external parameters for individual camera. A rectangular pyramidal method enforcing the rectangular geometric constraint is presented, to produce more stable initial parameter values. Then, according to imaging-point correspondences between the left and right camera, a re-projection error model is constructed for this stereo calibration system, of which all parameters are further optimized and calculated through the calibrated results of two separate cameras. Experimental results show that the proposed algorithm can achieve re-projection errors of no more than 0.5 pixels and converge fast usually with less than 10 interation times, whether under normal illumination or low illumination, so it can get better performance and realize a rapid re-calibration.

Parameterized Batch Reinforcement Learning for Longitudinal Control of Autonomous Land Vehicles

This paper presents a parameterized batch reinforcement learning algorithm for near-optimal longitudinal control of autonomous land vehicles (ALVs). The proposed approach uses an actor-critic architecture, where parameterized feature vectors based on kernels are learned from collected samples for approximating the value functions and policies. One difference between the parameterized batch actor-critic (PBAC) algorithm and previous actor-critic learning approaches is that the critic and actor in PBAC share the same linear features, which has been theoretically proved to be a beneficial property for the convergence of actor-critic learning approaches. In order to obtain better learning efficiency, least-squares-based batch updating rules are designed for the critic and actor, respectively. Based on the PBAC learning algorithm, a data-driven longitudinal control method is presented for ALVs to obtain near-optimal control policies which adaptively tune the fuel/brake control signals to track different speeds. A multiobjective reward function is designed so that both tracking precision and driving smoothness are considered. Extensive experiments were conducted on a real ALV platform while driving on flat, slippery, sloping, and bumpy roads. The experimental results illustrate the superiority of the PBAC-based self-learning controller over conventional longitudinal control methods such as proportional-integral (PI) control and learning-based PI control.

Guest Editorial Pushing for Higher Autonomy and Cooperative Behaviors in Maritime Robotics

The papers in this special section examine the technology of maritime robotics. These papers are the result of a collaborative effort between the IEEE Oceanic Engineering Society, the U.S. Office of Naval Research (ONR), and RoboNation, a nonprofit robotics organization formerly known as AUVSI foundation. The field of maritime robotics is swiftly moving toward integration of air, surface, and subsurface autonomous systems. For example, where autonomous maritime systems (AMS) are composed of heterogeneous assets, surface vehicles are now often capable of transporting aerial and underwater vehicles, leveraging the benefits of each to increase mission endurance and capabilities. In seeking to integrate land, sea, and air vehicle systems, it is natural to look toward leveraging advances made separately in each domain. For example, substantial similarities exist between the desired behavioral capabilities of autonomous land vehicles and those of autonomous marine systems. Recent advances in the field of driverless cars may therefore be applicable to autonomous surface vessels, underwater vehicles, and even aerial vehicles.

A hierarchical vision-based localization of rotor unmanned aerial vehicles for autonomous landing

The vision-based localization of rotor unmanned aerial vehicles for autonomous landing is challenging because of the limited detection range. In this article, to extend the vision detection and measurement range, a hierarchical vision-based localization method is proposed for unmanned aerial vehicle autonomous landing. In such a hierarchical framework, the landing is defined into three phases: “Approaching,”“Adjustment,” and “Touchdown,” in which visual artificial features at different scales can be detected from the designed object pattern for unmanned aerial vehicle pose recovery. The corresponding feature detection and pose estimation algorithms are also presented. In the end, typical simulation and field experiments have been carried out to illustrate the proposed method. The results show that our hierarchical vision-based localization has the ability to a consecutive unmanned aerial vehicle localization in a wider working range from far to near, which is significant for autonomous landing.

Deep learning for vision-based micro aerial vehicle autonomous landing

Vision-based techniques are widely used in micro aerial vehicle autonomous landing systems. Existing vision-based autonomous landing schemes tend to detect specific landing landmarks by identifying their straightforward visual features such as shapes and colors. Though efficient to compute, these schemes only apply to landmarks with limited variability and require strict environmental conditions such as consistent lighting. To overcome these limitations, we propose an end-to-end landmark detection system based on a deep convolutional neural network, which not only easily scales up to a larger number of various landmarks but also exhibit robustness to different lighting conditions. Furthermore, we propose a separative implementation strategy which conducts convolutional neural network training and detection on different hardware platforms separately, i.e. a graphics processing unit work station and a micro aerial vehicle on-board system, subject to their specific implementation requirements. To evaluate the performance of our framework, we test it on synthesized scenarios and real-world videos captured by a quadrotor on-board camera. Experimental results validate that the proposed vision-based autonomous landing system is robust to landmark variability in different backgrounds and lighting situations.

Two-Stage Road Terrain Identification Approach for Land Vehicles Using Feature-Based and Markov Random Field Algorithm

Road terrain identification is one of the important tasks for driving assistant systems or autonomous land vehicles. It plays a key role in improving driving strategy and enhancing fuel efficiency. In this paper, a two-stage approach using multiple sensors is presented. In the first stage, a feature-based identification approach is performed using an accelerometer, a camera, and downward-looking and forward-looking laser range finders (LRFs). This produces four classification label sequences. In the second stage, a majority vote is implemented for each label sequences to match them into synchronized road patches. Then a Markov Random Field (MRF) model is designed to generate the final optimized identification results to improve the forward-looking LRF. This approach enables the vehicle to observe the upcoming road terrain before moving onto it by fusing all the classification results using an MRF algorithm. The experiments show this approach improved the terrain identification accuracy and robustness significantly for some familiar road terrains.