Laser Imaging Detection And Ranging Research Articles

Bayesian modeling of incompatible spatial data: A case study involving Post-Adrian storm forest damage assessment

Modeling incompatible spatial data, i.e., data with different spatial resolutions, is a pervasive challenge in remote sensing data analysis. Typical approaches to addressing this challenge aggregate information to a common coarse resolution, i.e., compatible resolutions, prior to modeling. Such pre-processing aggregation simplifies analysis, but potentially causes information loss and hence compromised inference and predictive performance. To avoid losing potential information provided by finer spatial resolution data and improve predictive performance, we propose a new Bayesian method that constructs a latent spatial process model at the finest spatial resolution. This model is tailored to settings where the outcome variable is measured on a coarser spatial resolution than predictor variables—a configuration seen increasingly when high spatial resolution remotely sensed predictors are used in analysis. A key contribution of this work is an efficient algorithm that enables full Bayesian inference using finer resolution data while optimizing computational and storage costs. The proposed method is applied to a forest damage assessment for the 2018 Adrian storm in Carinthia, Austria, that uses high-resolution laser imaging detection and ranging (LiDAR) measurements and relatively coarse resolution forest inventory measurements. Extensive simulation studies demonstrate the proposed approach substantially improves inference for small prediction units.

Noise resilient and accurate target detection using fractional-order Fourier domain correlation

Quantum enhanced optical target detection provides a unique route to increased noise resilience of classical LiDARs (laser imaging, detection, and ranging) by using time correlation of non-classical photon pairs. Such enhancement is dictated by the detector temporal uncertainty that is typically orders of magnitude larger than the intrinsic correlation time. To circumvent such detector limitation, we explore the possibility of measuring correlation in the fractional-order Fourier domain (FrFD), which can be realized with the non-local dispersion cancellation. Experimentally, we verify this principle using a fiber-coupled waveguide source of photon pairs, showing enhanced noise rejection as compared with conventional time-domain coincidence detection and classical intensity detection. For false alarm rates of 10−9, an 89 dB improved detection rate is measured using receiver operating characteristics when comparing our FrFD protocol with classical intensity detection. Additionally, we discussed the resilience of FrFD correlation against intentionally prepared counterfeit signal photons. The possibility enabled by measuring correlation in the FrFD should also provide potential benefit for various sensing and communication protocols that relies on coincidence detection.

Mapping Foliar C, N, and P Concentrations in An Ecological Restoration Area with Mixed Plant Communities Based on LiDAR and Hyperspectral Data

Interactions between carbon (C), nitrogen (N), and phosphorus (P), the vital indicators of ecological restoration, play an important role in signaling the health of ecosystems. Rapidly and accurately mapping foliar C, N, and P is essential for interpreting community structure, nutrient limitation, and primary production during ecosystem recovery. However, research on how to rapidly map C, N, and P in restored areas with mixed plant communities is limited. This study employed laser imaging, detection, and ranging (LiDAR) and hyperspectral data to extract spectral, textural, and height features of vegetation as well as vegetation indices and structural parameters. Causal band, multiple linear regression, and random forest models were developed and tested in a restored area in northern China. Important parameters were identified including (1), for C, red-edge bands, canopy height, and vegetation structure; for N, textural features, height percentile of 40–95%, and vegetation structure; for P, spectral features, height percentile of 80%, and 1 m foliage height diversity. (2) R2 was used to compare the accuracy of the three models as follows: R2 values for C were 0.07, 0.42, and 0.56, for N they were 0.20, 0.48, and 0.53, and for P they were 0.32, 0.39, and 0.44; the random forest model demonstrated the highest accuracy. (3) The accuracy of the concentration estimates could be ranked as C > N > P. (4) The inclusion of LiDAR features significantly improved the accuracy of the C concentration estimation, with increases of 22.20% and 47.30% in the multiple linear regression and random forest models, respectively, although the inclusion of LiDAR features did not notably enhance the accuracy of the N and P concentration estimates. Therefore, LiDAR and hyperspectral data can be used to effectively map C, N, and P concentrations in a mixed plant community in a restored area, revealing their heterogeneity in terms of species and spatial distribution. Future efforts should involve the use of hyperspectral data with additional bands and a more detailed classification of plant communities. The application of this information will be useful for analyzing C, N, and P limitations, and for planning for the maintenance of restored plant communities.

Performance Evaluation of Onboard Processing Capability Reduction in Cooperative Vehicles Using 5G and Artificial Intelligence

Fifth-generation (5G) technology is one of the keys to the Industrial Revolution known as Industry 4.0 as it provides faster connectivity and allows a greater number of devices to be connected simultaneously. In the transport sector, newly produced vehicles are equipped with various sensors and applications to help drivers perform safe maneuvers. However, moving from semiautonomous to fully autonomous vehicles to cooperating systems remains a major challenge. Many researchers have focused on artificial intelligence (AI) techniques and the ability to share information to achieve this cooperative behavior. This information can be made up of different data, which can be obtained from different sensors such as laser imaging detection and ranging (LiDAR), radar, camera, global positioning system (GPS), or data related to the current speed, acceleration, or position. The combination of the different shared data is performed depending on the approach of each navigation algorithm. This data fusion will allow a better understanding of the environment but will overload the network, as the traffic generated will be massive. Therefore, this paper addresses the challenge of achieving this cooperation between vehicles from the point of view of network requirements and computational capacity. In addition, this study contributes to advancing theory into real-world practice by examining the performance of cooperative navigation algorithms in the midst of the migration of computational resources from onboard vehicle equipment to the cloud. In particular, it investigates the transition from a cooperative navigation algorithm based on a decentralized architecture to a semidecentralized one as computationally demanding processes previously performed onboard are performed in the cloud. Additionally, the paper discusses the indispensable role of 5G in fulfilling the escalating demands for high throughput and low latency in these services, particularly as the number of vehicles increases. The results of the tests show that the AI acting alone cannot achieve optimal performance, even using 100% of the computational capacity of the onboard equipment in the vehicle. However, a system that integrates 5G and AI-based joint decisions can achieve better performance, reduce the computational resources consumed in the vehicle, and increase the efficiency of collaborative choices by up to 83.3%.

Feasibility of Using Laser Imaging Detection and Ranging Technology for Contactless 3D Body Scanning and Anthropometric Assessment of Athletes.

The scope of this pilot study was to assess the feasibility of using the laser imaging detection and ranging (LiDAR) technology for contactless 3D body scanning of sports athletes and deriving anthropometric measurements of the lower limbs using available software. An Apple iPad Pro 3rd Generation with embedded LiDAR technology in combination with the iOS application Polycam were used. The effects of stance width, clothing, background, lighting, scan distance and measurement speed were initially assessed by scanning the lower limbs of one test person multiple times. Following these tests, the lower limbs of 12 male and 10 female participants were scanned. The resulting scans of the lower limbs were complete for half of the participants and categorized as good in quality, while the other scans were either distorted or presented missing data around the shank and/or the thigh. Bland-Altman plots between the LiDAR-based and manual anthropometric measures showed good agreement, with the coefficient of determination from correlation analysis being R2 = 0.901 for thigh length and R2 = 0.830 for shank length, respectively. The outcome of this pilot study is considered promising, and a further refinement of the proposed scanning protocol and advancement of available software for 3D reconstruction are recommended to exploit the full potential of the LiDAR technology for the contactless anthropometric assessment of athletes.

LiDAR point clouds analysis computer tools for teaching autonomous vehicles perception algorithms

AbstractThe technological developments behind autonomous vehicles cover several areas and engineers training in this field represents a challenge. The main layers include perception, decision making, and acting. In the first one, different technologies can be used. The processing of the information provided by the sensors must allow successive modules to understand the environment and Laser imaging Detection and Ranging (LiDAR) technology is one of the most promising ones nowadays for this task. It offers great robustness in detection, but the extraction of information from the point cloud involves the development of complex algorithms that could be very time‐consuming if an experimental teaching is intended. This article presents two educational solutions for deepening in perception algorithms using LiDAR for autonomous driving: a closed ad‐hoc computer application for two‐dimensional (2D) LiDAR point cloud processing and an oriented set of commands for three‐dimensional (3D) LiDARs in Matlab. Their use allows main concept exploration in practical sessions with little time consumption and provides students a general overview of the tasks that must be performed by the perception layer in the autonomous vehicles. Furthermore, these tools provide the possibility of organizing different activities in the classroom related to theoretical and experimental issues, and understanding of results because the most tedious tasks are eased.

Indexing the Maintenance Priority of Road Safety Barriers in Urban and Peri-Urban Contexts: Application of a Ranking Methodology in Bologna, Italy

The need for clear and updated information is pivotal when authorities plan and perform routinary, periodic and emergency maintenance of both road network and their roadside assets, e.g., curbs, signals, and barriers. With particular regard to road barriers, the development of remote sensing technologies, such as Laser Imaging Detection and Ranging (LiDAR), has played a disruptive role in acquiring information, so the surveys today are predominantly automatic, faster, and less biased than the traditional (i.e., visual and manual) inventorying methodologies. However, even though they are accurate, these emerging procedures usually focus only on the surveyed elements and do not provide any other information about the surrounding environment or about the qualitative degradation of the elements. The primary objective of this research effort was to present a ranking methodology for enhancing road safety in urban contexts. Due to an innovative synthetic index which takes into account both the deterioration and the location of the surveyed elements, maintenance priority of road barriers was outlined in Bologna, Italy. All the collected information was georeferenced in a Geographic Information System (GIS) environment and hence plotted in thematic maps for an easier analysis. In addition, compliance to the norm was verified. The research was tested to provide public authorities with an effective tool in the evaluation of maintenance activities and road safety policies.

A human-centric system combining smartwatch and LiDAR data to assess the risk of musculoskeletal disorders and improve ergonomics of Industry 5.0 manufacturing workers

More than one in four workers reportedly suffer from back pain worldwide, leading to 264 million work days lost yearly. In the U.S. alone, it causes $50 billion in healthcare costs every year, up to $100 billion if including decreased productivity and lost wages. The upcoming Industry 5.0 revolution will introduce human-centric manufacturing systems where workers’ well-being comes first while safeguarding their rights: privacy, autonomy, and human dignity. This paper presents a privacy-preserving system based on artificial intelligence that tracks the posture of assembly/disassembly line workers while performing typical standardized tasks on repeat: connecting and separating parts; screwing and unscrewing using an electric screwdriver; tin soldering and unsoldering. The proposed solution assesses the upper-body (dominant arm, dominant shoulder, and trunk) and lower-body (legs) postures according to the ISO 11226 European standard, based on inertial data recorded by a smartwatch and using Laser imaging Detection and Ranging (LiDAR), respectively. These techniques preserve privacy as they collect data that cannot reveal the worker’s identity or sensitive information. Experiments showed that the system recognized a worker’s posture with a mean accuracy close to 98%. The system could help detect bad posture habits and reduce the chances of musculoskeletal disorders while preserving the workers’ privacy, in compliance with the upcoming Industry 5.0 paradigm.

INTEGRATION OF IPHONE LiDAR WITH QUADCOPTER AND FIXED WING UAV PHOTOGRAMMETRY FOR THE FORESTRY APPLICATIONS

Abstract. The recent innovations in remote sensing technologies have given rise to the efficient mapping and monitoring of forests. The developments in the sensor implementation have mainly focused on optimizing the payload of the UAV system and allowed the users to acquire the data simultaneously with a range of active and passive sensors like high-resolution RGB cameras and multispectral cameras LiDAR (Laser Imaging Detection and Ranging). The main objective of this research contribution is to combine the Digital Elevation Model (DEMs) from quadcopter Unmanned Aerial Vehicles (UAVs), Fixed Wing UAV-based cameras, and iPhone datasets for the forest plots. The datasets from two vegetation seasons, namely leaf-off and leaf-on, were used to combine the Digital Elevation Models from different data acquisition platforms. This internship research work aims to create and experiment with new methods, techniques, and technologies for the applications of UAV photogrammetry and iPhone LiDAR in forest napping and inventory management. CHMs are also generated in this work which helps assess the conditions of the forests in the recreational areas, and the possibility of solutions like iPhone LiDAR and UAV photogrammetry would be highly efficient and economical. The leaf-off and leaf-on datasets were processed in Agisoft Metashape Professional software to generate dense point clouds for the forest plots. The point cloud from the leaf-on dataset was rasterized to generate a DSM whereas the leaf-off point cloud generated a DSM of the forest plots after ground filtering with Cloth Simulation Filter (CSF) plugin. The iPhone LiDAR point was also rasterized to a DTM product after pre-processing steps and noise removal. The Canopy Height Models (CHMs) were generated by subtracting UAV and iPhone LiDAR based DTMs from the UAV leaf on DSM. Finally, the accuracy assessment of CHMs from UAB datasets and their integration with iPhone LiDAR has been assessed using the accurate tree heights measured during the forest field visits. The proposed methodology can be used for forest mapping purposes where a moderate accuracy is requested.

Enhancing Maritime Safety and Efficiency: A Comprehensive Sea Fog Monitoring System for Ningbo Zhoushan Port

Sea fog poses a considerable challenge to port operations, impacting maritime safety and efficiency. During the past five years, the average annual downtime of the navigation dispatch department in Ningbo Zhoushan Port due to weather was 800–1000 h, of which approximately 300 h can be attributed to sea fog. This study addresses the issue by developing a comprehensive sea fog monitoring system for Ningbo Zhoushan Port. The system utilizes automatic weather stations (AWS) and visibility laser imaging, detection, and ranging (LIDAR) to assess sea fog severity and improve monitoring accuracy. By increasing monitoring frequency and adopting corresponding warning measures, the system aims to enhance maritime safety and efficiency in Ningbo Zhoushan Port. The results showed that the implemented system successfully determines sea fog severity, enables real-time monitoring, and provides precise visibility assessments. Joint assessments revealed a substantial increase in the annual operating time and revenue of the port. These findings underscore the importance of advanced monitoring techniques in optimizing port operations, reducing collision risks, and mitigating economic losses caused by sea fog.

A LiDAR-Camera-Inertial-GNSS Apparatus for 3D Multimodal Dataset Collection in Woodland Scenarios.

Forestry operations have become of great importance for a sustainable environment in the past few decades due to the increasing toll induced by rural abandonment and climate change. Robotics presents a promising solution to this problem; however, gathering the necessary data for developing and testing algorithms can be challenging. This work proposes a portable multi-sensor apparatus to collect relevant data generated by several onboard sensors. The system incorporates Laser Imaging, Detection and Ranging (LiDAR), two stereo depth cameras and a dedicated inertial measurement unit (IMU) to obtain environmental data, which are coupled with an Android app that extracts Global Navigation Satellite System (GNSS) information from a cell phone. Acquired data can then be used for a myriad of perception-based applications, such as localization and mapping, flammable material identification, traversability analysis, path planning and/or semantic segmentation toward (semi-)automated forestry actuation. The modular architecture proposed is built on Robot Operating System (ROS) and Docker to facilitate data collection and the upgradability of the system. We validate the apparatus' effectiveness in collecting datasets and its flexibility by carrying out a case study for Simultaneous Localization and Mapping (SLAM) in a challenging woodland environment, thus allowing us to compare fundamentally different methods with the multimodal system proposed.

Unmanned aerial vehicle navigation in underground structure inspection: A review

Many years after construction, a number of existing old tunnels and underground structures are deteriorating with time as evidenced by cracks, large deformations, water leakage and so forth, which usually require regular site inspections to record their structural deterioration by taking high‐pixel, high‐overlap images along miles of a tunnel network. For complex underground structures (e.g., long tunnels and large caves), unmanned aerial vehicles (UAVs) may be adaptive in acquiring images at multiple heights and angles with low operational costs. So far, UAV underground structural health monitoring has become mature for open‐air surveying with rapid developments in robotic software and hardware. However, the UAV image acquisition for underground working conditions still faces a number of key challenges. This paper aims to provide an overview of UAV navigation techniques in confined dark spaces for geotechnical engineers, geologists, drone developers and other interdisciplinary researchers & professionals in the structural health monitoring field. It specifies the challenges for UAV application in underground space, mainly including lack of Global Navigation Satellite System (GNSS) signals, poor lighting conditions, weak features and obstacle avoidance and then followed by strategic solutions. For example, in light of poor GNSS signals, the fusion of multi‐sensors (e.g., laser imaging, detection and ranging (LiDAR) and multi‐cameras) can enhance localization accuracy in low‐luminance underground conditions. To address obstacle avoidance, computer vision (CV)‐based navigation algorithms (e.g., deep reinforced learning [DRL]) enable effective navigation in complex 3D spaces, but their adaptability is limited by arithmetic power and pre‐training needs. The review of relevant previous studies concludes that further development for UAVs in underground space inspection may focus on operation in large‐scale geometric inspection environments, obstacle avoidance, features and semantic recognition.

Ackerman Unmanned Mobile Vehicle Based on Heterogeneous Sensor in Navigation Control Application.

With the advancement of science and technology, the development and application of unmanned mobile vehicles (UMVs) have emerged as topics of crucial concern in the global industry. The development goals and directions of UMVs vary according to their industrial uses, which include navigation, autonomous driving, and environmental recognition; these uses have become the priority development goals of researchers in various fields. UMVs employ sensors to collect environmental data for environmental analysis and path planning. However, the analysis function of a single sensor is generally affected by natural environmental factors, resulting in poor identification results. Therefore, this study introduces fusion technology that employs heterogeneous sensors in the Ackerman UMV, leveraging the advantages of each sensor to enhance accuracy and stability in environmental detection and identification. This study proposes a fusion technique involving heterogeneous imaging and LiDAR (laser imaging, detection, and ranging) sensors in an Ackerman UMV. A camera is used to obtain real-time images, and YOLOv4-tiny and simple online real-time tracking are then employed to detect the location of objects and conduct object classification and object tracking. LiDAR is simultaneously used to obtain real-time distance information of detected objects. An inertial measurement unit is used to gather odometry information to determine the position of the Ackerman UMV. Static maps are created using simultaneous localization and mapping. When the user commands the Ackerman UMV to move to the target point, the vehicle control center composed of the robot operating system activates the navigation function through the navigation control module. The Ackerman UMV can reach the destination and instantly identify obstacles and pedestrians when in motion.

Error Covariance Estimation of 3D Point Cloud Registration Considering Surrounding Environment

To realize autonomous vehicle safety, it is important to accurately estimate the vehicle’s pose. As one of the localization techniques, 3D point cloud registration is commonly used. However, pose errors are likely to occur when there are few features in the surrounding environment. Although many studies have been conducted on estimating error distribution of 3D point cloud registration, the real environment is not reflected. This paper presents real-time error covariance estimation in 3D point cloud registration according to the surrounding environment. The proposed method provides multiple initial poses for iterative optimization in the registration method. Using converged poses in multiple searches, the error covariance reflecting the real environment is obtained. However, the initial poses were limited to directions in which the pose error was likely to occur. Hence, the limited search efficiently determined local optima of the registration. In addition, the process was conducted within 10 Hz, which is laser imaging detection and ranging (LiDAR) period; however, the execution time exceeded 100 ms in some places. Therefore, further improvement is necessary.

Development of Three-Dimensional Breast Scan and Measurement Application Using Laser Imaging Detection and Ranging Sensor on iPhone.

Laser imaging detection and ranging (LiDAR) is a modern three-dimensional (3D) technology that uses a time-of-flight method based on the round-trip time of an infrared laser beam to detect the presence and features of objects. The iPhone 12 Pro is the first smart mobile device with built-in LiDAR sensors. The authors' team developed a software application based on iOS devices with built-in LiDAR sensors for 3D breast scanning and automatically analyzing the breast's geometric measurement. Breast geometry, including midclavicle-to-nipple distance, sternal notch-to-nipple distance, nipple-to-inframammary fold (IMF) distance, distance between nipples, and body circumference on nipple and IMF level were measured using the software application and tapeline. The relative technical error of measurement (rTEM) value was used to calculate the error ratios between the measurements acquired by the software application and those of the tapeline. Good rTEM values ranging from 2.99% to 5.19% were found in the midclavicle-to-nipple distance, sternal notch-to-nipple distance, distance between nipples, nipple-level circumference, and IMF-level circumference. However, there was a poor rTEM value greater than 10% in the nipple-to-IMF distance. The proposed software application using current iOS devices with built-in LiDAR sensors can provide an ideal 3D scanning system that has a low cost burden, good accuracy, portability, and ease of use.

Real-Time Indoor Path Planning Using Object Detection for Autonomous Flying Robots

Unknown closed spaces are a big challenge for the navigation of robots since there are no global and pre-defined positioning options in the area. One of the simplest and most efficient algorithms, the artificial potential field algorithm (APF), may provide real-time navigation in those places but fall into local minimum in some cases. To overcome this problem and to present alternative escape routes for a robot, possible crossing points in buildings may be detected by using object detection and included in the path planning algorithm. This study utilized a proposed sensor fusion method and an improved object classification method for detecting windows, doors, and stairs in buildings and these objects were classified as valid or invalid for the path planning algorithm. The performance of the approach was evaluated in a simulated environment with a quadrotor that was equipped with camera and laser imaging detection and ranging (LIDAR) sensors to navigate through an unknown closed space and reach a desired goal point. Inclusion of crossing points allows the robot to escape from areas where it is congested. The navigation of the robot has been tested in different scenarios based on the proposed path planning algorithm and compared with other improved APF methods. The results showed that the improved APF methods and the methods reinforced with other path planning algorithms were similar in performance with the proposed method for the same goals in the same room. For the goals outside the current room, traditional APF methods were quite unsuccessful in reaching the goals. Even though improved methods were able to reach some outside targets, the proposed method gave approximately 17% better results than the most successful example in achieving targets outside the current room. The proposed method can also work in real-time to discover a building and navigate between rooms.

The MPU RX-4 Project: Design, Electronics, and Software Development of a Geofence Protection System for a Fixed-Wing VTOL UAV

Unmanned aerial vehicles (UAVs) are at the forefront of this century’s technological shift, becoming ubiquitous in research and market areas. However, as a UAV navigates autonomously, there are unanticipated occasions, e.g., collisions with dynamic obstacles or loss of data provided by the global navigation satellite system (GNSS), where the aircraft has to change its mission plans. In particular, to be protected from possible accidents, the platform’s geofence protection system should adjust the trajectory appropriately when obstacles are detected and select the proper ground surface when emergency landing is prompted. As these processes require fast reaction times, utilizing low latency sensors and algorithms is necessary. This article proposes a complete and low-complexity geofence protection system for recognizing moving objects and assessing the ground surface’s suitability using onboard sensing and processing modules. The proposed system is implemented on a novel fixed-wing UAV, designated as MPU RX-4, which features an unconventional flying wing layout and vertical take-off and landing (VTOL) capabilities. Our system is based on a forward-facing laser imaging, detection, and ranging (LIDAR) sensor and three downward-facing laser rangefinders. We take advantage of the high-precision distance measurements and operational speed to identify moving obstacles using the LIDAR module, while the ground’s slope and the existence of any obstacle therein are computed through the rangefinders. First, the article describes the UAV design procedure and its aerodynamic performance characteristics, which allowed us to evaluate our approach on a testbed aircraft. Then, the evaluation protocol shows that our system can perform robustly and under real-time constraints reaching an overall latency of only 165.5 ms, sufficient for reliable detection and avoidance of moving obstacles.

Location Recognition of Indoor Firefighting Facilities based on RGB-D Camera and 3D LiDAR

Recently, as fires frequently occur in large buildings, digital twin (DT) technology that enables remote and real-time monitoring and control similar to the real world environment is being studied as a disaster response technology in buildings. In order to use DT technology, it is essential to collect the spatial data of actual building indoor environments and firefighting facilities. This study proposes an indoor spatial data collecting system that can generate the modeling data inside the building and location data of firefighting facilities using laser imaging detection and ranging (LiDAR) and RGB-D cameras. First, point clouds from three-dimensional (3D) LiDAR and the FAST-LIO2 (Fast LiDAR-Inertial Odometry) algorithm are used to obtain odometry information in an indoor environment. The firefighting facilities located inside the building are detected using RGB images and the deep learning model Faster region-based convolutional neural network (R-CNN) with Inception V2 architecture trained using RGB images of four types of firefighting facilities: fire extinguishers, fire hydrants, exit signs, and fire detectors. When a firefighting facility is detected, the relative distance between the RGB-D camera and the firefighting facility is calculated through the depth image and intrinsic parameters of the RGB-D camera. Afterwards, odometry information obtained from FAST-LIO2 and the relative distance are combined to obtain the 3D location of the firefighting facility. Point clouds of the FAST-LIO2 algorithm are then converted into models of the building indoor environment. Through this method, spatial data of an actual building can be constructed and used with DT technology.

Inter-row information recognition of maize in the middle and late stages via LiDAR supplementary vision.

In the middle and late stages of maize, light is limited and non-maize obstacles exist. When a plant protection robot uses the traditional visual navigation method to obtain navigation information, some information will be missing. Therefore, this paper proposed a method using LiDAR (laser imaging, detection and ranging) point cloud data to supplement machine vision data for recognizing inter-row information in the middle and late stages of maize. Firstly, we improved the YOLOv5 (You Only Look Once, version 5) algorithm based on the characteristics of the actual maize inter-row environment in the middle and late stages by introducing MobileNetv2 and ECANet. Compared with that of YOLOv5, the frame rate of the improved YOLOv5 (Im-YOLOv5) increased by 17.91% and the weight size decreased by 55.56% when the average accuracy was reduced by only 0.35%, improving the detection performance and shortening the time of model reasoning. Secondly, we identified obstacles (such as stones and clods) between the rows using the LiDAR point cloud data to obtain auxiliary navigation information. Thirdly, the auxiliary navigation information was used to supplement the visual information, so that not only the recognition accuracy of the inter-row navigation information in the middle and late stages of maize was improved but also the basis of the stable and efficient operation of the inter-row plant protection robot was provided for these stages. The experimental results from a data acquisition robot equipped with a camera and a LiDAR sensor are presented to show the efficacy and remarkable performance of the proposed method.

Trainable Quaternion Extended Kalman Filter with Multi-Head Attention for Dead Reckoning in Autonomous Ground Vehicles.

Extended Kalman filter (EKF) is one of the most widely used Bayesian estimation methods in the optimal control area. Recent works on mobile robot control and transportation systems have applied various EKF methods, especially for localization. However, it is difficult to obtain adequate and reliable process-noise and measurement-noise models due to the complex and dynamic surrounding environments and sensor uncertainty. Generally, the default noise values of the sensors are provided by the manufacturer, but the values may frequently change depending on the environment. Thus, this paper mainly focuses on designing a highly accurate trainable EKF-based localization framework using inertial measurement units (IMUs) for the autonomous ground vehicle (AGV) with dead reckoning, with the goal of fusing it with a laser imaging, detection, and ranging (LiDAR) sensor-based simultaneous localization and mapping (SLAM) estimation for enhancing the performance. Convolution neural networks (CNNs), backward propagation algorithms, and gradient descent methods are implemented in the system to optimize the parameters in our framework. Furthermore, we develop a unique cost function for training the models to improve EKF accuracy. The proposed work is general and applicable to diverse IMU-aided robot localization models.