Lidar Sensors Research Articles

Space Debris In-Orbit Detection with Commercial Automotive LiDAR Sensors

This article presents an alternative approach to detecting and mapping space debris in low Earth orbit by utilizing commercially available automotive LiDAR sensors mounted on CubeSats. The main objective is to leverage the compact size, low weight, and minimal power consumption of these sensors to create a “Large Cosmic LiDAR” (LCL) system. This LCL system would operate similarly to a giant radar circling the Earth, with strategically positioned LiDAR sensors along the target orbit. The article examines the feasibility of this concept by analyzing the relative orbital velocity between the sensor and debris objects, and calculating the time required to scan a complete orbit.

Color Models in the Process of 3D Digitization of an Artwork for Presentation in a VR Environment of an Art Gallery

This study deals with the color reproduction of a work of art to digitize it into a 3D realistic model. The experiment aims to digitize a work of art for application in a virtual reality environment concerning faithful color reproduction. Photogrammetry and scanning with a LiDAR sensor are used to compare the methods and work with colors during the reconstruction of the 3D model. An innovative tablet with a camera and LiDAR sensor is used for both methods. At the same time, current findings from the field of color vision and colorimetry are applied to 3D reconstruction. The experiment focuses on working with the RGB and L*a*b* color models and, simultaneously, on the sRGB, CIE XYZ, and Rec.2020(HDR) color spaces for transforming colors into a virtual environment. For this purpose, the color is defined in the Hex Color Value format. This experiment is a starting point for further research on color reproduction in the digital environment. This study represents a partial contribution to the much-discussed area of forgeries of works of art in current trends in forensics and forgery.

LiDAR-360 RGB Camera-360 Thermal Camera Targetless Calibration for Dynamic Situations

Integrating multiple types of sensors into autonomous systems, such as cars and robots, has become a widely adopted approach in modern technology. Among these sensors, RGB cameras, thermal cameras, and LiDAR are particularly valued for their ability to provide comprehensive environmental data. However, despite their advantages, current research primarily focuses on the one or combination of two sensors at a time. The full potential of utilizing all three sensors is often neglected. One key challenge is the ego-motion compensation of data in dynamic situations, which results from the rotational nature of the LiDAR sensor, and the blind spots of standard cameras due to their limited field of view. To resolve this problem, this paper proposes a novel method for the simultaneous registration of LiDAR, panoramic RGB cameras, and panoramic thermal cameras in dynamic environments without the need for calibration targets. Initially, essential features from RGB images, thermal data, and LiDAR point clouds are extracted through a novel method, designed to capture significant raw data characteristics. These extracted features then serve as a foundation for ego-motion compensation, optimizing the initial dataset. Subsequently, the raw features can be further refined to enhance calibration accuracy, achieving more precise alignment results. The results of the paper demonstrate the effectiveness of this approach in enhancing multiple sensor calibration compared to other ways. In the case of a high speed of around 9 m/s, some situations can improve the accuracy about 30 percent higher for LiDAR and Camera calibration. The proposed method has the potential to significantly improve the reliability and accuracy of autonomous systems in real-world scenarios, particularly under challenging environmental conditions.

Determining the usability of the ViDoc device, which integrates with smart phones, in documenting historical structures

The documentation of historical structures is considered as a crucial subject in international regulations. The accuracy and speed of documentation are among the significant factors for the qualified transmission of cultural values to future generations. In this study, the accuracy and position precision of ViDoc device, which integrates with an iPad Pro equipped with LIDAR system, have been determined. Furthermore, the advantages and disadvantages of this device have been elaborated in detail. The aim of the study is to determine whether the ViDoc device and iPad Pro can be used in documenting historical structures. The study covers the traditional method of documentation based on measurement with meter and analog devices, documentation with a total station, and documentation with the ViDoc device, using two historical fountains selected as examples. Two cultural heritage fountains located in Kastamonu province, Turkey, have been selected for documentation. Measurements of the fountains were taken using a precision meter and a laser meter in the classical method, and corner point coordinates were obtained with a total station. The measurement process was carried out in one go using a ViDoc device and a smartphone. As a result of the study, it was understood that the ViDoc device made very positive contributions to the scanning accuracy of smart devices with LIDAR sensors. When comparing the measurement values in the documentation of the structures, the accuracy precision was found to be less than 1 cm (0.7 cm), and the positioning accuracy was less than 5 cm. In the measurement conducted without using the ViDoc device, differences in position distances reached up to 79 m. This indicates that the device can be used for documentation in structures without creating disadvantages. However, it has been determined through the study that the device may not be usable for every structure. Nevertheless, the study also evaluates the device as a new alternative alongside traditional documentation, photogrammetry, and terrestrial laser scanning systems. The study is significant as it is the first to holistically determine the measurement accuracy of the device and the positioning accuracy specifically in relation to historical structures.

Machine Vision AI in Railroad Safety: Advanced Inspection Techniques

This article explores the transformative impact of machine vision AI in railroad safety, focusing on advanced inspection techniques. It examines the integration of technologies such as line scan cameras, LiDAR, and IoT sensors in enabling comprehensive 360-degree inspections of railway infrastructure and rolling stock. The article discusses key innovations in continuous monitoring, data processing, and AI-driven analysis, highlighting their potential to significantly enhance defect detection accuracy, reduce maintenance costs, and prevent accidents. Real-world applications in structural integrity assessment and overhead line inspection are presented, along with an analysis of current challenges and future developments in the field. The article underscores the paradigm shift from reactive to proactive maintenance strategies, promising a safer and more efficient future for rail transport.

Decentralized fault-tolerant control of multi-mobile robot system addressing LiDAR sensor faults

The control of multi-robot formations is a crucial aspect in various applications, such as transport, surveillance and monitoring environments. Maintaining robots in a specific formation pose or performing a cooperative task is a significant challenge when a fault occurs among any of the robots. This work presents a Decentralized Fault-Tolerant Control (DFTC) scheme that addresses lidar sensor faults within a system of multiple differential wheeled mobile robots. The robots change the formation shape according to the number of available robots within the formation. A Graph theory is implemented to represent the multi-robot formation and communication. Each mobile robot is equipped with three sensors: a wheel encoder, an Inertial Measurement Unit (IMU), and a lidar sensor. Sensor fault detection and isolation (FDI) is implemented at two levels. The pose estimation obtained from the wheel encoder and IMU is fused using an extended Kalman filter (EKF), and this pose estimation is utilized at the local level of lidar sensor FDI. At the system level, the FDI of the lidar sensor involves computing a residual by comparing the pose estimation with other lidar sensors mounted on other mobile robots within the formation. The presented FTC scheme is simulated in Simulink multi-robot environments.

Development and Validation of a Point Cloud Data Processing Algorithm for Obstacle Recognition in Double Hull Block

<i>Shipyards have recently been experiencing severe labor shortages, prompting increased adoption of production automation systems to mitigate this issue. This paper introduces a point cloud data acquisition and processing system designed to support automation operations within double-hull block environments. The acquisition system utilizes LiDAR sensors and is built as a portable device capable of conducting 360-degree scans inside double-hull blocks. The processing system integrates the RANSAC algorithm for plane recognition and a voxelization algorithm for object detection, enabling accurate identification of obstacles within the double-hull block. To validate the system, a full-scale test bench was constructed to replicate actual working conditions. Experimental results indicated that the system could detect the positions of various obstacles within the test bench with an accuracy of up to 100 mm, which is sufficient for the implementation of automation systems. The findings from this research are expected to facilitate the adoption of production automation in shipyards, enhancing productivity and addressing labor shortages in the industry.</i>

Real-Time Semantic Segmentation of 3D LiDAR Point Clouds for Aircraft Engine Detection in Autonomous Jetbridge Operations

This paper presents a study on aircraft engine identification using real-time 3D LiDAR point cloud segmentation technology, a key element for the development of automated docking systems in airport boarding facilities, known as jetbridges. To achieve this, 3D LiDAR sensors utilizing a spinning method were employed to gather surrounding environmental 3D point cloud data. The raw 3D environmental data were then filtered using the 3D RANSAC technique, excluding ground data and irrelevant apron areas. Segmentation was subsequently conducted based on the filtered data, focusing on aircraft sections. For the segmented aircraft engine parts, the centroid of the grouped data was computed to determine the 3D position of the aircraft engine. Additionally, PointNet was applied to identify aircraft engines from the segmented data. Dynamic tests were conducted in various weather and environmental conditions, evaluating the detection performance across different jetbridge movement speeds and object-to-object distances. The study achieved a mean intersection over union (mIoU) of 81.25% in detecting aircraft engines, despite experiencing challenging conditions such as low-frequency vibrations and changes in the field of view during jetbridge maneuvers. This research provides a strong foundation for enhancing the robustness of jetbridge autonomous docking systems by reducing the sensor noise and distortion in real-time applications. Our future research will focus on optimizing sensor configurations, especially in environments where sea fog, snow, and rain are frequent, by combining RGB image data with 3D LiDAR information. The ultimate goal is to further improve the system’s reliability and efficiency, not only in jetbridge operations but also in broader autonomous vehicle and robotics applications, where precision and reliability are critical. The methodologies and findings of this study hold the potential to significantly advance the development of autonomous technologies across various industrial sectors.

Modified ICP based Framework for 3D Localization and Mapping in GNSS-denied Environments using LiDAR

Abstract. Lidar odometry and mapping has emerged as fundamental technology in various fields, including accurate positioning and mapping, autonomous vehicles, robotics, and environmental monitoring. The ability to create accurate maps and calculate trajectories is essential for localization and navigation in complex environments, particularly in scenarios where GNSS signal are unavailable. In this context, we propose a novel method for simultaneous localization and mapping (SLAM) specifically designed for GNSS-denied environments and without the need for high-accuracy ranging or inertial measurement units (IMUs). Our approach aims to address the challenges posed by GNSS-denied environments and the cost constraints associated with using multiple sensors. By using only lidar sensors, specifically the Velodyne VLP 16 sensor, we offer a cost-effective solution. Our method utilizes a modified version of the Iterative Closest Point (ICP) algorithm to handle consecutive lidar scans seamlessly. In addition to our method, we conducted thorough accuracy assessments on the created map. The reason behind extensive accuracy evaluation is to ensure the quality of registration. By comparing alignment between the features of different frames within the map and determining the error between the registered features and the original, we gain a more comprehensive understanding of registration quality.

LUOJIA Explorer: An Auto-UAV for Unexposed Space Exploration

Abstract. With the advancement of technology, unexposed spaces have emerged as a new type of strategic area, attracting significant attention from researchers. These spaces often present complex environments, such as extreme lighting variations, irregular geometries, and the lack of external positioning signals, making human entry hazardous. Fortunately, advancements in robotics and artificial intelligence have enabled unmanned systems to mitigate these challenges, allowing for safer exploration of unexposed spaces. UAVs, as vital unmanned system platforms, are extensively used across various industrial scenarios and are particularly effective for detecting unexposed spaces. To facilitate automated exploration of these areas, we propose the autonomous UAV system, LUOJIA Explorer. Equipped with a LiDAR sensor, data transmission module, and external anti-collision devices to protect the aircraft, LUOJIA Explorer is designed for effective unexposed space detection. We have integrated a laser point cloud positioning and mapping subsystem, along with a planning and control subsystem, into the LUOJIA Explorer. Through both simulation and actual experiments, the Luojia Explorer has demonstrated the capability to achieve stable flight based on self-positioning in unexposed spaces, with an exploration efficiency of 88.83 m3/s.

Forest and Biodiversity 2: A tree diversity experiment to understand the consequences of multiple dimensions of diversity and composition for long‐term ecosystem function and resilience

Abstract We introduce a new “ecosystem‐scale” experiment at the Cedar Creek Ecosystem Science Reserve in central Minnesota, USA to test long‐term ecosystem consequences of tree diversity and composition. The experiment—the largest of its kind in North America—was designed to provide guidance on forest restoration efforts that will advance carbon sequestration goals and contribute to biodiversity conservation and sustainability. The new Forest and Biodiversity (FAB2) experiment uses native tree species in varying levels of species richness, phylogenetic diversity and functional diversity planted in 100 m2 and 400 m2 plots at 1 m spacing, appropriate for testing long‐term ecosystem consequences. FAB2 was designed and established in conjunction with a prior experiment (FAB1) in which the same set of 12 species was planted in 16 m2 plots at 0.5 m spacing. Both are adjacent to the BioDIV prairie‐grassland diversity experiment, enabling comparative investigations of diversity and ecosystem function relationships between experimental grasslands and forests at different planting densities and plot sizes. Within the first 6 years, mortality in 400 m2 monoculture plots was higher than in 100 m2 plots. The highest mortality occurred in Tilia americana and Acer negundo monocultures, but mortality for both species decreased with increasing plot diversity. These results demonstrate the importance of forest diversity in reducing mortality in some species and point to potential mechanisms, including light and drought stress, that cause tree mortality in vulnerable monocultures. The experiment highlights challenges to maintaining monoculture and low‐diversity treatments in tree mixture experiments of large extent. FAB2 provides a long‐term platform to test the mechanisms and processes that contribute to forest stability and ecosystem productivity in changing environments. Its ecosystem‐scale design, and accompanying R package, are designed to discern species and lineage effects and multiple dimensions of diversity to inform restoration of ecosystem functions and services from forests. It also provides a platform for improving remote sensing approaches, including Uncrewed Aerial Vehicles (UAVs) equipped with LiDAR, multispectral and hyperspectral sensors, to complement ground‐based monitoring. We aim for the experiment to contribute to international efforts to monitor and manage forests in the face of global change.

Low-Cost Thermal Point Clouds of Indoor Environments

Abstract. The integration of low-cost thermal sensors with Apple Smart Devices supports the generation of 3D point clouds that include temperature indicators. This generates a new perspective for the study of buildings, allowing for fast and reliable examination of physical building structures. To the best of our knowledge, this study is the first to demonstrate the use of affordable sensors for 3D thermal point cloud generation. The study involved capturing data from the LiDAR and thermal sensors, followed by an extrinsic calibration process to align the datasets. Subsequently, the point cloud was segmented based on different acquisition poses of the device and finally, the thermal data was projected onto the 3D model, integrating temperature information with spatial coordinates. Our results demonstrate the effectiveness of the approach for three-dimensional point cloud generation in indoor environments, highlighting significant thermal variations and enabling thermal mapping of building structures. Furthermore, our findings underscore the feasibility of employing low-cost sensors for generating detailed thermal models, opening possibilities for widespread adoption in various building analysis applications. This approach provides a comprehensive and cost-effective solution for building monitoring, democratizing access to advanced evaluation tools.

Simulation model to enhance Bragg reflector assisted GeSn SACM-SPAD performance for 1550 nm LIDAR applications in autonomous vehicles

A safe 3-D lidar sensor for autonomous vehicle creates a high demand on 1550 nm SPADs detectors. Due to the limitation of the energy band gab and absorption coefficient of Si and Ge, their photodetectors have low efficiency at the 1550 nm wavelength. Doping Ge with Sn reduces its bandgap and enables higher efficiency in this range while adding Bragg reflector represents a smart way to increase absorption area effective thickness. Here a simulation model for Ge(1-x)Snx SACM SPAD is proposed to work as a 1550 nm laser detector. Two Bragg reflectors are built using Si\\SiGe and Si\\SiGeSn layers. Results show a significant enhancement on detector optical properties. PDP reaches 38 % at room temperature and the increase in Pdp due to Bragg reflector reaches 66 %. Although DCR also increases, it can be handled with proper dead time configuration.

A CMOS Optoelectronic Transimpedance Amplifier Using Concurrent Automatic Gain Control for LiDAR Sensors

This paper presents a novel optoelectronic transimpedance amplifier (OTA) for short-range LiDAR sensors used in 180 nm CMOS technology, which consists of a main transimpedance amplifier (m-TIA) with an on-chip P+/N-well/Deep N-well avalanche photodiode (P+/NW/DNW APD) and a replica TIA with another on-chip APD, not only to acquire circuit symmetry but to also obtain concurrent automatic gain control (AGC) function within a narrow single pulse-width duration. In particular, for concurrent AGC operations, 3-bit PMOS switches with series resistors are added in parallel with the passive feedback resistor in the m-TIA. Then, the PMOS switches can be turned on or off in accordance with the DC output voltage amplitudes of the replica TIA. The post-layout simulations reveal that the OTA extends the dynamic range up to 74.8 dB (i.e., 1 µApp~5.5 mApp) and achieves a 67 dBΩ transimpedance gain, an 830 MHz bandwidth, a 16 pA/√Hz noise current spectral density, a −31 dBm optical sensitivity for a 10−12 bit error rate, and a 6 mW power dissipation from a single 1.8 V supply. The chip occupies a core area of 200 × 120 µm2.

Tree parameter estimation with iPhone point cloud data using multiple algorithms

ABSTRACT LiDAR technology introduced the possibility of indirectly estimating various tree parameters. In 2020, Apple incorporated LiDAR sensors into their iPhone 12 Pro. This brings an opportunity to make data collection and plot scanning incredibly convenient. The main aim is to search for a tool and software to estimate the most accurate Diameter at Breast Height (DBH) for individual trees. Therefore, this study focuses on using an iPhone 12 Pro for data collection of individual trees. The performance of three software tools (ForestScanner, rTLS, RANSAC) for the estimation of DBH with field-estimated DBH was compared. The investigation of the significance of the estimation of DBH for each tree species was performed using the three algorithms. In this context, the scanning of 123 trees comprising six species: pine, oak, beech, hornbeam, spruce, and fir was done. DBH was estimated of the scanned tree point cloud using a built-in algorithm of ForestScanner application in the iPhone 12 Pro, RANSAC (CloudCompare plugin), and rTLS (an R-based package). In this study, ForestScanner showed the best results. Therefore, ForestScanner can be used as a reliable tool for measuring DBH. Inferences confirmed by ANOVA and Tukey post-hoc tests show that species significantly influence DBH. Accurate DBH estimation is crucial, and using lightweight devices like the iPhone can revolutionize the forest inventory sector in terms of the estimation of DBH.

Precision detection and identification method for apparent damage in timber components of historic buildings based on portable LiDAR equipment

For the purpose of preservation and restoration of historic timber buildings, it is critical that damage to timber components must be detected and identified accurately since proper detection and identification lay the foundation for scientific utilization and preservation of cultural heritage. Portable smart devices with integrated LiDAR sensor technology have significantly advanced, enabling them to be major pedestals for damage detection. An iPhone Pro based technique is therefore suggested for the precise detection and diagnosis of apparent damage to the timber components. The study consists of three major procedures. First, an apparent data collection of the timber components was conducted. A high-resolution triangular mesh model was utilized for precision detection and identification. Point cloud and texture data were the main constituents of this data collection. Second, a technique for image processing and apparent damage analysis was proposed for crack detection in historic timber buildings. This approach can be used to identify, recognize,and assess damage to timber components such as cracks, rotting, and knots. Finally, actual measurements of the timber components in Datong, Shanxi Province, at the Great Temple Corner, proved the effectiveness of the iPhone 12 Pro as a substitute approach to the conventional inspection equipment. The findings indicate that the iPhone 12 Pro, which has a built-in LiDAR sensor, performs exceptionally well in terms of effectiveness, accuracy, and graphical interpretation when identifying apparent damage to historic timber components. Moreover, it has the potential to substitute traditional measurement, 3D scanning, and other detection and identification techniques. Even though damage detection and identification of historical timber buildings has reached a mature stage in terms of portable, refined, and digital transformation, such technique can serve as a compatible addition as a safe, affordable, and paradigm shifting technical solution for detection and identification of historic building components.

High precision DSRC and LiDAR data integration positioning method for autonomous vehicles based on CNN

In order to improve the safe driving and automatic positioning capability of autonomous vehicles, a high-precision DSRC and LiDAR data integration positioning technology for autonomous vehicles based on CNN is proposed. Import the data of Dedicated Short Range Communications and Light Detection and Ranging for automatic driving vehicle positioning, carry out kinematic analysis of autonomous driving vehicles under multi-sensor fusion, and transform the data of DSRC and LiDAR sensors into tightly coupled coordinate systems; The CNN depth learning method is used to compensate the position and attitude tracking estimation error under the overall time stamp synchronization of the sensor through adaptive information tracking; The first and second order feedforward compensation is made for the positioning parameters of the autonomous driving vehicle using the PID model, and the point cloud feature matching model is fused to complete the estimation of the positioning attitude parameters of the autonomous driving vehicle. In order to eliminate the noise interference under the DSRC communication mechanism, the Kalman filter function is used to automatically optimize the constraint parameters in the point cloud feature detection model, and the positioning error parameters are dynamically filtered and adjusted; Kinematics analysis is carried out for the driving state of the vehicle, and the positioning error in the vehicle movement is controlled through the difference technology to achieve high-precision DSRC and LiDAR data integration positioning. The simulation results show that this method can integrate and locate the high-precision DSRC and LiDAR data of the autonomous vehicle, and the attitude estimation and positioning accuracy of the vehicle is good, while the error of the attitude parameter estimation of the autonomous vehicle is low.

Research on Precise Landing of Rotor UAV Based on Multi-Sensor Fusion

With the rapid development of drone technology, the autonomous landing of drones has become a current research hotspot. This article introduces an autonomous drone landing system based on multi-sensor fusion. The system combines GPS, IMU, visual sensors, LiDAR, and other sensor data through Kalman filtering to achieve data fusion and improve the positioning accuracy and stability of the drone during landing.

Seal Pipeline: Enhancing Dynamic Object Detection and Tracking for Autonomous Unmanned Surface Vehicles in Maritime Environments

This study addresses the dynamic object detection problem for Unmanned Surface Vehicles (USVs) in marine environments, which is complicated by boat tilting and camera illumination sensitivity. A novel pipeline named “Seal” is proposed to enhance detection accuracy and reliability. The approach begins with an innovative preprocessing stage that integrates data from the Inertial Measurement Unit (IMU) with LiDAR sensors to correct tilt-induced distortions in LiDAR point cloud data and reduce ripple effects around objects. The adjusted data are grouped using clustering algorithms and bounding boxes for precise object localization. Additionally, a specialized Kalman filter tailored for maritime environments mitigates object discontinuities between successive frames and addresses data sparsity caused by boat tilting. The methodology was evaluated using the VRX simulator, with experiments conducted on the Volga River using real USVs. The preprocessing effectiveness was assessed using the Root Mean Square Error (RMSE) and tracking accuracy was evaluated through detection rate metrics. The results demonstrate a 25% to 30% improvement in detection accuracy and show that the pipeline can aid industry even with sparse object representation across different frames. This study highlights the potential of integrating sensor fusion with specialized tracking for accurate dynamic object detection in maritime settings, establishing a new benchmark for USV navigation systems’ accuracy and reliability.

Optimized Right-Turn Pedestrian Collision Avoidance System Using Intersection LiDAR

The incidence of right-turning pedestrian accidents is increasing in South Korea. Most of the accidents occur when a large vehicle is turning right, and the main cause of the accidents was found to be the driver’s limited field of vision. After these accidents, the government implemented a series of institutional measures with the objective of preventing such accidents. However, despite the institutional arrangements in place, pedestrian accidents continue to occur. We focused on the many limitations that autonomous vehicles, like humans, can face in such situations. To address this issue, we propose a right-turn pedestrian collision avoidance system by installing a LiDAR sensor in the center of the intersection to facilitate pedestrian detection. Furthermore, the urban road environment is considered, as this provides the optimal conditions for the model to perform at its best. During this research, we collected data on right-turn accidents using the CARLA simulator and ROS interface and demonstrated the effectiveness of our approach in preventing such incidents. Our results suggest that the implementation of this method can effectively reduce the incidence of right-turn accidents in autonomous vehicles.