CORRECTION AND DENSIFICATION OF UAS-BASED PHOTOGRAMMETRIC THERMAL POINT CLOUD

Abstract. Photogrammetric processing algorithms can suffer problems due to either the initial image quality (noise, low radiometric quality, shadows and so on) or to certain surface materials (shiny or textureless objects). This can result in noisy point clouds and/or difficulties in feature extraction. Specifically, dense point clouds which are generated with photogrammetric method using a lightweight thermal camera, are more noisy and sparse than the point clouds of high-resolution digital camera images. In this paper, new method which produces more reliable and dense thermal point cloud using the sparse thermal point cloud and high resolution digital point cloud was considered. Both thermal and digital images were obtained with UAS (Unmanned Aerial System) based lightweight Optris PI 450 and Canon EOS 605D camera images. Thermal and digital point clouds, and orthophotos were produced using photogrammetric methods. Problematic thermal point cloud was transformed to a high density thermal point cloud using image processing methods such as rasterizing, registering, interpolation and filling. The results showed that the obtained thermal point cloud - up to chosen processing parameters - was 87% more densify than the original point cloud. The second improvement was gained at the height accuracy of the thermal point cloud. New densified point cloud has more consistent elevation model while the original thermal point cloud shows serious deviations from the expected surface model.

Thermography is an efficient way of detecting the thermal problems of the roof as a major part of a building’s energy dissipation. Thermal images have a low spatial resolution, making it a challenge to produce a three-dimensional thermal model using aerial images. This paper proposes a combination of thermal and visible point clouds to generate a higher-resolution thermal point cloud from roofs of buildings. For this purpose, after obtaining the internal orientation parameters through camera calibration, visible and thermal point clouds were generated and then registered to each other using ground control points. The roofs of buildings were then extracted from the visible point cloud in four steps. First ground points were removed using cloth simulation filter (CSF), and then vegetation points were eliminated by applying entropy and red-green-blue vegetation index (RGBVI). Geometric features and a segmentation step were considered to filter roofs from other parts. Finally, by combining visible and thermal point clouds, the generated point had a high spatial resolution along with thermal information. In the achieved results, the thermal camera calibration was performed with an accuracy of 0.31 pixels, and the thermal and visible point clouds were registered with an absolute distance of < 0.3 m. The experimental results showed an accuracy of 18 cm for automated extraction of building roofs and 0.6 pixel for production of a high-resolution thermal point cloud, which was five times the density of the primary thermal point cloud and free from distortions.

Abstract. Thermal point clouds integrate thermal radiation and laser point clouds effectively. However, the semantic information for the interpretation of building thermal point clouds can hardly be precisely inferred. Transferring the semantics encapsulated in 3D building models at Level of Detail (LoD)3 has a potential to fill this gap. In this work, we propose a workflow enriching thermal point clouds with the geo-position and semantics of LoD3 building models, which utilizes features of both modalities: model point clouds are generated from LoD3 models, and thermal point clouds are co-registered by coarse-to-fine registration. The proposed method can automatically co-register the point clouds from different sources and enrich the thermal point cloud in facade-detailed semantics. The enriched thermal point cloud supports thermal analysis and can facilitate the development of currently scarce deep learning models operating directly on thermal point clouds.

Abstract. Thermography is a robust method for detecting thermal irregularities on the roof of the buildings as one of the main energy dissipation parts. Recently, UAVs are presented to be useful in gathering 3D thermal data of the building roofs. In this topic, the low spatial resolution of thermal imagery is a challenge which leads to a sparse resolution in point clouds. This paper suggests the fusion of visible and thermal point clouds to generate a high-resolution thermal point cloud of the building roofs. For the purpose, camera calibration is performed to obtain internal orientation parameters, and then thermal point clouds and visible point clouds are generated. In the next step, both two point clouds are geo-referenced by control points. To extract building roofs from the visible point cloud, CSF ground filtering is applied, and the vegetation layer is removed by RGBVI index. Afterward, a predefined threshold is applied to the normal vectors in the z-direction in order to separate facets of roofs from the walls. Finally, the visible point cloud of the building roofs and registered thermal point cloud are combined and generate a fused dense point cloud. Results show mean re-projection error of 0.31 pixels for thermal camera calibration and mean absolute distance of 0.2 m for point clouds registration. The final product is a fused point cloud, which its density improves up to twice of the initial thermal point cloud density and it has the spatial accuracy of visible point cloud along with thermal information of the building roofs.

Monitoring building efficiency is a hot topic for engineers and researchers. TIR (Thermal Infrared) images describe thermal attributes but require professional knowledge for analysis. Moreover, images can hardly describe the thermal attributes of the buildings in all aspects. Therefore, as-built thermal point clouds become the solution. In this case, geometric distortions of TIR images can not be neglected. To generate the thermal point clouds with TIR image sequences, we propose to use MLS (Mobile Laser Scanning) point clouds as control point sets for calibration and a workflow for the thermal point cloud generation. To begin with, intensity images are generated from the point clouds, and corresponding points are detected from the TIR image and intensity image frame. Moreover, the intrinsic parameters are estimated with corresponding points. Finally, the thermal point clouds are generated with calibrated TIR images and MLS point clouds. The result demonstrates that the automatic method can achieve comparable or even better calibration results than the manual method.

Fusion of thermal imagery with point clouds for building façade thermal attribute mapping

Abstract. In the last few years, Unmanned Aerial Vehicles (UAVs) are being frequently used to acquire high resolution photogrammetric images and consequently producing Digital Surface Models (DSMs) and orthophotos in a photogrammetric procedure for topography and surface processing applications. Thermal imaging sensors are mostly used for interpretation and monitoring purposes because of lower geometric resolution. But yet, thermal mapping is getting more important in civil applications, as thermal sensors can be used in condition that visible sensors cannot, such as foggy weather and night times which is not possible for visible cameras. But, low geometric quality and resolution of thermal images is a main drawback that 3D thermal modelling are encountered with. This study aims to offer a solution for to fixing mentioned problem and generating a thermal 3D model with higher spatial resolution based on thermal and visible point clouds integration. This integration leads to generate a more accurate thermal point cloud and DEM with more density and resolution which is appropriate for 3D thermal modelling. The main steps of this study are: generating thermal and RGB point clouds separately, registration of them in two course and fine level and finally adding thermal information to RGB high resolution point cloud by interpolation concept. Experimental results are presented in a mesh that has more faces (With a factor of 23) which leads to a higher resolution textured mesh with thermal information.

3D thermal models are associated with building inspection and energy efficiency evaluation. Fusing Thermal infrared (TIR) images with MLS (Mobile Laser scanning) point clouds enables the generation of thermal point clouds, which combine detailed geometric data with thermal attributes at each 3D point. RGB images are typically used to reconstruct a 3D point cloud and apply thermal textures to the model. Therefore, the generated thermal point cloud heavily relies on accurate RGB reconstruction and scale estimation. In this contribution, we introduce a novel image-feature alignment method to directly co-register TIR images with MLS point clouds. The intensity images are generated from the point clouds, and corresponding feature points are matched with the TIR images. With the estimated corresponding points, the pose can be calculated, and the thermal textures are projected onto the MLS point clouds for thermal point cloud generation. Our method achieves results comparable to manual labeling with a projection error of RMSE 3.4 pixels, offering an efficient and reliable solution to generate 3D thermal models for building energy evaluations.

Infrared thermal dense point clouds: A new frontier for remote landslide investigation

This work describes an efficient approach for generating large 3D thermal point clouds considering the occlusion of camera viewpoints. For that purpose, RGB and thermal imagery are first corrected and fused with an intensity correlation-based algorithm. Then, absolute temperature values are obtained from the normalized data. Finally, thermal imagery is mapped on the point cloud using the Graphics Processing Unit (GPU) hardware. The proposed occlusion-aware mapping algorithm is massively parallelized using OpenGL's compute shaders. Our solution allows generating dense thermal point clouds in a lower response time compared with other notable soft-ware solutions (e.g., Agisoft Metashape or Pix4Dmapper) that yield results with a significantly lower point density.

The study of strain effects in thermally-forced rock masses has gathered growing interest from engineering geology researchers in the last decade. In this framework, digital photogrammetry and infrared thermography have become two of the most exploited remote surveying techniques in engineering geology applications because they can provide useful information concerning geomechanical and thermal conditions of these complex natural systems where the mechanical role of joints cannot be neglected. In this paper, a methodology is proposed for generating point clouds of rock masses prone to failure, combining the high geometric accuracy of RGB optical images and the thermal information derived by infrared thermography surveys. Multiple 3D thermal point clouds and a high-resolution RGB point cloud were separately generated and co-registered by acquiring thermograms at different times of the day and in different seasons using commercial software for Structure from Motion and point cloud analysis. Temperature attributes of thermal point clouds were merged with the reference high-resolution optical point cloud to obtain a composite 3D model storing accurate geometric information and multitemporal surface temperature distributions. The quality of merged point clouds was evaluated by comparing temperature distributions derived by 2D thermograms and 3D thermal models, with a view to estimating their accuracy in describing surface thermal fields. Moreover, a preliminary attempt was made to test the feasibility of this approach in investigating the thermal behavior of complex natural systems such as jointed rock masses by analyzing the spatial distribution and temporal evolution of surface temperature ranges under different climatic conditions. The obtained results show that despite the low resolution of the IR sensor, the geometric accuracy and the correspondence between 2D and 3D temperature measurements are high enough to consider 3D thermal point clouds suitable to describe surface temperature distributions and adequate for monitoring purposes of jointed rock mass.

An optimized approach for generating dense thermal point clouds from UAV-imagery

Heat and excessive solar radiation can produce abiotic stresses during apple maturation, resulting fruit quality. Therefore, the monitoring of temperature on fruit surface (FST) over the growing period can allow to identify thresholds, above of which several physiological disorders such as sunburn may occur in apple.The current approaches neglect spatial variation of FST and have reduced repeatability, resulting in unreliable predictions. In this study, LiDAR laser scanning and thermal imaging were employed to detect the temperature on fruit surface by means of 3D point cloud. A process for calibrating the two sensors based on an active board target and producing a 3D thermal point cloud was suggested. After calibration, the sensor system was utilised to scan the fruit trees, while temperature values assigned in the corresponding 3D point cloud were based on the extrinsic calibration. Whereas a fruit detection algorithm was performed to segment the FST from each apple.•The approach allows the calibration of LiDAR laser scanner with thermal camera in order to produce a 3D thermal point cloud.•The method can be applied in apple trees for segmenting FST in 3D. Whereas the approach can be utilised to predict several physiological disorders including sunburn on fruit surface.

Thermal inspection of buildings regarding efficient energy use is an increasing need in today’s energy-demanding world. This paper proposes a framework for mapping temperature attributes from thermal point clouds onto building facades. The goal is to generate thermal textures for three-dimensional (3D) analysis. Classical texture generation methods project facade images directly onto a 3D building model. Due to the limited level of detail of these models, projection errors occur. Therefore, we use point clouds from mobile laser scanning extended by intensities extracted from thermal infrared (TIR) image sequences. We are not using 3D reconstructed point clouds because of the limited geometric density and accuracy of TIR images, which can lead to poor 3D reconstruction. We project these thermal point clouds onto facades using a mapping algorithm. The algorithm uses a nearest neighbor search to find an optimal nearest point with three different approaches: “Minimize angle to normal”, “Minimize perpendicular distance to normal”, and “Minimize only distance”. Instead of interpolation, nearest neighbor is used because it retains the original temperature values. The thermal intensities of the optimal nearest points are weighted by resolution layers and mapped to the facade. The approach “Minimize perpendicular distance to normal” yields the finest texture resolution at a reasonable processing time. The accuracy of the generated texture is evaluated based on estimating the shift of the window corner points from a ground truth texture. A performance metric root-mean-square deviation (RMSD) value that measures this shift is calculated. In terms of accuracy, the nearest neighbor method outperformed bilinear interpolation and an existing TIR image-based texturing method.

Fusion of urban 3D point clouds with thermal attributes using MLS data and TIR image sequences