Fusion of thermal imagery with point clouds for building façade thermal attribute mapping

Abstract

Thermal image data are widely used to assess the insulation quality of buildings and to detect thermal leakages. In our approach, we merge terrestrial thermal image data and 3D point clouds to perform thermal texture mapping for building facades. Since geo-referencing data of a hand-held thermal camera is usually not available in such applications, registration between thermal images and a 3D point cloud (for instance generated from RGB image data by structure-from-motion techniques) is essential. In our approach, thermal image data registration is conducted in four steps: First, another point cloud is generated from the thermal image data. Next, a coarse registration between thermal point cloud and RGB point cloud is performed using the fast global registration (FGR) algorithm. The best corresponding thermal-RGB image pairs are acquired by picking up the lowest Euclidean distance between the exterior orientation parameters of thermal images and transformed exterior orientation parameters of RGB images. Subsequently, radiation-invariant feature transform (RIFT), normalized barycentric coordinate system (NBCS) and random sample consensus (RANSAC) are employed to extract reliable matching features on thermal-RGB image pairs. Afterwards, a fine registration is performed by mono-plotting of the RGB image, followed by image resection of the thermal image. Finally, in terms of texture mapping algorithms, in order to remove the blur effects caused by small misalignments for different candidate images, a global image pose refinement approach, which aims to minimize the temperature disagreements provided by different images for the same object points, is proposed. In addition, in order to ensure high geometric and radiant accuracy, camera calibrations are performed. Experiments showed that the proposed method could not only achieve high geometric registration accuracy, but also provide a good radiometric accuracy with RMSE lower than 1.5 °C.