Intensifying the spatial resolution of 3D thermal models from aerial imagery using deep learning-based image super-resolution

Abstract

Nowadays, 3D thermal models can play an important role in buildings' energy management while acquiring multisource data to generate a high-resolution 3D thermal model. Consequently, in this article, a method for intensifying 3D thermal model using deep learning-based image super-resolution is presented. In the proposed method, first, the enhanced deep residual super-resolution (EDSR) deep network is re-trained based on thermal aerial images. Second, the resolution of low-resolution thermal images is enhanced using the newly trained network. Finally, the state-of-the-art structures from motion (SfM), semi global matching (SGM) and space intersection are utilized to generate intensified 3D thermal model from the resolution enhanced thermal images. Spatial evaluations indicate a 5% increase in edge-based image fusion metric (EFM) for the intensified 3D model. Besides, the evaluations show that the modulation transfer function (MTF) curves of the intensified 3D thermal model are closer to a reference model against the original 3D thermal model. Highlights A 3D thermal model intensification solution using EDSR is proposed which is independent of hardware techniques and multisource data. Considering the importance of edge sharpness in the intensified 3D thermal model, the quality of edges is assessed using MTF curves and the EFM metric. In comparison to the original 3D thermal model, the MTF curves of the intensified 3D thermal model are closer to the MTF curve of the high-resolution 3D model. The EFM metric shows higher values for MTF curves of the intensified 3D thermal model against MTF curves of the original 3D thermal model.