Performance Comparison of Deep Residual Networks-Based Super Resolution Algorithms Using Thermal Images: Case Study of Crowd Counting

Abstract

AbstractHumans are able to perceive objects only in the visible spectrum range which limits the perception abilities in poor weather or low illumination conditions. The limitations are usually handled through technological advancements in thermographic imaging. However, thermal cameras have poor spatial resolutions compared to RGB cameras. Super-resolution (SR) techniques are commonly used to improve the overall quality of low-resolution images. There has been a major shift of research among the Computer Vision researchers towards SR techniques particularly aimed for thermal images. This paper analyzes the performance of three deep learning-based state-of-the-art SR algorithms namely Enhanced Deep Super Resolution (EDSR), Residual Channel Attention Network (RCAN) and Residual Dense Network (RDN) on thermal images. The algorithms were trained from scratch for different upscaling factors of ×2 and ×4. The dataset was generated from two different thermal imaging sequences of BU-TIV benchmark. The sequences contain both sparse and highly dense type of crowds with a far field camera view. The trained models were then used to super-resolve unseen test images. The quantitative analysis of the test images was performed using common image quality metrics such as PSNR, SSIM and LPIPS, while qualitative analysis was provided by evaluating effectiveness of the algorithms for crowd counting application. After only 54 and 51 epochs of RCAN and RDN respectively, both approaches were able to output average scores of 37.878, 0.986, 0.0098 and 30.175, 0.945, 0.0636 for PSNR, SSIM and LPIPS respectively. The EDSR algorithm took the least computation time during both training and testing because of its simple architecture. This research proves that a reasonable accuracy can be achieved with fewer training epochs when an application-specific dataset is carefully selected.