CN4SRSS: Combined network for super-resolution reconstruction and semantic segmentation in frontal-viewing camera images of vehicle

Abstract

Recently, the importance of semantic segmentation research for scene understanding in frontal viewing camera images of autonomous vehicles has increased. The existing state-of-the-art (SOTA) methods for semantic segmentation exhibit high accuracy for high-resolution images and low-resolution (LR) images without degradation factors of blur and noise. Owing to the nature of vehicles, the need is increasing for the pre-judgment of emergencies through the accurate semantic segmentation of LR images with the degradation factors acquired by low-cost camera at far distance. However, no research exists on super-resolution reconstruction (SR)-based semantic segmentation of LR images with degradation factors. Therefore, this study proposes a novel combined network for a super-resolution reconstruction and semantic segmentation (CN4SRSS) framework based on attention and re-focus network (ARNet), which exhibits low computational cost and high semantic segmentation accuracy. The experimental results using LR image datasets based on CamVid and Minicity datasets, which are open databases, show that the semantic segmentation accuracy (pixel accuracy) based on the proposed CN4SRSS and DeepLab v3 + is 93.14% and 89.48%, respectively. Particularly, the proposed method shows higher accuracy when compared to the SOTA methods. Furthermore, the proposed method has been confirmed that requires lower computational cost in terms of the number of parameters, memory usage, number of multi-adds calculation, and floating-point operations per second (FLOPs) than the SOTA methods.