Scene Parsing Research Articles

Two‐stage coarse‐to‐fine method for pathological images in medical decision‐making systems

AbstractArtificial intelligence decision systems play an important supporting role in the field of medical information. Medical image analysis is an important part of decision systems and an even more important part of medical diagnosis and treatment. The wealth of cellular information in histopathological images makes them a reliable means of diagnosing tumors. However, due to the large size, high resolution, and complex background structure of pathology images, deep learning methods still have various difficulties in the recognition of pathology images. Based on this, this study proposes a two‐stage continuous improvement‐based approach for pathology image recognition in medical decision systems. For pathology images with complex backgrounds, normalization and enhancement is performed to remove the effects of noise color, and light‐dark inconsistencies on the segmentation network. The continuous refinement PSP Net (CRPSPNet) is then designed for accurate recognition of the pathology images. CRPSPNet is divided into two stages: Pyramid Scene Parsing Network segmentation to obtain coarse segmentation results; and continuous refinement model refines the results of the first stage. Experiments using more than 1,000 osteosarcoma pathology images have shown that the method gives more accurate results with fewer computer resources and processing time than traditional optimization models. Its Intersection over Union achieves 0.76.

Temporal Pixel-Level Semantic Understanding Through the VSPW Dataset.

Scene understanding through pixel-level semantic parsing is one of the main problems in computer vision. Till now, image-based methods and datasets for scene parsing have been well explored. However, the real world is naturally dynamic instead of a static state. Thus, learning to perform video scene parsing is more practical for real-world applications. Considering that few datasets cover an extensive range of scenes and object categories with temporal pixel-level annotations, in this work, we present a large-scale video scene parsing dataset, namely VSPW (Video Scene Parsing in the Wild). To be specific, there are a total of 251,633 frames from 3,536 videos with densely pixel-wise annotations in VSPW, including a large variety of 231 scenes and 124 object categories. Besides, VSPW is densely annotated with a high frame rate of 15 f/s, and over 96% of videos from VSPW have high spatial resolutions from 720P to 4 K. To the best of our knowledge, VSPW is the first attempt to address the challenging video scene parsing task in the wild by considering diverse scenes. Based on our VSPW, we further propose Temporal Attention Blending (TAB) Networks to harness temporal context information for better pixel-level semantic understanding of videos. Extensive experiments on VSPW well demonstrate the superiority of the proposed TAB over other baseline approaches. We hope the new proposed dataset and the explorations in this work can help advance the challenging yet practical video scene parsing task in the future. Both the dataset and the code are available at www.vspwdataset.com.

Rubber Tree Recognition Based on UAV RGB Multi-Angle Imagery and Deep Learning

The rubber tree (Hevea brasiliensis) is an important tree species for the production of natural latex, which is an essential raw material for varieties of industrial and non-industrial products. Rapid and accurate identification of the number of rubber trees not only plays an important role in predicting biomass and yield but also is beneficial to estimating carbon sinks and promoting the sustainable development of rubber plantations. However, the existing recognition methods based on canopy characteristic segmentation are not suitable for detecting individual rubber trees due to their high canopy coverage and similar crown structure. Fortunately, rubber trees have a defoliation period of about 40 days, which makes their trunks clearly visible in high-resolution RGB images. Therefore, this study employed an unmanned aerial vehicle (UAV) equipped with an RGB camera to acquire high-resolution images of rubber plantations from three observation angles (−90°, −60°, 45°) and two flight directions (SN: perpendicular to the rubber planting row, and WE: parallel to rubber planting rows) during the deciduous period. Four convolutional neural networks (multi-scale attention network, MAnet; Unet++; Unet; pyramid scene parsing network, PSPnet) were utilized to explore observation angles and directions beneficial for rubber tree trunk identification and counting. The results indicate that Unet++ achieved the best recognition accuracy (precision = 0.979, recall = 0.919, F-measure = 94.7%) with an observation angle of −60° and flight mode of SN among the four deep learning algorithms. This research provides a new idea for tree trunk identification by multi-angle observation of forests in specific phenological periods.

Research on the Efficiency of Bridge Crack Detection by Coupling Deep Learning Frameworks with Convolutional Neural Networks.

Bridge crack detection based on deep learning is a research area of great interest and difficulty in the field of bridge health detection. This study aimed to investigate the effectiveness of coupling a deep learning framework (DLF) with a convolutional neural network (CNN) for bridge crack detection. A dataset consisting of 2068 bridge crack images was randomly split into training, verification, and testing sets with a ratio of 8:1:1, respectively. Several CNN models, including Faster R-CNN, Single Shot MultiBox Detector (SSD), You Only Look Once (YOLO)-v5(x), U-Net, and Pyramid Scene Parsing Network (PSPNet), were used to conduct experiments using the PyTorch, TensorFlow2, and Keras frameworks. The experimental results show that the Harmonic Mean (F1) values of the detection results of the Faster R-CNN and SSD models under the Keras framework are relatively large (0.76 and 0.67, respectively, in the object detection model). The YOLO-v5(x) model of the TensorFlow2 framework achieved the highest F1 value of 0.67. In semantic segmentation models, the U-Net model achieved the highest detection result accuracy (AC) value of 98.37% under the PyTorch framework. The PSPNet model achieved the highest AC value of 97.86% under the TensorFlow2 framework. These experimental results provide optimal coupling efficiency parameters of a DLF and CNN for bridge crack detection. A more accurate and efficient DLF and CNN model for bridge crack detection has been obtained, which has significant practical application value.

Automated crack segmentation on 3D asphalt surfaces with richer attention and hybrid pyramid structures

ABSTRACT Automated pavement crack detection is crucial to supporting fine pavement maintenance and ensuring safety for road facilities. Due to the complex pavement condition and crack features, it is still a critical challenge in intelligent pavement surveys. This paper proposed a novel pixel-level pavement crack segmentation network, PCSNet, to provide a solution to this challenge. The network has richer attention and hybrid pyramid structures, which implement full-process crack feature fusion and enhancement. The richer attention module consists of cascaded self-attention and attention gate modules. It captures the crack spatial dependence information and prunes the feature response. The hybrid pyramid structures consist of a multistage convolutional pyramid module and a pyramid pooling module. It integrates contextual information at multiple receptive field scales to enhance the potential crack feature representation. The proposed structure enriches the crack details and optimises the scene parsing on the global geometry of the cracks. A sizeable 3D pavement crack dataset is built for training and testing. The proposed network exhibited the best performance, achieving F1-score, mean intersection of union, and mean pixel accuracy of 81.21%, 77.13%, and 87.17%, respectively. The network can reconstruct the complete crack geometry, preserve the crack edges well, and optimises the detection of shallow and complex cracks. The method exhibits superior and robust performance, facilitating accurate pavement technical condition assessment and maintenance decisions.

Sıtma Hastalığının Otomatik Tespiti için EfficientNet Tabanlı Segmentasyon Modellerinin Performans Analizi

Malaria is a disease caused by the Plasmodium parasite, which is common in the tropics. The traditional methods commonly used to diagnose malaria, one of the world's deadliest diseases, are microscopic diagnostic methods in which blood samples taken from suspected individuals are manually examined, or rapid diagnostic tests that are sensitive to human errors. These processes are inexpensive, but experienced and qualified clinicians are needed. Due to this shortcoming, modern diagnostic tools are crucial in the struggle against the disease. In this study, an approach based on deep learning methods was used, which offers beneficial solutions in the diagnosis of disease from medical images. In the proposed approach, U-Net, Pyramid Scene Parsing Network, LinkNet, and Feature Pyramid Network segmentation methods were modified with 8 different pre-trained variants of the EfficientNet deep learning model to obtain improved models. In the malaria segmentation performed with these models, the highest Dice score of 91.50% was achieved in the use of the U-Net model with EfficientNetB6. This model offers a faster and more robust solution to detecting parasites compared to traditional methods.

Automatic Segmentation of Cervical Precancerous Lesions in Colposcopy Image Using Pyramid Scene Parsing Network and Transfer Learning

Automatic Segmentation of Cervical Precancerous Lesions in Colposcopy Image Using Pyramid Scene Parsing Network and Transfer Learning

Using Deep Learning and Google Street View Imagery to Assess and Improve Cyclist Safety in London

Cycling is a sustainable mode of transportation with significant benefits for society. The number of cyclists on the streets depends heavily on their perception of safety, which makes it essential to establish a common metric for determining and comparing risk factors related to road safety. This research addresses the identification of cyclists’ risk factors using deep learning techniques applied to a Google Street View (GSV) imagery dataset. The research utilizes a case study approach, focusing on London, and applies object detection and image segmentation models to extract cyclists’ risk factors from GSV images. Two state-of-the-art tools, You Only Look Once version 5 (YOLOv5) and the pyramid scene parsing network (PSPNet101), were used for object detection and image segmentation. This study analyzes the results and discusses the technology’s limitations and potential for improvements in assessing cyclist safety. Approximately 2 million objects were identified, and 250 billion pixels were labeled in the 500,000 images available in the dataset. On average, 108 images were analyzed per Lower Layer Super Output Area (LSOA) in London. The distribution of risk factors, including high vehicle speed, tram/train rails, truck circulation, parked cars and the presence of pedestrians, was identified at the LSOA level using YOLOv5. Statistically significant negative correlations were found between cars and buses, cars and cyclists, and cars and people. In contrast, positive correlations were observed between people and buses and between people and bicycles. Using PSPNet101, building (19%), sky (15%) and road (15%) pixels were the most common. The findings of this research have the potential to contribute to a better understanding of risk factors for cyclists in urban environments and provide insights for creating safer cities for cyclists by applying deep learning techniques.

EDET: Entity Descriptor Encoder of Transformer for Multi-Modal Knowledge Graph in Scene Parsing

In scene parsing, the model is required to be able to process complex multi-modal data such as images and contexts in real scenes, and discover their implicit connections from objects existing in the scene. As a storage method that contains entity information and the relationship between entities, a knowledge graph can well express objects and the semantic relationship between objects in the scene. In this paper, a new multi-phase process was proposed to solve scene parsing tasks; first, a knowledge graph was used to align the multi-modal information and then the graph-based model generates results. We also designed an experiment of feature engineering’s validation for a deep-learning model to preliminarily verify the effectiveness of this method. Hence, we proposed a knowledge representation method named Entity Descriptor Encoder of Transformer (EDET), which uses both the entity itself and its internal attributes for knowledge representation. This method can be embedded into the transformer structure to solve multi-modal scene parsing tasks. EDET can aggregate the multi-modal attributes of entities, and the results in the scene graph generation and image captioning tasks prove that EDET has excellent performance in multi-modal fields. Finally, the proposed method was applied to the industrial scene, which confirmed the viability of our method.

Remora Namib Beetle Optimization Enabled Deep Learning for Severity of COVID-19 Lung Infection Identification and Classification Using CT Images.

Coronavirus disease 2019 (COVID-19) has seen a crucial outburst for both females and males worldwide. Automatic lung infection detection from medical imaging modalities provides high potential for increasing the treatment for patients to tackle COVID-19 disease. COVID-19 detection from lung CT images is a rapid way of diagnosing patients. However, identifying the occurrence of infectious tissues and segmenting this from CT images implies several challenges. Therefore, efficient techniques termed as Remora Namib Beetle Optimization_ Deep Quantum Neural Network (RNBO_DQNN) and RNBO_Deep Neuro Fuzzy Network (RNBO_DNFN) are introduced for the identification as well as classification of COVID-19 lung infection. Here, the pre-processing of lung CT images is performed utilizing an adaptive Wiener filter, whereas lung lobe segmentation is performed employing the Pyramid Scene Parsing Network (PSP-Net). Afterwards, feature extraction is carried out wherein features are extracted for the classification phase. In the first level of classification, DQNN is utilized, tuned by RNBO. Furthermore, RNBO is designed by merging the Remora Optimization Algorithm (ROA) and Namib Beetle Optimization (NBO). If a classified output is COVID-19, then the second-level classification is executed using DNFN for further classification. Additionally, DNFN is also trained by employing the newly proposed RNBO. Furthermore, the devised RNBO_DNFN achieved maximum testing accuracy, with TNR and TPR obtaining values of 89.4%, 89.5% and 87.5%.

EAAINet: An Element-Wise Attention Network With Global Affinity Information for Accurate Indoor Visual Localization

Visual localization, a vital component of many visual applications, has been tackled by scene coordinates regression (SCoRe) methods that leverage neural networks to predict scene coordinates, followed by a PnP algorithm to recover camera pose. However, these methods do not consider the relationship between image patches, known as relative features or affinity information, which is instrumental for network to perform complete scene parsing. Besides, owing to the visual similarity between image patches, these methods are weak in extracting reliable absolute features that represent the context information of the image patches, resulting in inferior localization performance. In response, we propose EAAINet that is based on classical SCoRe approaches and consists of two novel modules: the Global Affinity Aggregation Module (GAAM) and the Element-wise Attention Module (EAM). Specifically, GAAM employs an interval sampling strategy to sample image patches to construct sparse graph neural networks (GNNs), from which global affinity information between image patches is retrieved, hence ensuring precise scene parsing. EAM integrates multi-level features to generate reliable absolute features to regress accurate scene coordinates, with the key insight that the structure information is essential to differentiate similar image patches and the semantic information assists in modeling regression problems. Technically, EAM predicts element-wise soft attention masks to reconcile multi-level feature maps, enabling efficient feature fusion. Positional encoding and uncertainty modeling are also employed to enhance visual localization performance. Experimental results show that EAAINet significantly outperforms the state-of-the-arts on multiple benchmarks with faster speed and less model parameters. Source code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/DK-HU/EAAINet</uri> .

Geometric Constraint-Based and Improved YOLOv5 Semantic SLAM for Dynamic Scenes

When using deep learning networks for dynamic feature rejection in SLAM systems, problems such as a priori static object motion leading to disturbed build quality and accuracy and slow system runtime are prone to occur. In this paper, based on the ORB-SLAM2 system, we propose a method based on improved YOLOv5 networks combined with geometric constraint methods for SLAM map building in dynamic environments. First, this paper uses ShuffleNetV2 to lighten the YOLOv5 network, which increases the improved network’s operation speed without reducing the accuracy. At the same time, a pyramidal scene parsing network segmentation head is added to the head part of the YOLOv5 network to achieve semantic extraction in the environment, so that the improved YOLOv5 network has both target detection and semantic segmentation functions. In order to eliminate the objects with low dynamic features in the environment, this paper adopts the method of geometric constraints to extract and eliminate the dynamic features of the low dynamic objects. By combining the improved YOLOv5 network with the geometric constraint method, the robustness of the system is improved and the interference of dynamic targets in the construction of the SLAM system map is eliminated. The test results on the TUM dataset show that, when constructing a map in a dynamic environment, compared with the traditional ORB-SLAM2 algorithm, the accuracy of map construction in a dynamic environment is significantly improved. The absolute trajectory error is reduced by 97.7% compared with ORB-SLAM2, and the relative position error is reduced by 59.7% compared with ORB-SLAM2. Compared with DynaSLAM for dynamic scenes of the same type, the accuracy of map construction is slightly improved, but the maximum increase in keyframe processing time is 94.7%.

LCANet: A Lightweight Context-Aware Network for Bladder Tumor Segmentation in MRI Images

Accurate segmentation of the lesion area from MRI images is essential for diagnosing bladder cancer. However, the precise segmentation of bladder tumors remains a massive challenge due to their similar intensity distributions, various tumor morphologies, and blurred boundaries. While some seminal studies, such as those using CNNs combined with transformer segmentation methods, have made significant progress, (1) how to reduce the computational complexity of the self-attention mechanism in the transformer while maintaining performance and (2) how to build a better global feature fusion process to improve segmentation performance still require further exploration. Considering the complexity of bladder MRI images, we developed a lightweight context-aware network (LCANet) to automatically segment bladder lesions from MRI images. Specifically, the local detail encoder generates local-level details of the lesion, the lightweight transformer encoder models the global-level features with different resolutions, the pyramid scene parsing module extracts high-level and multiscale semantic features, and the decoder provides high-resolution segmentation results by fusing local-level details with global-level cues at the channel level. A series of empirical studies on T2-weighted MRI images from 86 patients show that LCANet achieves an overall Jaccard index of 89.39%, a Dice similarity coefficient of 94.08%, and a Class pixel accuracy of 94.10%. These advantages show that our method is an efficient tool that can assist in reducing the heavy workload of radiologists.

EWPCO-enabled Shepard convolutional neural network for classification of brain tumour using MRI image

ABSTRACT Numerous imaging techniques, like X-rays, Computerized Tomography (CT) scans, and ultrasound are utilized to predict brain tumours, but these imaging techniques experience difficulties in generating accurate results. To overcome such limitations, an effectual approach for the classification of brain cancer utilizing the proposed Exponentially Weighted Pelican Chimp Optimization-based Shepard Convolutional Neural Network (EWPCO-ShCNN) is introduced. At first, preprocessing is carried out employing a median filter, and Region of Interest (RoI) extraction and segmentation are performed utilizing a Pyramid Scene Parsing Network (PSP-Net), which is trained by Pelican Chimp Optimization (PCO) algorithm. After that, data augmentation and feature extraction are performed for more processing. Thereafter, the categorization is executed by ShCNN, which is instructed by the proposed Exponential Weighted Pelican Chimp Optimization (EWPCO) algorithm. Furthermore, the proposed EWPCO-ShCNN has attained better sensitivity of 95.90%, accuracy of 94.90% and specificity of 95.60% respectively.

Application of UAV RGB Images and Improved PSPNet Network to the Identification of Wheat Lodging Areas

As one of the main disasters that limit the formation of wheat yield and affect the quality of wheat, lodging poses a great threat to safety production. Therefore, an improved PSPNet (Pyramid Scene Parsing Network) integrating the Normalization-based Attention Module (NAM) (NAM-PSPNet) was applied to the high-definition UAV RGB images of wheat lodging areas at the grain-filling stage and maturity stage with the height of 20 m and 40 m. First, based on the PSPNet network, the lightweight neural network MobileNetV2 was used to replace ResNet as the feature extraction backbone network. The deep separable convolution was used to replace the standard convolution to reduce the amount of model parameters and calculations and then improve the extraction speed. Secondly, the pyramid pool structure of multi-dimensional feature fusion was constructed to obtain more detailed features of UAV images and improve accuracy. Then, the extracted feature map was processed by the NAM to identify the less significant features and compress the model to reduce the calculation. The U-Net, SegNet and DeepLabv3+ were selected as the comparison models. The results show that the extraction effect at the height of 20 m and the maturity stage is the best. For the NAM-PSPNet, the MPA (Mean Pixel Accuracy), MIoU (Mean Intersection over Union), Precision, Accuracy and Recall is, respectively, 89.32%, 89.32%, 94.95%, 94.30% and 95.43% which are significantly better than the comparison models. It is concluded that NAM-PSPNet has better extraction performance for wheat lodging areas which can provide the decisionmaking basis for severity estimation, yield loss assessment, agricultural operation, etc.

Multi-scale convolution based breast cancer image segmentation with attention mechanism in conjunction with war search optimization

Numerous studies have explored different techniques for segmenting breast cancer images, in particular deep learning-based Computer-Aided Diagnosis (CAD) has recently netted attention. However, due to their down-and-out pursuance, the existing approaches like FCN (Fully Convolutional Network), PSPNet (Pyramid Scene Parsing Network), U-Net, and SegNet still required improvement for offering better semantic segmentation while identifying breast cancer. In this paper, the newly proposed breast cancer tumor segmentation method consists of four steps pre-processing, augmentation, segmenting image using multi-scale convolution and multi- attention mechanisms respectively. The proposed method utilizes the ResNet (Residual Network) backbone network with multi-scale convolution for feature map prediction. Also, the effectiveness of the multi-channel attention module with a pyramid dilated nodule is employed for semantic segmentation. Gated axial, position, and channel attention are combined to create a multi-channel attention mechanism. Additionally, War Search Optimization (WSO) algorithm is being utilized to enhance the accuracy of the segmented images. Experimentations are conducted on two datasets, viz., Breast Cancer Cell Segmentation Database and Breast Cancer Semantic Segmentation (BCSS) Database, with different existing networks. The effectiveness of the network is evaluated based on various criteria in terms of precision, accuracy, recall, (mean Intersection of Union), (Intersection of Union), etc.

Using street view images and a geographical detector to understand how street-level built environment is associated with urban poverty: A case study in Guangzhou

Understanding and ending poverty has become one of the most important SDG (Sustainable Development Goals) all over the world. The street-level built environment is an important indicator to reflect urban poverty. However, traditional data such as satellite imagery may not provide fine-grained information of built environment at the street level. In recent years, street view image has become promising data for assessing an urban micro environment. This study aimed to use street view data and deep learning technique to examine the association between street-level built environment and urban poverty in Guangzhou, China, from a geographical heterogeneity perspective. First, we measured urban poverty in Guangzhou based on the Index of Multiple Deprivation. Second, we used the Pyramid Scene Parsing Network model for image segmentation and then performed principal component analysis to extract five major street view factors (i.e., vegetation enclosure sense, color complexity sense, road openness sense, sky openness sense, and building enclosure sense) from the street view data. Third, we conducted the geographical detector analysis to examine how street view factors is associated with urban poverty. Results suggested that vegetation enclosure sense, color complexity sense, and road openness sense are significantly related to the spatial heterogeneity of urban poverty. Among all factors, vegetation enclosure sense played a leading role. The results also confirmed the coexistence of different street view factors have association with the spatial heterogeneity of urban poverty. In conclusion, street-level built environment is generally associated with urban poverty, and therefore our proposed method can be considered as an efficiently method for identifying urban poor communities.

Deep learning-based concrete defects classification and detection using semantic segmentation

Visual damage detection of infrastructure using deep learning (DL)-based computational approaches can facilitate a potential solution to reduce subjectivity yet increase the accuracy of the damage diagnoses and accessibility in a structural health monitoring (SHM) system. However, despite remarkable advances with DL-based SHM, the most significant challenges to achieving the real-time implication are the limited available defects image databases and the selection of DL networks depth. To address these challenges, this research has created a diverse dataset with concrete crack (4087) and spalling (1100) images and used it for damage condition assessment by applying convolutional neural network (CNN) algorithms. CNN-classifier models are used to identify different types of defects and semantic segmentation for labeling the defect patterns within an image. Three CNN-based models—Visual Geometry Group (VGG)19, ResNet50, and InceptionV3 are incorporated as CNN-classifiers. For semantic segmentation, two encoder-decoder models, U-Net and pyramid scene parsing network architecture are developed based on four backbone models, including VGG19, ResNet50, InceptionV3, and EfficientNetB3. The CNN-classifier models are analyzed on two optimizers—stochastic gradient descent (SGD), root mean square propagation (RMSprop), and learning rates—0.1, 0.001, and 0.0001. However, the CNN-segmentation models are analyzed for SGD and adaptive moment estimation, trained with three different learning rates—0.1, 0.01, and 0.0001, and evaluated based on accuracy, intersection over union, precision, recall, and F1-score. InceptionV3 achieves the best performance for defects classification with an accuracy of 91.98% using the RMSprop optimizer. For crack segmentation, EfficientNetB3-based U-Net, and for spalling segmentation, IncenptionV3-based U-Net model outperformed all other algorithms, with an F1-score of 95.66 and 89.43%, respectively.

Robust Scene Parsing by Mining Supportive Knowledge From Dataset.

Scene parsing, or semantic segmentation, aims at labeling all pixels in an image with the predefined categories of things and stuff. Learning a robust representation for each pixel is crucial for this task. Existing state-of-the-art (SOTA) algorithms employ deep neural networks to learn (discover) the representations needed for parsing from raw data. Nevertheless, these networks discover desired features or representations only from the given image (content), ignoring more generic knowledge contained in the dataset. To overcome this deficiency, we make the first attempt to explore the meaningful supportive knowledge, including general visual concepts (i.e., the generic representations for objects and stuff) and their relations from the whole dataset to enhance the underlying representations of a specific scene for better scene parsing. Specifically, we propose a novel supportive knowledge mining module (SKMM) and a knowledge augmentation operator (KAO), which can be easily plugged into modern scene parsing networks. By taking image-specific content and dataset-level supportive knowledge into full consideration, the resulting model, called knowledge augmented neural network (KANN), can better understand the given scene and provide greater representational power. Experiments are conducted on three challenging scene parsing and semantic segmentation datasets: Cityscapes, Pascal-Context, and ADE20K. The results show that our KANN is effective and achieves better results than all existing SOTA methods.

Embedded Control Gate Fusion and Attention Residual Learning for RGB–Thermal Urban Scene Parsing

The semantic segmentation of road scenes is an important task in autonomous driving. Deep learning has enabled the development of a variety of semantic segmentation networks using RGB and depth data. However, poor lighting conditions and long-distance sensing limit the applicability of RGB and depth cameras. Nevertheless, many existing methods still rely on precise depth maps for scene segmentation. Unlike depth information, thermal imaging provides a visual heat representation that remains accurate under a variety of lighting conditions and over longer distances. For robust and accurate segmentation of scenes collected during autonomous driving, we used the advanced MobileNetV2 network for feature extraction and a fusion strategy with an embedded control gate. In addition, we adopted an encoder–decoder scheme for semantic segmentation and developed an attention residual learning strategy to restore the resolution of the feature map. Finally, semantic and boundary supervision is introduced to optimize parameters of the proposed network. Experimental results show that the proposed network outperforms existing networks on segmentation of urban scenes, and our network can be generalized to depth data.