Convolutional Network For Semantic Segmentation Research Articles

Self-Constructing Graph Convolutional Networks for Semantic Segmentation of Historical Maps

Self-Constructing Graph Convolutional Networks for Semantic Segmentation of Historical Maps

A Strip Dilated Convolutional Network for Semantic Segmentation

A Strip Dilated Convolutional Network for Semantic Segmentation

Evaluation of Deep Learning Segmentation Models for Detection of Pine Wilt Disease in Unmanned Aerial Vehicle Images

Pine wilt disease (PWD) is a serious threat to pine forests. Combining unmanned aerial vehicle (UAV) images and deep learning (DL) techniques to identify infected pines is the most efficient method to determine the potential spread of PWD over a large area. In particular, image segmentation using DL obtains the detailed shape and size of infected pines to assess the disease’s degree of damage. However, the performance of such segmentation models has not been thoroughly studied. We used a fixed-wing UAV to collect images from a pine forest in Laoshan, Qingdao, China, and conducted a ground survey to collect samples of infected pines and construct prior knowledge to interpret the images. Then, training and test sets were annotated on selected images, and we obtained 2352 samples of infected pines annotated over different backgrounds. Finally, high-performance DL models (e.g., fully convolutional networks for semantic segmentation, DeepLabv3+, and PSPNet) were trained and evaluated. The results demonstrated that focal loss provided a higher accuracy and a finer boundary than Dice loss, with the average intersection over union (IoU) for all models increasing from 0.656 to 0.701. From the evaluated models, DeepLLabv3+ achieved the highest IoU and an F1 score of 0.720 and 0.832, respectively. Also, an atrous spatial pyramid pooling module encoded multiscale context information, and the encoder–decoder architecture recovered location/spatial information, being the best architecture for segmenting trees infected by the PWD. Furthermore, segmentation accuracy did not improve as the depth of the backbone network increased, and neither ResNet34 nor ResNet50 was the appropriate backbone for most segmentation models.

MC-Net: Multiple max-pooling integration module and cross multi-scale deconvolution network

To better retain the deep features of an image and solve the sparsity problem of the end-to-end segmentation model, we propose a new deep convolutional network model for medical image pixel segmentation, called MC-Net. The core of this network model consists of four parts, namely, an encoder network, a multiple max-pooling integration module, a cross multiscale deconvolution decoder network and a pixel-level classification layer. Each max-pooling layer (the pooling size of each layer is different) is spliced after each convolution to achieve the translation invariance of the feature maps of each submodule. The multiscale convolution of each submodule in the decoder network is cross-fused with the feature maps generated by the corresponding multiscale convolution in the encoder network. Using the above feature map processing methods solves the sparsity problem after the max-pooling layer-generating matrix and enhances the robustness of the classification. We compare our proposed model with the well-known Fully Convolutional Networks for Semantic Segmentation (FCNs), DecovNet, PSPNet, U-net, SgeNet and other state-of-the-art segmentation networks such as HyperDenseNet, MS-Dual, Espnetv2, Denseaspp using one binary dataset and two multiclass dataset and obtain encouraging experimental results.

Domain Adaptation for Semantic Segmentation of Historical Panchromatic Orthomosaics in Central Africa

Multitemporal environmental and urban studies are essential to guide policy making to ultimately improve human wellbeing in the Global South. Land-cover products derived from historical aerial orthomosaics acquired decades ago can provide important evidence to inform long-term studies. To reduce the manual labelling effort by human experts and to scale to large, meaningful regions, we investigate in this study how domain adaptation techniques and deep learning can help to efficiently map land cover in Central Africa. We propose and evaluate a methodology that is based on unsupervised adaptation to reduce the cost of generating reference data for several cities and across different dates. We present the first application of domain adaptation based on fully convolutional networks for semantic segmentation of a dataset of historical panchromatic orthomosaics for land-cover generation for two focus cities Goma-Gisenyi and Bukavu. Our experimental evaluation shows that the domain adaptation methods can reach an overall accuracy between 60% and 70% for different regions. If we add a small amount of labelled data from the target domain, too, further performance gains can be achieved.

An Improved Fully Convolutional Network for Semantic Segmentation

A novel loss function based on category distribution for semantic segmentation is proposed in this paper. The new loss function is composed of two parts: (1) Pixel-wise cross-entropy. (2) Category distribution loss proposed. Most of existing semantic segmentation networks adopt a single pixel-wise cross-entropy loss function, which guides the network to independently predict the class each pixel belongs to. The downside is that ignoring global information of the image -- the distribution of all kinds of objects in the image, resulting in an unsatisfactory segmentation result. Category distribution loss proposed in this paper obtins category distribution information of the whole image by calculating the percentage of pixels to each class of objets. Acquired category distribution information can be represented as a feature vector, the distance of two vectors between prediction and ground truth forms category distribution loss. The new loss function measures the difference between pixels as well as the difference between global information. We apply the new loss to several classical networks, its pixel accuracy and mIoU accuracy on two benchmark datasets CamVid and Pascal VOC2012 are improved compared to using only pixel-wise cross-entropy. In addition, we also introduced the attention mechanism, which proved to be able to improve segmentation accuracy of the networks as well.

Evidential fully convolutional network for semantic segmentation

We propose a hybrid architecture composed of a fully convolutional network (FCN) and a Dempster-Shafer layer for image semantic segmentation. In the so-called evidential FCN (E-FCN), an encoder-decoder architecture first extracts pixel-wise feature maps from an input image. A Dempster-Shafer layer then computes mass functions at each pixel location based on distances to prototypes. Finally, a utility layer performs semantic segmentation from mass functions and allows for imprecise classification of ambiguous pixels and outliers. We propose an end-to-end learning strategy for jointly updating the network parameters, which can make use of soft (imprecise) labels. Experiments using three databases (Pascal VOC 2011, MIT-scene Parsing and SIFT Flow) show that the proposed combination improves the accuracy and calibration of semantic segmentation by assigning confusing pixels to multi-class sets.

Multilevel feature fusion dilated convolutional network for semantic segmentation

Recently, convolutional neural network (CNN) has led to significant improvement in the field of computer vision, especially the improvement of the accuracy and speed of semantic segmentation tasks, which greatly improved robot scene perception. In this article, we propose a multilevel feature fusion dilated convolution network (Refine-DeepLab). By improving the space pyramid pooling structure, we propose a multiscale hybrid dilated convolution module, which captures the rich context information and effectively alleviates the contradiction between the receptive field size and the dilated convolution operation. At the same time, the high-level semantic information and low-level semantic information obtained through multi-level and multi-scale feature extraction can effectively improve the capture of global information and improve the performance of large-scale target segmentation. The encoder–decoder gradually recovers spatial information while capturing high-level semantic information, resulting in sharper object boundaries. Extensive experiments verify the effectiveness of our proposed Refine-DeepLab model, evaluate our approaches thoroughly on the PASCAL VOC 2012 data set without MS COCO data set pretraining, and achieve a state-of-art result of 81.73% mean interaction-over-union in the validate set.

Pixel Level Instance Segmentation using Single Shot Detectors and Semantic Segmentation Networks

Object detection algorithms are vital in computer vision applications especially the one regarding segmentation of objects in a scene. Instance segmentation using mask RCNN is the state of the art computer vision application using deep neural networks. However it has a complex pipeline for training and real-time testing. Single shot detectors based on CNN such as YOLO are robust in real time object detection compared to Faster RCNN. Combining convolutional networks for semantic segmentation along with single shot object detectors can give superior instance segmentation in real time at less computational complexity. Independent training of instance detection and segmentation networks makes it possible to create application specific pipelines based on context. Lighter architecture using single shot detectors and segmentation networks reduces latency on common processors thereby achieving higher frame rate. Class specific training of detectors improves the average precision of detection. The entire architecture is developed in python over Keras deeplearning framework.OpenCV library is used to render and process the input and output images.

Semantic segmentation-based parking space detection with standalone around view monitoring system

An auto-parking system is one of the promising technologies to reduce accidents and enhance driver convenience in parking lots. To accomplish collision-free parking, precise and robust parking space detection is required. However, harsh conditions such as varied illumination in outdoor parking lots and high reflection in indoor parking lots degrade the reliability of parking space detection. In this paper, we propose a unified structure for parking space detection to detect parking slot markings and static obstacles. A fully convolutional network for semantic segmentation can immediately identify free spaces, slot markings, vehicles, and other objects without using a range sensor or 3D reconstruction algorithm. Furthermore, a vertical grid encoding method can simultaneously detect unoccupied slots identified by parking slot markings and empty spaces created by surrounding static objects without sensor fusion. Experimental results show the robustness of the proposed method in various different parking scenarios. Even in challenging conditions such as dark shaded or high-glare areas, the detection performance maintains a precision rate of 96.81% and recall rate of 97.80%.

Multi-scale deep context convolutional neural networks for semantic segmentation

Recent years have witnessed the great progress for semantic segmentation using deep convolutional neural networks (DCNNs). This paper presents a novel fully convolutional network for semantic segmentation using multi-scale contextual convolutional features. Since objects in natural images tend to be with various scales and aspect ratios, capturing the rich contextual information is very critical for dense pixel prediction. On the other hand, when going deeper in convolutional layers, the convolutional feature maps of traditional DCNNs gradually become coarser, which may be harmful for semantic segmentation. According to these observations, we attempt to design a multi-scale deep context convolutional network (MDCCNet), which combines the feature maps from different levels of network in a holistic manner for semantic segmentation. The segmentation outputs of MDCCNets are further enhanced using dense connected conditional random fields (CRF). The proposed network allows us to fully exploit local and global contextual information, ranging from an entire scene to every single pixel, to perform pixel-wise label estimation. The experimental results demonstrate that our method outperforms or is comparable to state-of-the-art methods on PASCAL VOC 2012 and SIFTFlow semantic segmentation datasets.

Fully Convolutional Networks for Semantic Segmentation of Very High Resolution Remotely Sensed Images Combined With DSM

Recently, approaches based on fully convolutional networks (FCN) have achieved state-of-the-art performance in the semantic segmentation of very high resolution (VHR) remotely sensed images. One central issue in this method is the loss of detailed information due to downsampling operations in FCN. To solve this problem, we introduce the maximum fusion strategy that effectively combines semantic information from deep layers and detailed information from shallow layers. Furthermore, this letter develops a powerful backend to enhance the result of FCN by leveraging the digital surface model, which provides height information for VHR images. The proposed semantic segmentation scheme has achieved an overall accuracy of 90.6% on the ISPRS Vaihingen benchmark.

Fully Convolutional Networks for Semantic Segmentation

Convolutional networks are powerful visual models that yield hierarchies of features. We show that convolutional networks by themselves, trained end-to-end, pixels-to-pixels, improve on the previous best result in semantic segmentation. Our key insight is to build "fully convolutional" networks that take input of arbitrary size and produce correspondingly-sized output with efficient inference and learning. We define and detail the space of fully convolutional networks, explain their application to spatially dense prediction tasks, and draw connections to prior models. We adapt contemporary classification networks (AlexNet, the VGG net, and GoogLeNet) into fully convolutional networks and transfer their learned representations by fine-tuning to the segmentation task. We then define a skip architecture that combines semantic information from a deep, coarse layer with appearance information from a shallow, fine layer to produce accurate and detailed segmentations. Our fully convolutional networks achieve improved segmentation of PASCAL VOC (30% relative improvement to 67.2% mean IU on 2012), NYUDv2, SIFT Flow, and PASCAL-Context, while inference takes one tenth of a second for a typical image.