Multi-scale Segmentation Research Articles

Extracting Wetland Type Information with a Deep Convolutional Neural Network.

Wetlands have important ecological value. The application of wetland remote sensing is essential for the timely and accurate analysis of the current situation in wetlands and dynamic changes in wetland resources, but high-resolution remote sensing images display nonobvious boundaries between wetland types. However, high classification accuracy and time efficiency cannot be guaranteed simultaneously. Extraction of wetland type information based on high-spatial-resolution remote sensing images is a bottleneck that has hindered wetland development research and change detection. This paper proposes an automatic and efficient method for extracting wetland type information. First, the object-oriented multiscale segmentation method is used to realize the fine segmentation of high-resolution remote sensing images, and then the deep convolutional neural network model AlexNet is used to classify automatically the types of wetland images. The method is verified in a case study involving field-measured data, and the classification results are compared with those of traditional classification methods. The results show that the proposed method can more accurately and efficiently extract different wetland types in high-resolution remote sensing images than the traditional classification methods. The proposed method will be helpful in the extension and application of wetland remote sensing technology and will provide technical support for the protection, development, and utilization of wetland resources.

Characteristics and Carbon Storage of a Typical Mangrove Island Ecosystem in Beibu Gulf, South China Sea

By studying the structural characteristics and carbon storage of the mangrove island ecosystem in the Beibu Gulf, this study provides a scientific basis for mangrove ecological compensation in the coastal areas of Guangxi, South China Sea. On the basis of the unmanned aerial vehicle remote sensing images and a sample plot survey, the object-oriented multi-scale segmentation algorithm is used to extract the mangrove community type information, and one-way analysis of variance is conducted to analyse the structural characteristics of the mangrove community. The carbon storage and carbon density of different mangrove ecosystems were obtained based on the allometric growth equation of mangrove plants. The analysis yielded four main results. (1) The island group covers about 27.10 ha, 41.32% (11.20 ha) of which represents mangrove areas. The mangrove forest is widely distributed in the tidal flats around the islands. (2) The main mangrove types were Aegiceras corniculatum, Kandelia obovata + Aegiceras corniculatum, Avicennia marina + Aegiceras corniculatum and Avicennia marina communities. (3) Amongst the mangrove plants, Avicennia marina had the highest biomass (18.52 kg plant–1), followed by Kandelia obovata (7.84 kg plant–1) and Aegiceras corniculatum (3.85 kg plant–1). (4) The mangrove carbon density difference was significant. Kandelia obovata had the highest carbon density (148.03 t ha–1), followed by Avicennia marina (104.79 t ha–1) and Aegiceras corniculatum (99.24 t ha–1). The carbon storage of the mangrove island ecosystem was 1194.70 t, which was higher than in other areas with the same latitude. The carbon sequestration capacity of the mangrove was relatively strong.

Multi-scale organs image segmentation method improved by squeeze-and-attention based on partially supervised learning

Radiotherapy is one of most treatments for tumors. To accurately control the radiation dose distribution and lessen the radiation damage to normal tissues and organs in radiotherapy, it is essential to delineate organs at risk (OARs) precisely. However, manual delineating and some traditional methods are labor-intensive and time-consuming. There is an urgent need for fast and precise segmentation methods in radiotherapy. This paper proposes a fully automatic segmentation method based on the 3D U-Net for multi-organ in head and neck. It introduces squeeze-and-attention blocks to gather multi-scale context information and the receptive field block to balance the performance between large-sized and small-sized organs. Furthermore, it is trained by the marginal and exclusion loss function in a partially supervised learning mode. We evaluated the model with dice similarity coefficient (DSC), 95% Hausdorff distance (95HD) and inference time. Its average DSC is 0.829, which is 4.5%, 3.2%, and 2.4% higher than AnatomyNet's, nnU-net's, and FocusNet's, respectively, and its average 95HD is 2.19. Moreover, its inference time and parameters are 63% and 60% less than FocusNetv2's. For the segmentation of OARs in head and neck, our model is more accurate than AnatomyNet, faster than FocusNetv2, and better balances between segmentation accuracy and inference time. It demonstrates that our method is more applicable for clinical treatment.

A multi-temporal method for detection of underground natural gas leakage using hyperspectral imaging

Hyperspectral remote sensing images can indirectly detect underground natural gas leakage through the spectral and spatial variation of surface vegetation. However, due to the complexity of surface environment and the phenomenon of "different samples demonstrating the same spectrum", using a spectral-spatial based method may result in misidentification. The spectral and spatial characteristics of surface vegetation caused by natural gas micro-leakage will change with the increase of stress time and the growth of vegetation. Therefore, a field simulation experiment of natural gas micro-leakage vegetation stress was set up. Multi-temporal hyperspectral images of bean, corn and grassland were obtained, and a new spectral-spatial-temporal based methodology was proposed to identify natural gas micro-leakage points and stressed vegetation areas. First, multi-scale segmentation and rotation forest classification algorithm were used to conduct a spectral-spatial based classification for natural gas leakage stressed vegetation on each phase of the image. The precision and recall rate of the leak point detection results were 93% and 88%, respectively. Then, the classification and recognition result images of different time phases on the same plot were weighted stacked to obtain a spectral-spatial-temporal features fusion image. Finally, the fusion image was used to construct a spectral-spatial-temporal features fused recognition model to determine the final natural gas micro-leakage stress vegetation areas and the locations of suspected leak points. The precision and recall rate of final detection results were both 100%.

Defect Detection of Subway Tunnels Using Advanced U-Net Network

In this paper, we present a novel defect detection model based on an improved U-Net architecture. As a semantic segmentation task, the defect detection task has the problems of background–foreground imbalance, multi-scale targets, and feature similarity between the background and defects in the real-world data. Conventionally, general convolutional neural network (CNN)-based networks mainly focus on natural image tasks, which are insensitive to the problems in our task. The proposed method has a network design for multi-scale segmentation based on the U-Net architecture including an atrous spatial pyramid pooling (ASPP) module and an inception module, and can detect various types of defects compared to conventional simple CNN-based methods. Through the experiments using a real-world subway tunnel image dataset, the proposed method showed a better performance than that of general semantic segmentation including state-of-the-art methods. Additionally, we showed that our method can achieve excellent detection balance among multi-scale defects.

Information Extraction from High-Resolution Remote Sensing Images Based on Multi-Scale Segmentation and Case-Based Reasoning

In object-oriented information extraction from high-resolution remote sensing images, the segmentation and classification of images involves considerable manual participation, which limits the development of automation and intelligence for these purposes. Based on the multi-scale segmentation strategy and case-based reasoning, a new method for extracting high-resolution remote sensing image information by fully using the image and nonimage features of the case object is proposed. Feature selection and weight learning are used to construct a multi-level and multi-layer case library model of surface cover classification reasoning. Combined with image mask technology, this method is applied to extract surface cover classification information from remote sensing images using different sensors, time, and regions. Finally, through evaluation of the extraction and recognition rates, the accuracy and effectiveness of this method was verified.

Single image dehazing based on multi-scale segmentation and deep learning

Single image dehazing based on multi-scale segmentation and deep learning

Automatic and Accurate Extraction of Sea Ice in the Turbid Waters of the Yellow River Estuary Based on Image Spectral and Spatial Information

Sea ice is an important part of the global cryosphere and an important variable in the global climate system. Sea ice also presents one of the major natural disasters in the world. The automatic and accurate extraction of sea ice extent is of great significance for the study of climate change and disaster prevention. The accuracy of sea ice extraction in the Yellow River Estuary is low due to the large dynamic changes in the suspended particulate matter (SPM). In this study, a set of sea ice automatic extraction method systems combining image spectral information and textural information is developed. First, a sea ice spectral information index that can adapt to sea areas with different turbidity levels is developed to mine the spectral information of different types of sea ice. In addition, the image’s textural feature parameters and edge point density map are extracted to mine the spatial information concerning the sea ice. Then, multi-scale segmentation is performed on the image. Finally, the OTSU algorithm is used to determine the threshold to achieve automatic sea ice extraction. The method was successfully applied to Gaofen-1 (GF1), Sentinel-2, and Landsat 8 images, where the extraction accuracy of sea ice was over 93%, which was more than 5% higher than that of SVM and K-Means. At the same time, the method was applied to the Liaodong Bay area, and the extraction accuracy reached 99%. These findings reveal that the method exhibits good reliability and robustness.

Land use classification of open-pit mine based on multi-scale segmentation and random forest model.

The mining industry production is an important pillar industry in China, while its extensive production activities have led to several ecological and environmental problems. Earth observation technology using high-resolution satellite imagery can help us efficiently obtain information on surface elements, surveying and monitoring various land occupation issues arising from open-pit mining production activities. Conventional pixel-based interpretation methods for high-resolution remote sensing images are restricted by “salt and pepper” noise caused by environmental factors, making it difficult to meet increasing requirements for monitoring accuracy. With the Jingxiang phosphorus mining area in Jingmen Hubei Province as the studied area, this paper uses a multi-scale segmentation algorithm to extract large-scale main characteristic information using a layered mask method based on the hierarchical structure of the image object. The remaining characteristic elements were classified and extracted in combination with the random forest model and characteristic factors to obtain land occupation information related mining industry production, which was compared with the results of the Classification and Regression Tree model. 23 characteristic factors in three aspects were selected, including spectral, geometric and texture characteristics. The methods employed in this study achieved 86% and 0.78 respectively in overall extraction accuracy analysis and the Kappa coefficient analysis, compared to 79% and 0.68 using the conventional method.

Comparison of Segmentation-Based Image Compression Using Threshold, Region Growing, and Edge Detection

In the presenting paper we are dealing with to develop a lossless image compression (IC) method to utilize spatial redundancies inbuilt in image data which employs a best possible amount of segmentation information. To obtaining Multiscale segmentation we are using a earlier proposed transform which gives a tree-structured segmentation of the picture into regions which are identified by grayscale homogeneity. In the given proposed algor we have to shorten the tree to controlling the size and no of regions so that we can get a rate balance between the derived the coding gain and the operating cost inherent in coding the segmented data. Another uniqueness of the given proposed approach is that we are using an image model contain individual descriptions of the pixels lying close to the edges of a section and others lying in the center. In our results we can see that this proposed algorithm is providing better performance comparable to all the best available methods and it provides 15-20% better compression if we compare it with the JPEG lossless image compression standard for a enormous variety of images.

Research on classification method based on multi-scale segmentation and hierarchical classification

This paper transmission line corridors covering area in Hebei north area as the research object to explore multi-scale segmentation threshold suitable for Hebei north image, found applicable to Hebei north region segmentation threshold rules. Main methods are the object-oriented multi-scale segmentation and hierarchical classification, using image segmentation principle, make full use of high resolution image rich features such as shape, texture, object relationships. It is used for the follow-up investigation of hidden danger of external damage of power transmission channel in northern Hebei region. The main conclusions of the experiment are as follows: 1. By comparing and analyzing the results of five groups of different thresholds (40, 50, 60, 70 and 80), it is concluded that the single threshold of the multi-scale segmentation method suitable for most mountainous images in the study area is 60, and this method can achieve high-precision ground object classification for images in northern Hebei region. 2. The ground feature cover classification method that is more suitable for the parallel processing of a large number of study areas is the object-oriented hierarchical classification method, and preliminary exploration results of detailed parameters have been achieved.

A Modified Residual Network Based on Multi-scale Segmentation for Aerobics Motion Image Recognition

<p>Image recognition is an important field in artificial intelligence, it makes use of the computer to conduct image processing, analysis and understanding to recognize a variety of different objects. And it uses a series of enhancement and reconstruction methods to effectively improve the image quality. Traditional deep Convolutional Neural Network (DCNN) not only improves the recognition accuracy, but also reduces the recognition speed. How to improve the speed while maintaining the accuracy has become an important direction in image recognition. In this paper, we propose a modified Residual network based on multi-scale segmentation for aerobics motion image recognition. The new residual network has the characteristics of shorter network length and faster recognition speed. First, it reduces the length of the network and gets a new residual network with seven layers. Then, combining with the multi-scale segmentation method, an image recognition residual network is obtained. Finally, experiments on the CIFAR10 dataset, the results show that the proposed new motion image recognition method has better recognition accuracy and faster recognition speed.</p> <p> </p>

Multiple Superpixel Graphs Learning Based on Adaptive Multiscale Segmentation for Hyperspectral Image Classification

Hyperspectral image classification (HSIC) methods usually require more training samples for better classification performance. However, a large number of labeled samples are difficult to obtain because it is cost- and time-consuming to label an HSI in a pixel-wise way. Therefore, how to overcome the problem of insufficient accuracy and stability under the condition of small labeled training sample size (SLTSS) is still a challenge for HSIC. In this paper, we proposed a novel multiple superpixel graphs learning method based on adaptive multiscale segmentation (MSGLAMS) for HSI classification to address this problem. First, the multiscale-superpixel-based framework can reduce the adverse effect of improper selection of a superpixel segmentation scale on the classification accuracy while saving the cost to manually seek a suitable segmentation scale. To make full use of the superpixel-level spatial information of different segmentation scales, a novel two-steps multiscale selection strategy is designed to adaptively select a group of complementary scales (multiscale). To fix the bias and instability of a single model, multiple superpixel-based graphical models obatined by constructing superpixel contracted graph of fusion scales are developed to jointly predict the final results via a pixel-level fusion strategy. Experimental results show that the proposed MSGLAMS has better performance when compared with other state-of-the-art algorithms. Specifically, its overall accuracy achieves 94.312%, 99.217%, 98.373% and 92.693% on Indian Pines, Salinas and University of Pavia, and the more challenging dataset Houston2013, respectively.

Ancient mural segmentation based on a deep separable convolution network

Traditional methods for ancient mural segmentation have drawbacks, including fuzzy target boundaries and low efficiency. Targeting these problems, this study proposes a pyramid scene parsing MobileNetV2 network (PSP-M) by fusing a deep separable convolution-based lightweight neural network with a multiscale image segmentation model. In this model, deep separable convolution-fused MobileNetV2, as the backbone network, is embedded in the image segmentation model, PSPNet. The pyramid scene parsing structure, particularly owned by the two models, is used to process the background features of images, which aims to reduce feature loss and to improve the efficiency of image feature extraction. In the meantime, atrous convolution is utilized to expand the perceptive field, aiming to ensure the integrity of image semantic information without changing the number of parameters. Compared with traditional image segmentation models, PSP-M increases the average training accuracy by 2%, increases the peak signal-to-noise ratio by 1–2 dB and improves the structural similarity index by 0.1–0.2.

An Exploratory Evaluation of Multiscale Data Analysis for Landform Element Detection on High-Resolution DEM

The representation of landform element varies over multiple scales, or multiresolution digital elevation model (DEM) and its derivatives. When more details of land surfaces are available to be characterized based on the existing high spatial resolution elevation products, the influence of scale variation might become more significant. This poses a demand for determining a scale-independent approach being competent to support multiscale landform element detection on high-resolution DEMs. Although the practicability of the state-of-the-art scale-independent approaches have been reported on moderate-resolution DEMs, how these approaches perform based on the multiscale data including high and moderate spatial-resolution DEMs is still unexplored. This letter evaluates the performance of four scale-independent techniques including filtering, spatial pyramid, multiscale segmentation, and spatial-contextual approach in landform element detection on different spatial resolution DEMs. The experimental results show that spatial–contextual approach is more effective to support multiscale landform element detection than others.

Object-Based Sorted-Histogram Similarity Measurement for Detecting Land Cover Change With VHR Remote Sensing Images

Land cover change detection (LCCD) with very high-resolution (VHR) remote sensing images has been widely used in various applications. However, pseudo-changes and noise usually affect the performance of detection map. In this letter, an object-oriented sorted-histogram similarity measurement (OSSM) is proposed for measuring the change magnitude between bi-temporal remote sensing images. First, multi-scale objects are acquired for the post-event image using a multi-scale segmentation algorithm, and then the pixels within each object are considered to construct the pairwise histograms and the bin of each histogram is sorted in descending order. Second, a bin-to-bin (B2B) distance is defined to measure the change magnitude between the pairwise object-based histograms, and the change magnitude image (CMI) is generated after all the bi-temporal images are scanned object by object. Finally, a simple yet effective method called Otsu is used to divide the CMI into binary change detection maps. The experiments on three pairs of VHR images produced promising results compared with five popular LCCD approaches, for example, the improvement is about 2.5% for F-score.

A Hierarchical Region-Merging Algorithm for 3-D Segmentation of Individual Trees Using UAV-LiDAR Point Clouds

Over an extended period, remote-sensing-based individual tree analysis has played a critical role in modern forest inventory and management research. The segmentation of individual trees from aerial point clouds usually depends on the characteristics of peak-like uplift on the crown surface; however, the performance inevitably decreases with increasing visibility of such features in point clouds, especially for high-density forests. Herein, we developed a novel hierarchical region-merging algorithm that first over-segmented the entire forest scene based on local density and then merged the over-segmented partitions into pairs through a stepwise optimal process to produce the final segmentation. In the region-merging method, a global merging cost was introduced to shift from local detection of crown features to use the overall compactness of forest point clouds. The experiments were conducted using unmanned aerial vehicle light detection and ranging (UAV-LiDAR) point clouds from three coniferous stands with different densities and a high-density coniferous and broad-leaved mixed stand. A total of 5510 field-measured trees in 36 plots were used to assess the accuracy of the proposed method. Our method achieved F-scores of 0.91, 0.88, 0.84, and 0.80 for low- (~700 stems/ha), medium- (~1000 stems/ha), and high-density (~2000 stems/ha) conifer stands and coniferous and broad-leaved mixed forests (~1800 stems/ha), respectively. Compared with the classical individual tree segmentation methods (marker-controlled watershed segmentation and point cloud region-growing algorithm), our method obtained comparable performance in low-density conifer stands and superior performance in the other stands. Furthermore, the region-merging algorithm could detect 10% more suppressed trees on average, which led to an apparent improvement in detection accuracy. The proposed algorithm provides a flexible segmentation framework that could be further improved by a different design that merges costs or applies multiscale segmentation with different stopping criteria.

Hybrid Segmentation Method for Boundary Delineation of Agricultural Fields in Multitemporal Satellite Image using HS-PSO-FCNN

Demonstrating the ability to map and track the geographic distribution of agricultural resources is crucial for enhancing agricultural production and guaranteeing food security in numerous parts of the world. A satellite-based technique can significantly cut expenses and boost productivity when compared to traditional field surveys, allowing for the comprehensive mapping of agricultural resources across broad geographic areas. This work focuses on delineating agricultural fields from multispectral satellite images, a field is a piece of land used for agriculture that contains specific land, water, and other features. In this paper, morphological leveling and morphological continuity are utilized to formalize a morphological characteristic. In Multiscale Segmentation (MS), this attribute is defined by taking the derivative of the morphological profile. MS is very beneficial for complicated image scenarios such as aerial or fine-resolution satellite imagery. Based on Hybrid Segmentation of Particle-Swarm-Optimization with a Fully Convolution Network (HS-PSO-FCN), the efficiency of the work has been increased. PSNR processing time has been taken as output metrics used to assess the quality.

Multi-scale brain tumor segmentation combined with deep supervision.

Fully convolutional neural networks (FCNNs) have achieved good performance in the field of medical image segmentation. FCNNs that use multimodal images and multi-scale feature extraction have higher accuracy for brain tumor segmentation. Therefore, we have made some improvements to U-Net for fully automated segmentation of gliomas using multimodal images. And we named it multi-scale dilate network with deep supervision (MSD-Net). MSD-Net is a symmetrical structure composed of a down-sampling process and an up-sampling process. In the down-sampling process, we use the multi-scale feature extraction block (ME) to extract multi-scale features and focus on primary features. Unlike other methods, ME consists of dilate convolution and standard convolution. Dilate convolution extracts multi-scale informations and standard convolution merges features of different scales. Hence, the output of the ME contains local information and global information. During the up-sampling process, we add a deep supervision block (DSB), which can shorten the length of back-propagation. In this paper, we pay more attention to the importance of shallow features for feature restoration. Our network validated in the BraTS17's validation dataset. The DSC scores of MSD-Net for complete tumor, tumor core and enhancing tumor were 0.88, 0.81 and 0.78, respectively, which outperforms most networks. This study shows that ME enhances the feature extraction ability of the network and improves the accuracy of segmentation results. DSB speeds up the convergence of the network. In addition, we should also pay attention to the contribution of shallow features to feature restoration.

Segmentation Scale Effect Analysis in the Object-Oriented Method of High-Spatial-Resolution Image Classification.

High-spatial-resolution images play an important role in land cover classification, and object-based image analysis (OBIA) presents a good method of processing high-spatial-resolution images. Segmentation, as the most important premise of OBIA, significantly affects the image classification and target recognition results. However, scale selection for image segmentation is difficult and complicated for OBIA. The main challenge in image segmentation is the selection of the optimal segmentation parameters and an algorithm that can effectively extract the image information. This paper presents an approach that can effectively select an optimal segmentation scale based on land object average areas. First, 20 different segmentation scales were used for image segmentation. Next, the classification and regression tree model (CART) was used for image classification based on 20 different segmentation results, where four types of features were calculated and used, including image spectral bands value, texture value, vegetation indices, and spatial feature indices, respectively. WorldView-3 images were used as the experimental data to verify the validity of the proposed method for the selection of the optimal segmentation scale parameter. In order to decide the effect of the segmentation scale on the object area level, the average areas of different land objects were estimated based on the classification results. Experiments based on the multi-scale segmentation scale testify to the validity of the land object’s average area-based method for the selection of optimal segmentation scale parameters. The study results indicated that segmentation scales are strongly correlated with an object’s average area, and thus, the optimal segmentation scale of every land object can be obtained. In this regard, we conclude that the area-based segmentation scale selection method is suitable to determine optimal segmentation parameters for different land objects. We hope the segmentation scale selection method used in this study can be further extended and used for different image segmentation algorithms.