Segmentation Of High-resolution Images Research Articles

Lightweight land cover classification via semantic segmentation of remote sensing imagery and analysis of influencing factors

Land cover classification is of great value and can be widely used in many fields. Earlier land cover classification methods used traditional image segmentation techniques, which cannot fully and comprehensively extract the ground information in remote sensing images. Therefore, it is necessary to integrate the advanced techniques of deep learning into the study of semantic segmentation of remote sensing images. However, most of current high-resolution image segmentation networks have disadvantages such as large parameters and high network training cost. In view of the problems above, a lightweight land cover classification model via semantic segmentation, DeepGDLE, is proposed in this paper. The model DeepGDLE is designed on the basis of DeeplabV3+ network and utilizes the GhostNet network instead of the backbone feature extraction network in the encoder. Using Depthwise Separable Convolution (DSC) instead of dilation convolution. This reduces the number of parameters and increases the computational speed of the model. By optimizing the dilation rate of parallel convolution in the ASPP module, the “grid effect” is avoided. ECANet lightweight channel attention mechanism is added after the feature extraction module and the pyramid pooling module to focus on the important weights of the model. Finally, the loss function Focal Loss is utilized to solve the problem of category imbalance in the dataset. As a result, the model DeepGDLE effectively reduces the parameters of the network model and the network training cost. And extensive experiments compared with several existing semantic segmentation algorithms such as DeeplabV3+, UNet, SegNet, etc. show that DeepGDLE improves the quality and efficiency of image segmentation. Therefore, compared to other networks, the DeepGDLE network model can be more effectively applied to land cover classification. In addition, in order to investigate the effects of different factors on the semantic segmentation performance of remote sensing images and to verify the robustness of the DeepGDLE model, a new remote sensing image dataset, FRSID, is constructed in this paper. This dataset takes into account more influences than the public dataset. The experimental results show that on the WHDLD dataset, the experimental metrics mIoU, mPA, and mRecall of the proposed model, DeepGDLE, are 62.29%, 72.85%, and 72.46%, respectively. On the FRSID dataset, the metrics mIoU, mPA, and mRecall are 65.89%, 74.43%, and 74.08%, respectively. For the future scope of research in this field, it may focus on the fusion of multi-source remote sensing data and the intelligent interpretation of remote sensing images.

Sustainable Inference of Remote Sensing Data by Recursive Semantic Segmentation – a Flood Extent Mapping Study

In times of climate change and large machine learning models with petabytes of training data, the demand for responsible AI methodologies is more pressing than ever. This is in-particular true for the vast amount of remote sensing data. Its value to explore and inform about earth processes is of paramount importance, especially in times of global warming. Thus, it is nearly ironic that such applications can be the cause of substantial green-house-gas emissions, through energy demands from compute and communication systems. Thus, this study aims at reducing the data transfer between data centers while maintaining near-real time insights from remote sensing data. A recursive inference approach is introduced consisting of three steps: i) Data pyramid preparation in the host data center (sequence of upscaled raster data). ii) The transfer of low-resolution images to the service data center, where a deep-learning model performs a semantic segmentation task, including an uncertainty estimation. Images of higher resolution are then requested and segmented in a recursive fashion, in areas of high uncertainty only. iii) Finally, the merging of the predictions at different resolutions is performed to result in the final pixel-wise segmentation at scale. The method is demonstrated on synthetic and real-world data for a flood mapping task. A U-Net encoder-decoder model is used for the semantic segmentation task, using Monte-Carlo dropout to result in the uncertainty map. The proof-of-concept demonstrated a 35-38% performance gain per transferred pixel compared to high-resolution image segmentation only. Further, we perform a scaling study to estimate the true potential of the recursive inference approach, indicating the potential to reduce a data transfer by up to 98%, considering four hierarchy levels in the data pyramid. With this study, we hope to have contributed a small but important step towards sustainable machine learning.

Comparing the Semantic Segmentation of High-Resolution Images Using Deep Convolutional Networks: SegNet, HRNet, CSE-HRNet and RCA-FCN

Comparing the Semantic Segmentation of High-Resolution Images Using Deep Convolutional Networks: SegNet, HRNet, CSE-HRNet and RCA-FCN

An Efficient and Light Transformer-Based Segmentation Network for Remote Sensing Images of Landscapes

High-resolution image segmentation for landscape applications has garnered significant attention, particularly in the context of ultra-high-resolution (UHR) imagery. Current segmentation methodologies partition UHR images into standard patches for multiscale local segmentation and hierarchical reasoning. This creates a pressing dilemma, where the trade-off between memory efficiency and segmentation quality becomes increasingly evident. This paper introduces the Multilevel Contexts Weighted Coupling Transformer (WCTNet) for UHR segmentation. This framework comprises the Mult-level Feature Weighting (MFW) module and Token-based Transformer (TT) designed to weigh and couple multilevel semantic contexts. First, we analyze the multilevel semantics within a local patch without image-level contextual reasoning. It avoids complex image-level contextual associations and eliminates the misleading information carried. Second, MFW is developed to weigh shallow and deep features for enhancing object-related attention at different grain sizes from multilevel semantics. Third, the TT module is introduced to couple multilevel semantic contexts and transform them into semantic tokens using spatial attention. Then, we can capture token interactions and obtain clearer local representations. The suggested contextual weighting and coupling of single-scale patches empower WCTNet to maintain a well-balanced relationship between accuracy and computational overhead. Experimental results show that WCTNet achieves state-of-the-art performance on two UHR datasets of DeepGlobe and Inria Aerial.

Digital histology of tissue with Mueller microscopy and FastDBSCAN.

We present the results of the automated post-processing of Mueller microscopy images of skin tissue models with a new fast version of the algorithm of density-based spatial clustering of applications with noise (FastDBSCAN) and discuss the advantages of its implementation for digital histology of tissue. We demonstrate that using the FastDBSCAN algorithm, one can produce the diagnostic segmentation of high resolution images of tissue by several orders of magnitude faster and with high accuracy (>97%) compared to the original version of the algorithm.

Multiscale depthwise separable convolution based network for high-resolution image segmentation

ABSTRACT Deep learning-based segmentation methods have demonstrated significant performance over their traditional counterparts. However, striving for better accuracy with such networks usually leads to the deterioration of the network’s computational efficiency, thereby rendering them inefficient for deployment on resource constraint devices. Establishing the required tradeoff between the accuracy of pixel prediction and computational efficiency remains challenging. In this article, a lightweight multiscale segmentation framework is proposed. We leverage the representation power of different receptive fields to attain optimal accuracy while maintaining computational efficiency by embedding the sparse network architecture with the depthwise separable convolution at the multiscale level. Experimental results from two challenging remote sensing segmentation datasets show that the proposed network can achieve substantial pixel prediction accuracy at relatively low computational overhead compared to state-of-the-art networks.

Convolution Optimized Density Clustering Algorithm for Image Segmentation

In order to improve the usability of DBSCAN in the field of high-resolution image segmentation, according to the characteristics of digital images, an efficient DBSCAN based on three-dimensional convolution is proposed, and on this basis, a color image segmentation method is implemented. First, convolution is introduced into DBSCAN, three-dimensional convolution is used to obtain the core object, avoiding distance calculation and region retrieval, and reducing the time complexity of this step from the second order to a constant level. Then, the three-dimensional convolution is optimized based on dynamic programming, so that the convolution time is independent of the size of the convolution kernel. Finally, the neighborhood search range of the sample points is simplified according to the ordered core object set. Compared with the existing algorithms, the experimental results on the Berkeley BSD300 show that the algorithm is effective, the CH index and the time efficiency increase by 17% and 49% on average. The results of experiments using the remote sensing image set Mts-WH show that the computational efficiency of the algorithm is significantly improved. Compared with the DBSCAN optimized by the <italic>k</italic>D tree, the efficiency is increased by an average of 19 times, and the improvement effect is more obvious as the number of sample points increases. When the number of points exceeds 500 000, the efficiency is increased by 169 times.

A robust and high-precision edge segmentation and refinement method for high-resolution images.

Limited by GPU memory, high-resolution image segmentation is a highly challenging task. To improve the accuracy of high-resolution segmentation, High-Resolution Refine Net (HRRNet) is proposed. The network is divided into a rough segmentation module and a refinement module. The former improves DeepLabV3+ by adding the shallow features of 1/2 original image size and the corresponding skip connection to obtain better rough segmentation results, the output of which is used as the input of the latter. In the refinement module, first, the global context information of the input image is obtained by a global process. Second, the high-resolution image is divided into patches, and each patch is processed separately to obtain local details in a local process. In both processes, multiple refine units (RU) are cascaded for refinement processing, and two cascaded residual convolutional units (RCU) are added to the different output paths of RU to improve the mIoU and the convergence speed of the network. Finally, according to the context information of the global process, the refined patches are pieced to obtain the refined segmentation result of the whole high-resolution image. In addition, the regional non-maximum suppression is introduced to improve the Sobel edge detection, and the Pascal VOC 2012 dataset is enhanced, which improves the segmentation accuracy and robust performance of the network. Compared with the state-of-the-art semantic segmentation models, the experimental results show that our model achieves the best performance in high-resolution image segmentation.

BASNet: Burned Area Segmentation Network for Real-Time Detection of Damage Maps in Remote Sensing Images

Since remote sensing images of post-fire vegetation are characterized by high resolution, multiple interferences, and high similarities between the background and the target area, it is difficult for existing methods to detect and segment the burned area in these images with sufficient speed and accuracy. In this paper, we apply Salient Object Detection (SOD) to burned area segmentation, the first time this has been done, and propose an efficient burned area segmentation network (BASNet) to improve the performance of unmanned aerial vehicle (UAV) high-resolution image segmentation. BASNet comprises positioning module and refinement module. The positioning module efficiently extracts high-level semantic features and general contextual information via global average pooling layer and convolutional block to determine the coarse location of the salient region. The refinement module adopts the convolutional block attention module to effectively discriminate the spatial location of objects. In addition, to effectively combine edge information with spatial location information in the lower layer of the network and the high-level semantic information in the deeper layer, we design the residual fusion module to perform feature fusion by level to obtain the prediction results of the network. Extensive experiments on two UAV datasets collected from Chongli in China and Andong in South Korea, demonstrate that our proposed BASNet significantly outperforms state-of-the-art SOD methods quantitatively and qualitatively. BASNet also achieves a promising prediction speed for processing high-resolution UAV images, thus providing wide-ranging applicability in post-disaster monitoring and management.

Real-Time Semantic Segmentation With Fast Attention

In deep CNN based models for semantic segmentation, high accuracy relies on rich spatial context (large receptive fields) and fine spatial details (high resolution), both of which incur high computational costs. In this letter, we propose a novel architecture that addresses both challenges and achieves state-of-the-art performance for semantic segmentation of high-resolution images and videos in real-time. The proposed architecture relies on our fast spatial attention, which is a simple yet efficient modification of the popular self-attention mechanism and captures the same rich spatial context at a small fraction of the computational cost, by changing the order of operations. Moreover, to efficiently process high-resolution input, we apply an additional spatial reduction to intermediate feature stages of the network with minimal loss in accuracy thanks to the use of the fast attention module to fuse features. We validate our method with a series of experiments, and show that results on multiple datasets demonstrate superior performance with better accuracy and speed compared to existing approaches for real-time semantic segmentation. On Cityscapes, our network achieves 74.4% mIoU at 72 FPS and 75.5% mIoU at 58 FPS on a single Titan X GPU, which is ~50% faster than the state-of-the-art while retaining the same accuracy.

Patch Proposal Network for Fast Semantic Segmentation of High-Resolution Images

Despite recent progress on the segmentation of high-resolution images, there exist an unsolved problem, i.e., the trade-off among the segmentation accuracy, memory resources and inference speed. So far, GLNet is introduced for high or ultra-resolution image segmentation, which has reduced the computational memory of the segmentation network. However, it ignores the importances of different cropped patches, and treats tiled patches equally for fusion with the whole image, resulting in high computational cost. To solve this problem, we introduce a patch proposal network (PPN) in this paper, which adaptively distinguishes the critical patches from the trivial ones to fuse with the whole image for refining segmentation. PPN is a classification network which alleviates network training burden and improves segmentation accuracy. We further embed PPN in a global-local segmentation network, instructing global branch and refinement branch to work collaboratively. We implement our method on four image datasets:DeepGlobe, ISIC, CRAG and Cityscapes, the first two are ultra-resolution image datasets and the last two are high-resolution image datasets. The experimental results show that our method achieves almost the best segmentation performance compared with the state-of-the-art segmentation methods and the inference speed is 12.9 fps on DeepGlobe and 10 fps on ISIC. Moreover, we embed PPN with the general semantic segmentation network and the experimental results on Cityscapes which contains more object classes demonstrate the generalization ability on general semantic segmentation.

Laplacian Coordinates: Theory and Methods for Seeded Image Segmentation

Seeded segmentation methods have gained a lot of attention due to their good performance in fragmenting complex images, easy usability and synergism with graph-based representations. These methods usually rely on sophisticated computational tools whose performance strongly depends on how good the training data reflect a sought image pattern. Moreover, poor adherence to the image contours, lack of unique solution, and high computational cost are other common issues present in most seeded segmentation methods. In this work we introduce Laplacian Coordinates, a quadratic energy minimization framework that tackles the issues above in an effective and mathematically sound manner. The proposed formulation builds upon graph Laplacian operators, quadratic energy functions, and fast minimization schemes to produce highly accurate segmentations. Moreover, the presented energy functions are not prone to local minima, i.e., the solution is guaranteed to be globally optimal, a trait not present in most image segmentation methods. Another key property is that the minimization procedure leads to a constrained sparse linear system of equations, enabling the segmentation of high-resolution images at interactive rates. The effectiveness of Laplacian Coordinates is attested by a comprehensive set of comparisons involving nine state-of-the-art methods and several benchmarks extensively used in the image segmentation literature.

Computer Medical Image Segmentation Based on Neural Network

Image segmentation in medical imaging has long been a problem in radiological image processing. Most of the image segmentation methods in traditional vision algorithms are difficult to achieve high-resolution image segmentation due to the complexity of the algorithm. This article proposes an image segmentation method based on an optimized cellular neural network. This method introduces a non-linear template and data quantization on the basis of a basic network model, which greatly reduces the computational complexity while maintaining the accuracy of image segmentation. We then applied the method to a computer-aided system to classify tumor lesions in mammograms. Finally, we propose an FPGA-based multilevel optimization architecture for energy-efficient cellular neural networks. The optimization scheme includes three levels: system level, module level, and design space. This solution improves computing performance by increasing system parallelism, using data reuse technology to fully utilize loading bandwidth, and using data quantization to reduce computational redundancy. It also introduces pipeline and dual cache structures to optimize memory access, and analyzes the limited resources through the Roofline model. System for best performance. The experimental results show that the FPGA accelerator in this article can improve unit performance by 34% compared with other existing research work. The nonlinear quantified cellular neural network proposed in this article can reduce LUT resource consumption by 74% and energy of 48.2%. Compared with the original network, in the two projection position segmentation results of the mammogram, only 1.5% and 0.6% of the accuracy loss, respectively.

OBJECT BASED “DAYAS” CLASSIFICATION USING SENTINEL A-2 SATELLITE IMAGERY CASE STUDY CITY OF BENSLIMANE

Abstract. The management of “DAYAS” is a major issue in the preservation and maintain of biodiversity and environmental balance, especially in a context where this fragile ecosystems face many degradation factors. The extraction of Dayas is a key component in the management process of this type of wetlands, and has been the subject of many researches related to remote sensing. The methods and instrumentation for optical remote sensing are used to improve the mapping of Dayas, based on the radiometric characteristics of local hydrosystems. The present paper studies the inputs of different methods for the delimitation and extraction of Dayas in the realm of Benslimane city, using Sentinel A-2 imagery for the mapping. The methodology for the application of the pixel-based and the object-oriented approaches requires many steps, starting from an image pre-processing with Sentinel-2 calibration, the calculation of NDWI index, to proceed to the extraction of Dayas from the very high resolution image segmentation, then the application of the object-oriented classification to validate the results. The cartographic results demonstrate the input of the applied methodology in the Dayas extraction in different situations and timing (winter/summer), and allow to measure the cartographic accuracy for each approach, finding 65% of accuracy for the pixel-based approach with Kappa index = 0.40 versus 75% of accuracy for the object-oriented approach with Kappa index = 0.72. The results achieved inform and orient about optimisation measures and regulations of the Kappa index to improve the Dayas extraction and mapping.

Efficient segmentation method using quantised and non‐linear CeNN for breast tumour classification

A new segmentation method for mammography imaging system is proposed. Segmentation of masses is always a difficult problem in radiological image interpretation. Conventional methods such as region growing suffer from their computational complexity and hence can hardly be used for segmentation of high-resolution images. In order to achieve efficiency in both computational complexity and accuracy, a novel digital cellular neural network (CeNN) based approach is presented for segmentation. The approach is featured with quantisation to significantly reduce the computational complexity and non-linear template for robustness. After segmentation, a multilayer perceptron classifier is used for feature extraction and classification. Compared with other prior works, the proposed work is able to reduce resource overhead up to 63% and energy consumption up to 41% on FPGA while maintaining only up to 1.5 and 0.6% accuracy deviations for mediolateral-oblique and cranial-caudal views, respectively.

Semantic segmentation of high-resolution images

Image semantic segmentation is a research topic that has emerged recently. Although existing approaches have achieved satisfactory accuracy, they are limited to handling low-resolution images owing to their large memory consumption. In this paper, we present a semantic segmentation method for high-resolution images. First, we downsample the input image to a lower resolution and then obtain a low-resolution semantic segmentation image using state-of-the-art methods. Next, we use joint bilateral upsampling to upsample the low-resolution solution and obtain a high-resolution semantic segmentation image. To modify joint bilateral upsampling to handle discrete semantic segmentation data, we propose using voting instead of interpolation in filtering computation. Compared to state-of-the-art methods, our method significantly reduces memory cost without reducing result quality.

Learning Dual Multi-Scale Manifold Ranking for Semantic Segmentation of High-Resolution Images

Semantic image segmentation has recently witnessed considerable progress by training deep convolutional neural networks (CNNs). The core issue of this technique is the limited capacity of CNNs to depict visual objects. Existing approaches tend to utilize approximate inference in a discrete domain or additional aides and do not have a global optimum guarantee. We propose the use of the multi-label manifold ranking (MR) method in solving the linear objective energy function in a continuous domain to delineate visual objects and solve these problems. We present a novel embedded single stream optimization method based on the MR model to avoid approximations without sacrificing expressive power. In addition, we propose a novel network, which we refer to as dual multi-scale manifold ranking (DMSMR) network, that combines the dilated, multi-scale strategies with the single stream MR optimization method in the deep learning architecture to further improve the performance. Experiments on high resolution images, including close-range and remote sensing datasets, demonstrate that the proposed approach can achieve competitive accuracy without additional aides in an end-to-end manner.

A novel region-merging approach guided by priority for high resolution image segmentation

ABSTRACTIn order to improve the segmentation accuracy for high resolution remote sensing image, merging priority is modeled and used to guide merging process. Within-segment spectral variance and inter-segment edge strength are exploited for model construction. In the proposed approach, the segment with the lowest model value is prioritized for processing. Local-mutual-best-fitting rule is then used to find the appropriate neighbor of the segment under processing. Two scenes of GF-2 multi-spectral images were adopted for algorithm validation. The results indicated that the proposed method outperformed some competitive approaches.

GeoEye Image Fusion Vegetation Information Extraction Based on Blue Noise Measurement Texture

Vegetation high-resolution image segmentation is an important kind of target, while how to fuse image by using spatial and spectral information of GeoEye image, and make effective determination of the scale of vegetation texture segmentation is an important topic. This paper presents an algorithm of GeoEye image fusion vegetation information extraction based on blue noise measurement texture, which the vegetation information spectral response is calculated based on fast Fourier transform, in order to obtain the texture gray scale distribution of vegetation information, thus realizing accurate extraction of vegetation information. The simulation result got by the algorithm shows the deviation between NDVI index and original value index is below 0.05, which is obviously better than those obtained with the two comparison fusion algorithms of PCA transform (smaller than 0.4) and Brovey (smaller than 0.25), and provides the best parameter combination form [120,0.34,0.48]. Hence, it is an effective algorithm for high accuracy GeoEye image vegetation information acquisition.

Object-Based Change Detection in Urban Areas: The Effects of Segmentation Strategy, Scale, and Feature Space on Unsupervised Methods

Object-based change detection (OBCD) has recently been receiving increasing attention as a result of rapid improvements in the resolution of remote sensing data. However, some OBCD issues relating to the segmentation of high-resolution images remain to be explored. For example, segmentation units derived using different segmentation strategies, segmentation scales, feature space, and change detection methods have rarely been assessed. In this study, we have tested four common unsupervised change detection methods using different segmentation strategies and a series of segmentation scale parameters on two WorldView-2 images of urban areas. We have also evaluated the effect of adding extra textural and Normalized Difference Vegetation Index (NDVI) information instead of using only spectral information. Our results indicated that change detection methods performed better at a medium scale than at a fine scale where close to the pixel size. Multivariate Alteration Detection (MAD) always outperformed the other methods tested, at the same confidence level. The overall accuracy appeared to benefit from using a two-date segmentation strategy rather than single-date segmentation. Adding textural and NDVI information appeared to reduce detection accuracy, but the magnitude of this reduction was not consistent across the different unsupervised methods and segmentation strategies. We conclude that a two-date segmentation strategy is useful for change detection in high-resolution imagery, but that the optimization of thresholds is critical for unsupervised change detection methods. Advanced methods need be explored that can take advantage of additional textural or other parameters.