Boundary Adherence Research Articles

OEC-RNN: Object-Oriented Delineation of Rooftops With Edges and Corners Using the Recurrent Neural Network From the Aerial Images

It is an important task to automatically and accurately map rooftops from very high resolution remote sensing images since buildings are very closely related to human activity. Two typical technologies are often utilized to accomplish the task, i.e., semantic segmentation and instance segmentation. The semantic segmentation is to independently allocate a label (e.g., “building” or not) to each pixel, resulting in blob-like segments. On the contrary, one might model the boundary of a rooftop as a polygon to improve the shape of the rooftop by encouraging vertices of polygon to adhere to the rooftop’s boundary. Following this line of work, we present a multitask learning approach to predict rooftop corners in a sequent way using the attention learned from where the boundaries are in a given image region. The approach simulates the process of manual delineation of rooftops’ outline in a given image, which can produce accurate boundaries of rooftops with sharp corners and straight lines between them. Specifically, the proposed method consists of three components, i.e., object detection, pixel-by-pixel classification of both edges and corners, and delineation of rooftops in a sequent manner using a convolutional recurrent neural network (RNN). It is called as object-oriented, edges and corners (OEC)-RNN in this article. Three image datasets of buildings are employed to validate the performance of the OEC-RNN, which are compared with state-of-the-art methods for instance segmentation. The experimental results show that the OEC-RNN achieves the best performance in terms of overlay, boundary adherence, and vertex location between ground-truth and predicted polygons.

Adaptive tuning of SLIC parameter K

The well-known simple linear iterative clustering (SLIC) is the most effective among the existing algorithms for superpixel segmentation, which requires manual tuning of the number of superpixels K. The optimal value of the parameter K of the SLIC algorithm for a given image is yet an open issue. In this work, we present granulometry and quality metrics based methods for adaptive tuning of the parameter K. The proposed granulometric method exploits the weighted average of the image pattern spectrum for the adaptive tuning of the parameter K. In the quality metrics method, we use majority voting scheme based on information, texture and ground truth independent quality metrics. The experimental results demonstrate that the K SLIC superpixels from the proposed methods achieved good boundary adherence of the ground truth for the images with high value of the compactness.

SLIC Superpixel Segmentation for Polarimetric SAR Images

Superpixel segmentation approaches for polarimetric synthetic aperture radar (SAR) images have only been studied in recent years. Simple linear iterative clustering (SLIC) is a simple and efficient superpixel segmentation method, first proposed for optical images. It basically includes three implementation steps, i.e., initialization, local <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> -means clustering, and postprocessing. The challenge of applying SLIC to polarimetric SAR images lies in constructing the effective spatial and feature similarity and proposing the efficient segmentation procedure. In this study, to address both issues, we modify the SLIC clustering function to adapt the characteristics of polarimetric statistical measures. A new initialization method is proposed, which exploits the image gradient information to produce robust cluster centers. Furthermore, in an effort to give a comprehensive comparison and provide a fair assessment of the feature similarities for polarimetric SAR imagery, four classic statistical distances, among which two were not studied along with the SLIC previously, are embedded in the modified clustering function. The proposed method is validated by comparing with state-of-the-art SLIC-based algorithms and also the Ncut and TurboPixel algorithms. Experiments on extensive polarimetric SAR data sets show that the proposed method can significantly improve the segmentation results, with producing better boundary adherence and compact as well as uniform superpixels. We also obtain distinct conclusions that are different from the existing studies when investigating the performances of the statistical measures.

Convex and Compact Superpixels by Edge- Constrained Centroidal Power Diagram.

Superpixel segmentation, as a central image processing task, has many applications in computer vision and computer graphics. Boundary alignment and shape compactness are leading indicators to evaluate a superpixel segmentation algorithm. Furthermore, convexity can make superpixels reflect more geometric structures in images and provide a more concise over-segmentation result. In this paper, we consider generating convex and compact superpixels while satisfying the constraints of adhering to the boundary as far as possible. We formulate the new superpixel segmentation into an edge-constrained centroidal power diagram (ECCPD) optimization problem. In the implementation, we optimize the superpixel configurations by repeatedly performing two alternative operations, which include site location updating and weight updating through a weight function defined by image features. Compared with existing superpixel methods, our method can partition an image into fully convex and compact superpixels with better boundary adherence. Extensive experimental results show that our approach outperforms existing superpixel segmentation methods in boundary alignment and compactness for generating convex superpixels.

Superpixels With Content-Adaptive Criteria.

Superpixels are widely used in computer vision applications. Most of the existing superpixel methods use established criteria to indiscriminately process all pixels, resulting in superpixel boundary adherence and regularity being unnecessarily inter-inhibitive. This study builds upon a previous work by proposing a new segmentation strategy that classifies image content into meaningful areas containing object boundaries and meaningless parts that include color-homogeneous and texture-rich regions. Based on this classification, we design two distinct criteria to process the pixels in different environments to achieve highly accurate superpixels in content-meaningful areas and keep the regularity of the superpixels in content-meaningless regions. Additionally, we add a group of weights when adopting the color feature, successfully reducing the undersegmentation error. The superior accuracy and the moderate compactness achieved by the proposed method in comparative experiments with several state-of-the-art methods indicate that the content-adaptive criteria efficiently reduce the compromise between boundary adherence and compactness.

Scattering Feature-Driven Superpixel Segmentation for Polarimetric SAR Images

Superpixel segmentation for polarimetric synthetic aperture radar (PolSAR) images plays a fundamental role in various PolSAR applications. Subject to the intrinsic limitations, the existing methods generally produce over or undersegmented superpixels in highly complex scenes of PolSAR images. In this article, we propose an efficient and effective scattering feature-driven superpixel segmentation method for PolSAR images, which is capable of preserving the global structure information and producing superpixels with both high boundary adherence and visual compactness. First, a hierarchical version of the refined five-component decomposition is proposed. The derived scattering features along with other well-used features are then combined to construct a low-dimensional feature vector. Second, a modified normalized cuts formulation using a distance measurement is presented, in which the feature similarity and the space proximity are both considered. During the superpixel segmentation, the edge information derived from our scattering mechanism-optimal contrast-based edge detector is incorporated. On this basis, a tradeoff factor according to the edge information and the equivalent number of looks is determined, which balances the relationship of feature similarity and space proximity so as to adaptively control the size and shape of superpixels. The performance of the proposed method is demonstrated and evaluated with fully PolSAR data over different test sites. The outputs show that the proposed method outperforms the state-of-the-art methods and the produced superpixels scale well especially for heterogeneous scattering cases.

Image clustering algorithm using superpixel segmentation and non‐symmetric Gaussian–Cauchy mixture model

In this study, an unsupervised clustering algorithm is proposed to label superpixel density images. Firstly, the authors propose a novel superpixel segmentation algorithm driven by a modified fuzzy C-means objective function, Kullback–Leibler (KL) divergence, and an entropy term, which generate superpixels with good boundary adherence and intensity homogeneity. In this model, the logarithm of Gaussian distribution as a new distance metric is used to improve the accuracy of boundary pixel classification, the KL divergence is applied to regularise the fuzzy objective function. Based on this model, the generated superpixel intensity images with a highly distinctive background colour from the colour of the target are obtained. Grouping cues generated by superpixels can affect the performance of image clustering greatly. Next, according to the small amount of clustering data generated by the superpixel intensity images, they construct a non-symmetric mixture model based on a mixture of Gaussian distribution and Cauchy distribution for implementing image clustering. Thus, clustering of colour images is transformed into clustering of these newly generated data. The advantage of this model is its well adaption to different shapes of observed data. Experimental results on publicly available data sets are provided to demonstrate the effectiveness of the proposed algorithm.

AWkS: adaptive, weighted k-means-based superpixels for improved saliency detection

Clustering inspired superpixel algorithms perform a restricted partitioning of an image, where each visually coherent region containing perceptually similar pixels serves as a primitive in subsequent processing stages. Simple linear iterative clustering (SLIC) has emerged as a standard superpixel generation tool due to its exceptional performance in terms of segmentation accuracy and speed. However, SLIC applies a manually adjusted distance measure for dis-similarity computation which directly affects the quality of superpixels. In this work, self-adjustable distance measures are adapted from the weighted k-means clustering (W-k-means) for generating superpixel segmentation. In the proposed distance measures, an adaptive weight associated with each variable reflects its relevance in the clustering process. Intuitively, the variable weights correspond to the normalization terms in SLIC that affect the trade-off between superpixels boundary adherence and compactness. Weights that influence consistency in superpixel generation are automatically updated. The variable weights update is accomplished during optimization with a closed-form solution based on the current image partition. The proposed adaptive, W-k-means-based superpixels (AWkS) experimented on three benchmarks under different distance measure outperform the conventional SLIC algorithm with respect to various boundary adherence metrics. Finally, the effectiveness of the AWkS over SLIC is demonstrated for saliency detection.

An Integrated Object and Machine Learning Approach for Tree Canopy Extraction from UAV Datasets

Unmanned aerial vehicles (UAV) have emerged as new platforms for acquiring ultra-high resolution images, which are challenging for extraction of features using conventional image processing approaches. Tree canopies are required to be constantly monitored for better planning and management. UAV is currently one way to survey canopies over a large area for precisely estimating their geometry. Conventional segmentation techniques are extensively used for image feature extraction. However, they lack in accuracy and require high computational processing when used for ultra-high resolution UAV datasets. These issues can be handled by superpixel segmentation algorithms which have good boundary adherence and are computationally efficient. Simple linear iterative clustering (SLIC) is a subset of superpixel segmentation technique which uses minimum tuning parameters making it most efficient. As the random forest is known for handling multiple attributes and robustness, it can be used for classifying and extracting features from segmented image generated using SLIC. The present study mainly focuses on the automation for extraction of tree canopies along with their object-based attributes from the UAV dataset. The data acquisition was carried out using Trimble UX5 fixed-wing UAV which was further orthorectified at a spatial resolution of 13 cm. The ortho-image was further segmented using SLIC algorithm. Canopy segments are then identified and classified using random forest, which is then merged into trees objects on the basis of their proximity. Accuracy assessment was then carried out for extracted tree canopies and was found that the aforesaid approach could achieve 93% similarity index. The current study highlights the potential of using SLIC segmentation and random forest classification method for tree canopy extraction from the ultra-high resolution ortho-image derived from UAV platforms.

Superpixel-Based Shallow Convolutional Neural Network (SSCNN) for Scanned Topographic Map Segmentation

Motivated by applications in topographic map information extraction, our goal was to discover a practical method for scanned topographic map (STM) segmentation. We present an advanced guided watershed transform (AGWT) to generate superpixels on STM. AGWT utilizes the information from both linear and area elements to modify detected boundary maps and sequentially achieve superpixels based on the watershed transform. With achieving an average of 0.06 on under-segmentation error, 0.96 on boundary recall, and 0.95 on boundary precision, it has been proven to have strong ability in boundary adherence, with fewer over-segmentation issues. Based on AGWT, a benchmark for STM segmentation based on superpixels and a shallow convolutional neural network (SCNN), termed SSCNN, is proposed. There are several notable ideas behind the proposed approach. Superpixels are employed to overcome the false color and color aliasing problems that exist in STMs. The unification method of random selection facilitates sufficient training data with little manual labeling while keeping the potential color information of each geographic element. Moreover, with the small number of parameters, SCNN can accurately and efficiently classify those unified pixel sequences. The experiments show that SSCNN achieves an overall F1 score of 0.73 on our STM testing dataset. They also show the quality of the segmentation results and the short run time of this approach, which makes it applicable to full-size maps.

Variational Fuzzy Superpixel Segmentation

This article presents a novel variational model based on fuzzy clustering and total variation regularization for superpixel segmentation. Compared with the classical hard-labeled methodologies, our approach gives soft results via the fuzzy membership function, and moreover, the use of total variation provides additional information that can enhance the superpixel regularity, which in turn improves the segmentation performance. To efficiently minimize the energy functional of the proposed model, we adopt an alternating direction method of multipliers with the modified Chambolle’s fast duality projection algorithm. Our algorithm can generate regular and compact superpixels with high segmentation accuracy, satisfactory boundary adherence, and low computational cost. Comparative experimental results with the current state-of-the-art approaches reveal the superior performance of the proposed method.

GRID: GRID Resample by Information Distribution

This paper exploits a concise yet efficient initialization strategy to optimize grid sampling-based superpixel segmentation algorithms. Rather than straight distributing all initial seeds evenly, it adopts a context-aware approach to modify their positions and total number via a coarse-to-fine manner. Firstly, half the expected number of seeds are regularly sampled on the image grid, thereby creating a rough distribution of color information for all rectangular cells. A series of fission is then performed on cells that contain excessive color information recursively. In each cell, the local color uniformity is balanced by a dichotomy on one original seed, which generates two new seeds and settles them to spatially symmetrical sub-regions. Therefore, the local concentration of seeds is adaptive to the complexity of regional information. In addition, by calculating the amount of color via a summed area table (SAT), the informative regions can be located at a very low time cost. As a result, superpixels are produced from ideal original seeds with an exact number and exhibit better boundary adherence. Experiments demonstrate that the proposed strategy effectively promotes the performance of simple linear iterative clustering (SLIC) and its variants in terms of several quality measures.

SUPERPIXEL SEGMENTATION FOR POLSAR IMAGES BASED ON HEXAGON INITIALIZATION AND EDGE REFINEMENT

Abstract. Superpixel segmentation for PolSAR images can heavily decrease the number of primitives for subsequent interpretation while reducing the impact of speckle noise. However, traditional superpixel segmentation methods for PolSAR images only focus on the boundary adherence, the significance of superpixel segmentation will be lost when the accuracy is improved at the expense of computation efficiency. To solve this problem, this paper proposes a novel superpixel segmentation algorithm for PolSAR images based on hexagon initialization and edge refinement. First, the PolSAR image is initialized as hexagonal distribution, where the complexity of searching pixels for relabelling in the local regions can be reduced by 30% theoretically. Second, all pixels in the PolSAR image are initialized as unstable pixels based on the hexagonal superpixels, which can boost the segmentation performance in the heterogeneous regions and effectively maintain all the potential edge pixels. Third, the revised Wishart distance and the spatial distance are integrated as a distance measure to relabel all unstable pixels. Finally, the postprocessing procedure based on a dissimilarity measure is applied to generate the final superpixels. Extensive experiments conducted on both the simulated and real-world PolSAR images demonstrate the superiority and effectiveness of our proposed algorithm in terms of computation efficiency and segmentation accuracy, compared to three other state-of-the-art methods.

Image segmentation using dense and sparse hierarchies of superpixels

We investigate the intersection between hierarchical and superpixel image segmentation. Two strategies are considered: (i) the classical region merging, that creates a dense hierarchy with a higher number of levels, and (ii) the recursive execution of some superpixel algorithm, which generates a sparse hierarchy with fewer levels. We show that, while dense methods can capture more intermediate or higher-level object information, sparse methods are considerably faster and usually with higher boundary adherence at finer levels. We first formalize the two strategies and present a sparse method, which is faster than its superpixel algorithm and with similar boundary adherence. We then propose a new dense method to be used as post-processing from the intermediate level, as obtained by our sparse method, upwards. This combination results in a unique strategy and the most effective hierarchical segmentation method among the compared state-of-the-art approaches, with efficiency comparable to the fastest superpixel algorithms.

An edge attention‐based geodesic distance for PolSAR image superpixel segmentation

The simple linear iterative clustering (SLIC) is the most popular superpixel segmentation method for its simplicity and effectiveness. In this Letter, we propose an edge attention-based geodesic distance, which can be applied to the SLIC framework for polarimetric synthetic aperture radar image superpixel segmentation. Image edges are especially considered to generate superpixels with better boundary adherence property, while the computing burden does not increase. Specifically, the authors first detect image edges, then fuse the edge map and polarimetric dissimilarity, and finally define the geodesic distance based on the fused results. Experiments performed on real scene images demonstrate the superiority of the proposed distance in both qualitative and quantitative ways.

SVLA: A compact supervoxel segmentation method based on local allocation

With the development of three-dimensional (3D) point cloud acquisition technologies, supervoxels have become increasingly important, as they provide compact and uniform representations. In this study, a novel supervoxel segmentation method, supervoxels based on local allocation (SVLA), is proposed. SVLA is composed of three steps, namely extreme point determination, local allocation (LA), and connectivity insurance. LA defines a novel cost function for preserving instance boundaries and enforces local minimization. To test the performance of SVLA, the non-compactness error (NCE) is newly defined to evaluate the compactness, and three commonly used evaluation metrics are employed. Both indoor and outdoor datasets are utilized to perform the experiments. Based on the visual and quantitative analysis of the segmentation results, SVLA demonstrates fulfillment of boundary adherence, compact constraints, and low computational complexity. Compared to state-of-the-art algorithms, SVLA yields superior results, especially with regard to indoor point clouds.

Watershed-Based Superpixels With Global and Local Boundary Marching

Superpixels are widely used in computer vision applications, as they conserve the running costs of subsequent processing while preserving the original performance. In most of the existing algorithms, the boundary adherence and the compactness of superpixels are necessarily inter-inhibitive because the color/gradient information is balanced against the position constraints, and the set criteria define all pixels indiscriminately. In this paper, we present a two-phase superpixel segmentation method based on the watershed transformation. After designing a new approach for calculating the flooding priority, we propose a new strategy with two distinct criteria for global and local refinement of the boundary pixels. These criteria reduce the compromise between the boundary adherence and compactness. Unlike the indiscriminate standards, our method applies different treatments to pixels in different environments, preserving the color homogeneity in content-rich areas while improving the regularity of the superpixels in content-plain regions. The superior accuracy and computing time of our proposed method are verified in comparison experiments with several state-of-the-art methods.

A Superpixel Boundary Optimization (SBO) Framework Based on Information Measure Function

Superpixel is an essential tool for computer vision. In practice, classic superpixel algorithms do not exhibit good boundary adherence with fewer superpixels, which will greatly hamper further analysis. To remedy the defect, a superpixel boundary optimization framework is proposed in this paper. There are three steps in the framework. Firstly, based on the proposed information measure function, the under-segmented superpixels generated by classic superpixel algorithms are screened out. Secondly, with the two invariant centroids method, these under-segmented superpixels are re-segmented to improve the accuracy in boundary adherence. Finally, smaller superpixels are merged to maintain the same number with initial superpixels. Quantitative evaluations on the BSDS500 exhibit that the performance of the classic superpixel algorithms is improved by employing the framework, especially on the condition of fewer superpixels.

Dynamic Random Walk for Superpixel Segmentation.

In this paper, we propose a novel random walk model, called Dynamic Random Walk (DRW), which adds a new type of dynamic node to the original RW model and reduces redundant calculation by limiting the walk range. To solve the seed-lacking problem of the proposed DRW, we redefine the energy function of the original RW and use the first arrival probability among each node pair to avoid the interference for each partition. Relaxation of our DRW is performed with the help of a greedy strategy and the Weighted Random Walk Entropy(WRWE) that uses the gradient feature to approximate the stationary distribution. The proposed DRW not only can enhance the boundary adherence but also can run with linear time complexity. To extend our DRW for superpixel segmentation, a seed initialization strategy is proposed. It can evenly distribute seeds in both 2D and 3D space and generate superpixels in only one iteration. The experimental results demonstrate that our DRW is faster than existing RW models and better than the state-of-the-art superpixel segmentation algorithms with respect to both efficiency and segmentation effects.

Fast marching based superpixels

AbstractIn this article, we present a fast-marching based algorithm for generating superpixel (FMS) partitions of images. The idea behind the algorithm is to draw an analogy between waves propagating in a heterogeneous medium and regions growing on an image at a rate depending on the local color and texture. The FMS algorithm is evaluated on the Berkeley Segmentation Dataset 500. It yields results in terms of boundary adherence that are slightly better than the ones obtained with similar approaches including the Simple Linear Iterative Clustering, the Eikonal-based region growing for efficient clustering and the Iterative Spanning Forest framework for superpixel segmentation algorithms. An interesting feature of the proposed algorithm is that it can take into account texture information to compute the superpixel partition. We illustrate the interest of adding texture information on a specific set of images obtained by recombining textures patches extracted from images representing stripes, originally constructed by Giraudet al.[20]. On this dataset, our approach works significantly better than color based superpixel algorithms.