Supervoxel Segmentation Method Research Articles

Fast supervoxel segmentation of connectivity median simulation based on Manhattan distance

Supervoxels provide a more natural and compact 3D point cloud representation, which is a fundamental problem in point cloud processing and has attracted extensive attention from many scholars. At present, although the supervoxel segmentation has achieved significant results, the processing efficiency still restricts its further processing in the face of large-scale point cloud processing. Therefore, this paper proposes a new supervoxel segmentation method for point clouds to achieve fast supervoxel over-segmentation of large-scale point clouds while maintaining good boundaries and accuracy. Firstly, the supervoxel segmentation is formulated as a subset selection problem. With the idea of greedy strategy, the interval quick sorting and region growing methods is designed to realize the rapid merging of subsets in the iterative process. Secondly, the Manhattan distance metric and the seed point median simulation method are proposed to enhance the precision of iterative optimization and improve the efficiency of point cloud supervoxel segmentation. Experimental results show that, compared with state-of-the-art supervoxel segmentation method, the proposed method achieves the best results under UE, GCE, BR and other indicators, while reducing the running time of the algorithm by 10.6% to 28.3%.

A Method for Tomato Plant Stem and Leaf Segmentation and Phenotypic Extraction Based on Skeleton Extraction and Supervoxel Clustering

To address the current problem of the difficulty of extracting the phenotypic parameters of tomato plants in a non-destructive and accurate way, we proposed a method of stem and leaf segmentation and phenotypic extraction of tomato plants based on skeleton extraction and supervoxel clustering. To carry out growth and cultivation experiments on tomato plants in a solar greenhouse, we obtained multi-view image sequences of the tomato plants to construct three-dimensional models of the plant. We used Laplace’s skeleton extraction algorithm to extract the skeleton of the point cloud after removing the noise points using a multi-filtering algorithm, and, based on the plant skeleton, searched for the highest point path, height constraints, and radius constraints to separate the stem from the leaf. At the same time, a supervoxel segmentation method based on Euclidean distance was used to segment each leaf. We extracted a total of six phenotypic parameters of the plant: height, stem diameter, leaf angle, leaf length, leaf width and leaf area, using the segmented organs, which are important for the phenotype. The results showed that the average accuracy, average recall and average F1 scores of the stem and leaf segmentation were 0.88, 0.80 and 0.84, and the segmentation indexes were better than the other four segmentation algorithms; the coefficients of determination between the measurement values of the phenotypic parameters and the real values were 0.97, 0.84, 0.88, 0.94, 0.92 and 0.93; and the root-mean-square errors were 2.17 cm, 0.346 cm, 5.65°, 3.18 cm, 2.99 cm and 8.79 cm2. The measurement values of the proposed method had a strong correlation with the actual values, which could satisfy the requirements of daily production and provide technical support for the extraction of high-throughput phenotypic parameters of tomato plants in solar greenhouses.

A Supervoxel Segmentation Method With Adaptive Centroid Initialization for Point Clouds

Supervoxels find applications as a pre-processing step in many image processing problems due to their ability to present a regional representation of points by correlating them into a set of clusters. Besides reducing the overall computational time for subsequent algorithms, the desirable properties in supervoxels are adherence to object boundaries and compactness. Existing supervoxel segmentation methods define the size of a supervoxel based on a user inputted resolution value. A fixed resolution results in poor performance in point clouds with non-uniform density. Whereas, other methods, in their quest for better boundary adherence, produce supervoxels with irregular shapes and elongated boundaries. In this article, we propose a new supervoxel segmentation method, based on k-means algorithm, with dynamic cluster seed initialization to ensure uniform distribution of cluster seeds in point clouds with variable densities. We also propose a new cluster seed initialization strategy, based on histogram binning of surface normals, for better boundary adherence. Our algorithm is parameter-free and gives equal importance to the color, spatial location and orientation of the points resulting in compact supervoxels with tight boundaries. We test the efficacy of our algorithm on a publicly available point cloud dataset consisting of 1449 pairs of indoor RGB-D images, i.e., color (RGB) images coupled with depth information (D) mapped per pixel. Results are compared against three state-of-the-art algorithms based on four quality metrics. Results show that our method provides significant improvement over other methods in the undersegmentation error and compactness metrics and, performs equally well in the boundary recall and contour density metrics.

Identification of Building Damage from UAV-Based Photogrammetric Point Clouds Using Supervoxel Segmentation and Latent Dirichlet Allocation Model.

Accurate assessment of building damage is very important for disaster response and rescue. Traditional damage detection techniques using 2D features at a single observing angle cannot objectively and accurately reflect the structural damage conditions. With the development of unmanned aerial vehicle photogrammetric techniques and 3D point processing, automatic and accurate damage detection for building roof and facade has become a research hotspot in recent work. In this paper, we propose a building damage detection framework based on the boundary refined supervoxel segmentation and random forest–latent Dirichlet allocation classification. First, the traditional supervoxel segmentation method is improved to segment the point clouds into good boundary refined supervoxels. Then, non-building points such as ground and vegetation are removed from the generated supervoxels. Next, latent Dirichlet allocation (LDA) model is used to construct the high-level feature representation for each building supervoxel based on the selected 2D image and 3D point features. Finally, LDA model and random forest algorithm are employed to identify the damaged building regions. This method is applied to oblique photogrammetric point clouds collected from the Beichuan Country Earthquake Site. The research achieves the 3D damage assessment for building facade and roof. The result demonstrates that the proposed framework is capable of achieving around 94% accuracy for building point extraction and around 90% accuracy for damage identification. Moreover, both of the precision and recall for building damage detection reached around 89%. Concluded from comparison analysis, the proposed method improved the damage detection accuracy and the highest improvement ratio is over 8%.

A BOUNDARY-ENHANCED SUPERVOXEL METHOD FOR EXTRACTION OF ROAD EDGES IN MLS POINT CLOUDS

Abstract. Road extraction plays a significant role in production of high definition maps (HD maps). This paper presents a novel boundary-enhanced supervoxel segmentation method for extracting road edge contours from MLS point clouds. The proposed method first leverages normal feature judgment to obtain 3D point clouds global geometric information, then clusters points according to an existing method with global geometric information to enhance the boundaries. Finally, it utilizes the neighbor spatial distance metric to extract the contours and drop out existing outliers. The proposed method is tested on two datasets acquired by a RIEGL VMX-450 MLS system that contain the major point cloud scenes with different types of road boundaries. The experimental results demonstrate that the proposed method provides a promising solution for extracting contours efficiently and completely. Results show that the precision values are 1.5 times higher and approximately equal than the other two existing methods when the recall value is 0 for both tested two road datasets.

SVLA: A compact supervoxel segmentation method based on local allocation

With the development of three-dimensional (3D) point cloud acquisition technologies, supervoxels have become increasingly important, as they provide compact and uniform representations. In this study, a novel supervoxel segmentation method, supervoxels based on local allocation (SVLA), is proposed. SVLA is composed of three steps, namely extreme point determination, local allocation (LA), and connectivity insurance. LA defines a novel cost function for preserving instance boundaries and enforces local minimization. To test the performance of SVLA, the non-compactness error (NCE) is newly defined to evaluate the compactness, and three commonly used evaluation metrics are employed. Both indoor and outdoor datasets are utilized to perform the experiments. Based on the visual and quantitative analysis of the segmentation results, SVLA demonstrates fulfillment of boundary adherence, compact constraints, and low computational complexity. Compared to state-of-the-art algorithms, SVLA yields superior results, especially with regard to indoor point clouds.

Leaf and wood separation of poplar seedlings combining locally convex connected patches and K-means++ clustering from terrestrial laser scanning data

Achieving parameter measurements and three-dimensional visualization for leaves and wood that have been accurately separated from a whole plant is of great significance. We develop a leaf and wood separation method that combines locally convex connected patches (LCCPs) and K-means++ clustering with poplar seedlings as the research object. First, point cloud data of the research samples are collected using terrestrial laser scanning (TLS). Second, an individual poplar seedling point cloud is separated from the complex background using the pass-through filtering algorithm, and noise and outliers are filtered using the statistical filtering algorithm to improve the separation accuracy. Subsequently, the supervoxel segmentation method is introduced to oversegment the poplar seedlings into patches, and the LCCP clustering algorithm is utilized to segment the leaves and branches. The modified K-means++ clustering algorithm is used to further separate connected leaves and branches. To validate the separation method, numerous experiments on normal and water-deficient poplar seedlings are carried out, and the results illustrate that the segmentation accuracy of the leaves are 93.66% and 90.77%, respectively. The results indicate that the proposed method can effectively separate the leaves and wood of poplar seedlings from TLS data.

Learning Long-term Structural Dependencies for Video Salient Object Detection.

Existing video salient object detection (VSOD) methods focus on exploring either short-term or long-term temporal information. However, temporal information is exploited in a global frame-level or regular grid structure, neglecting interframe structural dependencies. In this paper, we propose to learn long-term structural dependencies with a structure-evolving graph convolutional network (GCN). Particularly, we construct a graph for the entire video using a fast supervoxel segmentation method, in which each node is connected according to spatio-temporal structural similarity. We infer the inter-frame structural dependencies of salient object using convolutional operations on the graph. To prune redundant connections in the graph and better adapt to the moving salient object, we present an adaptive graph pooling to evolve the structure of the graph by dynamically merging similar nodes, learning better hierarchical representations of the graph. Experiments on six public datasets show that our method outperforms all other state-of-the-art methods. Furthermore, We also demonstrate that our proposed adaptive graph pooling can effectively improve the supervoxel algorithm in the term of segmentation accuracy.

Supervoxel-based segmentation of 3D imagery with optical flow integration for spatiotemporal processing

The past 20 years has seen a progressive evolution of computer vision algorithms for unsupervised 2D image segmentation. While earlier efforts relied on Markov random fields and efficient optimization (graph cuts, etc.), the next wave of methods beginning in the early part of this century were, in the main, stovepiped. Of these 2D segmentation efforts, one of the most popular and, indeed, one that comes close to being a state of the art method is the ultrametric contour map (UCM). The pipelined methodology consists of (i) computing local, oriented responses, (ii) graph creation, (iii) eigenvector computation (globalization), (iv) integration of local and global information, (v) contour extraction, and (vi) superpixel hierarchy construction. UCM performs well on a range of 2D tasks. Consequently, it is somewhat surprising that no 3D version of UCM exists at the present time. To address that lack, we present a novel 3D supervoxel segmentation method, dubbed 3D UCM, which closely follows its 2D counterpart while adding 3D relevant features. The methodology, driven by supervoxel extraction, combines local and global gradient-based features together to first produce a low-level supervoxel graph. Subsequently, an agglomerative approach is used to group supervoxel structures into a segmentation hierarchy with explicitly imposed containment of lower-level supervoxels in higher-level supervoxels. Comparisons are conducted against state of the art 3D segmentation algorithms. The considered applications are 3D spatial and 2D spatiotemporal segmentation scenarios. For the latter comparisons, we present results of 3D UCM with and without optical flow video pre-processing. As expected, when motion correction beyond a certain range is required, we demonstrate that 3D UCM in conjunction with optical flow is a very useful addition to the pantheon of video segmentation methods.

Toward better boundary preserved supervoxel segmentation for 3D point clouds

Supervoxels provide a more natural and compact representation of three dimensional point clouds, and enable the operations to be performed on regions rather than on the scattered points. Many state-of-the-art supervoxel segmentation methods adopt fixed resolution for each supervoxel, and rely on the initialization of seed points. As a result, they may not preserve well the boundaries of the point cloud with a non-uniform density. In this paper, we present a simple but effective supervoxel segmentation method for point clouds, which formalizes supervoxel segmentation as a subset selection problem. We develop an heuristic algorithm that utilizes local information to efficiently solve the subset selection problem. The proposed method can produce supervoxels with adaptive resolutions, and dose not rely the selection of seed points. The method is fully tested on three publicly available point cloud segmentation benchmarks, which cover the major point cloud types. The experimental results show that compared with the state-of-the-art supervoxel segmentation methods, the supervoxels extracted using our method preserve the object boundaries and small structures more effectively, which is reflected in a higher boundary recall and lower under-segmentation error.

Supervoxel Segmentation and Bias Correction of MR Image with Intensity Inhomogeneity

Supervoxel segmentation has become an essential tool to medical image analysis for three-dimension MR image. However, in no consideration of the intensity inhomogeneity in 2D/3D MR image, the state-of-the-art supervoxel segmentation methods do not satisfy the further analysis, such as tissue classification according to intensity feature. In order to overcome the above-mentioned issues, we propose a modified supervoxel segmentation method for three-dimension MR image, which integrates the bias field into the weighted distance metric to determine the nearest cluster center. The supervoxel segmentation and bias correction can be simultaneously completed in our method. Especially, the bias corrected image lays the foundation for the supervoxel classification in accordance with the intensity feature. The experimental results and quantitative evaluation showed that the supervoxels obtained by our method are adherence to the MR tissue boundaries, and the bias corrected image is positive for the intensity feature extraction.

Supervoxel-based statistical analysis of diffusion tensor imaging in schizotypal personality disorder

To study white matter changes in schizotypal personality disorder (SPD), we developed a new statistical analysis method based on supervoxels for diffusion tensor imaging. Twenty patients with SPD and eighteen healthy controls were recruited from a pool of 3000 first-year university undergraduates, and underwent MRI using a 3T scanner. Diffusion tensors were first normalized into ICBM-152 space followed by a supervoxel segmentation based on graph clustering to segment white matter tensors into diffusion homogeneous supervoxels. Fractional anisotropy (FA) values in supervoxels were compared between SPD and healthy controls using permutation test. Suprathreshold cluster size test was used to correct multiple comparison. At last, fibers with significant differences were extracted from supervoxel clusters with significance level P < 0.05. Results showed that FA values in genu of corpus callosum were significantly reduced (P = 0.012) in patients with SPD (FA = 0.565) compared with healthy controls (FA = 0.593). In summary, this study proposed a novel supervoxel segmentation method for diffusion tensor imaging using graph-based clustering, and extended permutation test and suprathreshold cluster size test to supervoxels for detection of white matter changes.