Multi-layer Superpixels Research Articles

Food volume estimation by multi-layer superpixel.

Estimating the volume of food plays an important role in diet monitoring. However, it is difficult to perform this estimation automatically and accurately. A new method based on the multi-layer superpixel technique is proposed in this paper to avoid tedious human-computer interaction and improve estimation accuracy. Our method includes the following steps: 1) obtain a pair of food images along with the depth information using a stereo camera; 2) reconstruct the plate plane from the disparity map; 3) warp the input image and the disparity map to form a new direction of view parallel to the plate plane; 4) cut the warped image into a series of slices according to the depth information and estimate the occluded part of the food; and 5) rescale superpixels for each slice and estimate the food volume by accumulating all available slices in the segmented food region. Through a combination of image data and disparity map, the influences of noise and visual error in existing interactive food volume estimation methods are reduced, and the estimation accuracy is improved. Our experiments show that our method is effective, accurate and convenient, providing a new tool for promoting a balanced diet and maintaining health.

Composite Kernel of Mutual Learning on Mid-Level Features for Hyperspectral Image Classification.

By training different models and averaging their predictions, the performance of the machine-learning algorithm can be improved. The performance optimization of multiple models is supposed to generalize further data well. This requires the knowledge transfer of generalization information between models. In this article, a multiple kernel mutual learning method based on transfer learning of combined mid-level features is proposed for hyperspectral classification. Three-layer homogenous superpixels are computed on the image formed by PCA, which is used for computing mid-level features. The three mid-level features include: 1) the sparse reconstructed feature; 2) combined mean feature; and 3) uniqueness. The sparse reconstruction feature is obtained by a joint sparse representation model under the constraint of three-scale superpixels' boundaries and regions. The combined mean features are computed with average values of spectra in multilayer superpixels, and the uniqueness is obtained by the superposed manifold ranking values of multilayer superpixels. Next, three kernels of samples in different feature spaces are computed for mutual learning by minimizing the divergence. Then, a combined kernel is constructed to optimize the sample distance measurement and applied by employing SVM training to build classifiers. Experiments are performed on real hyperspectral datasets, and the corresponding results demonstrated that the proposed method can perform significantly better than several state-of-the-art competitive algorithms based on MKL and deep learning.

LESC: Superpixel cut‐based local expansion for accurate stereo matching

The rapid estimation of the accurate disparity between pixels is the goal of stereo matching. However, it is very difficult for the 3D labels-based methods due to huge search space of 3D labels, especially for high-resolution images. In this paper, a novel superpixel cut-based method is proposed, in an attempt to get the accurate disparity map efficiently, including the multi-layer superpixel optimization and iteractive local α-expansion in parallel. As for the multi-layer superpixel optimization, feature point optimization is designed to get accurate candidate labels that are set for most pixels using non-local cost aggregation strategy and update per-pixel labels of the corresponding superpixels from the candidate label sets on the small-size superpixel layer, and then update the middle to large-size superpixel layers progressively using non-local cost aggregation strategy. In order to provide more prior information to identify weak texture and textureless regions in non-local cost aggregation, the weight combination of “intensity + gradient + binary image” is proposed for constructing an optimal minimum spanning tree (MST) to calculate the aggregated matching cost and obtain the labels of minimum aggregated matching cost. Moreover, the local patch surrounding the corresponding superpixels is designed to accelerate superpixel optimization in parallel, and a neighborhood structure is presented to optimize the algorithm in this study, including superpixel neighborhood and patch neighborhood. As for the iteractive local α-expansion, three layers of patch structure corresponding to the superpixel neighborhood structure is proposed for optimizing the algorithm in this study. The experimental results show that higher accuracy could be achieved via the method in this study compared with some known state-of-the-art stereo methods on KITTI 2015 and Middlebury benchmark V3, which are the standard benchmarks for testing the stereo matching methods.

Hierarchical-model salient object detection based on manifold ranking

Salient object detection has attracted a lot of focused research and has resulted in many applications, it is a challenge to detect the most important scene from the input image. Different from most previous methods that only modeled the spatial connectivity of every region is modeled using k-regular graph, and do not consider the deficiency between multi-layer super pixel segmentations based on manifold ranking, we propose a multi-scale approach. First, we tackle an image from a scale point of view and use a multi-scale approach to analyze saliency cues. Second, through building a graph model which is on the basis of k-regular graph, we connect the nodes belonging to the same cluster and located in the same spatial connected area with edges, to highlight the whole goal more uniformly and evenly, then used manifold ranking to generate multi-layer saliency map. Finally, the final saliency map is got through weighted linear fusion. Extensive experiments on six benchmark datasets demonstrate efficiency of the proposed method against the state-of-the-art methods.

Hyperspectral Image Classification via Joint Sparse representation of Multi-layer Superpixles

In this paper, a novel spectral-spatial joint sparse representation algorithm for hyperspectral image classification is proposed based on multi-layer superpixels in various scales. Superpixels of various scales can provide complete yet redundant correlated information of the class attribute for test pixels. Therefore, we design a joint sparse model for a test pixel by sampling similar pixels from its corresponding superpixels combinations. Firstly, multi-layer superpixels are extracted on the false color image of the HSI data by principal components analysis model. Secondly, a group of discriminative sampling pixels are exploited as reconstruction matrix of test pixel which can be jointly represented by the structured dictionary and recovered sparse coefficients. Thirdly, the orthogonal matching pursuit strategy is employed for estimating sparse vector for the test pixel. In each iteration, the approximation can be computed from the dictionary and corresponding sparse vector. Finally, the class label of test pixel can be directly determined with minimum reconstruction error between the reconstruction matrix and its approximation. The advantages of this algorithm lie in the development of complete neighborhood and homogeneous pixels to share a common sparsity pattern, and it is able to achieve more flexible joint sparse coding of spectral-spatial information. Experimental results on three real hyperspectral datasets show that the proposed joint sparse model can achieve better performance than a series of excellent sparse classification methods and superpixels-based classification methods.

PMSC: PatchMatch-Based Superpixel Cut for Accurate Stereo Matching

Estimating the disparity and normal direction of one pixel simultaneously, instead of only disparity, also known as 3D label methods, can achieve much higher subpixel accuracy in the stereo matching problem. However, it is extremely difficult to assign an appropriate 3D label to each pixel from the continuous label space R3 while maintaining global consistency because of the infinite parameter space. In this paper, we propose a novel algorithm called PatchMatch-based superpixel cut to assign 3D labels of an image more accurately. In order to achieve robust and precise stereo matching between local windows, we develop a bilayer matching cost, where a bottom-up scheme is exploited to design the two layers. The bottom layer is employed to measure the similarity between small square patches locally by exploiting a pretrained convolutional neural network, and then, the top layer is developed to assemble the local matching costs in large irregular windows induced by the tangent planes of object surfaces. To optimize the spatial smoothness of local assignments, we propose a novel strategy to update 3D labels. In the procedure of optimization, both segmentation information and random refinement of PatchMatch are exploited to update candidate 3D label set for each pixel with high probability of achieving lower loss. Since pairwise energy of general candidate label sets violates the submodular property of graph cut, we propose a novel multilayer superpixel structure to group candidate label sets into candidate assignments, which thereby can be efficiently fused by α-expansion graph cut. Extensive experiments demonstrate that our method can achieve higher subpixel accuracy in different data sets, and currently ranks first on the new challenging Middlebury 3.0 benchmark among all the existing methods.

A novel combination of second-order statistical features and segmentation using multi-layer superpixels for salient object detection

Salient object detection is one of the outstanding capabilities of the human visual system (HVS). The researcher community aims at developing a salient object detection model that matches the detection accuracy as well as computation time taken by the humans. These models can be developed in either spatial domain or frequency domain. Spatial domain models provide good detection accuracy at the cost of high computational time while frequency domain models offer fast computational speed to meet real-time requirements at the cost of poor detection accuracy. In order to induce a trade-off between computational time and accuracy, we propose a model which provides high detection accuracy without taking much of computation time. To detect the salient object with an accurate shape, we first segment the given image by utilizing a bipartite graph partitioning approach which aggregates multi-layer superpixels in a principled and effective manner. Second, the saliency of each segmented region is computed based on a hypercomplex Fourier transform (HFT) saliency map reconstructed using amplitude spectrum, filtered at an appropriate scale chosen using statistical features extracted from grey- level co-occurrence matrix and original phase spectrum. Finally, a saliency map is generated by taking average of the HFT coefficients of each region in the segmented image and then using the average HFT intensity value of the entire image as a threshold to clearly separate salient object from the background. The performance of the proposed model is evaluated in terms of F---measure, area under curve (AUC), and computation time using six publicly available image datasets. Both qualitative and quantitative evaluations on six publicly available datasets demonstrate the robustness and efficiency of the proposed model against twenty popular state-of-the-art methods.

Semisupervised synthetic aperture radar image segmentation with multilayer superpixels

Image segmentation plays a significant role in synthetic aperture radar (SAR) image processing. However, SAR image segmentation is challenging due to speckle. We propose a semisupervised bipartite graph method for segmentation of an SAR image. First, the multilayer over-segmentation of the SAR image, referred to as superpixels, is computed using existing segmentation algorithms. Second, an unbalanced bipartite graph is constructed in which the correlation between pixels is replaced by the texture similarity between superpixels, to reduce the dimension of the edge matrix. To also improve efficiency, we define a new method, called the combination of the Manhattan distance and symmetric Kullback–Leibler divergence, to measure texture similarity. Third, by the Moore–Penrose inverse matrix and semisupervised learning, we construct an across-affinity matrix. A quantitative evaluation using SAR images shows that the new algorithm produces significantly high-quality segmentations as compared with state-of-the-art segmentation algorithms.