Multi-scale Structure Tensor Research Articles

Multiscale Structure Tensor for Improved Feature Extraction and Image Regularization.

Regularization methods are used widely in image selective smoothing and edge preserving restoration of noisy images. Traditional methods utilize image gradients within regularization function for controlling the smoothing and can produce artifacts when noise levels are higher. In this paper, we consider a robust image adaptive exponent driven regularization for filtering noisy images with salient feature preservation. Our spatially adaptive variable exponent function depends on a continuous switch based on the eigenvalues of structure tensor which identifies noisy edges, and corners with higher accuracy. Structure tensor eigenvalues encode various image features and we consider a spatially varying continuous map which provides multiscale edge maps of natural images. By embedding the structure tensor-based exponent in a well-defined regularization model, we obtain denoising filters which are capable of obtaining good feature preserving image restoration. The GPU-based implementation computes the edge map in real time at 45-60 frames/s depending on the GPU card. Multiscale structure tensor-based spatially adaptive variable exponent provides reliable edge maps and compared with standard edge detectors it is robust under various noisy conditions. Moreover, filtering based on the multiscale variable exponent map method outperforms L0 sparse gradient-based image smoothing and related filters.

Color-texture cosegmentation based on nonlinear compact multi-scale structure tensor and TV-flow

Image segmentation is a fundamental problem in computer vision, and the color and texture information are usually both employed to obtain more satisfactory segmentation results. However, the traditional color-texture segmentation methods usually assume that the color-based and texture-based segmentation results are globally consistent, which is not always the case. Sometimes, the color-texture based segmentation results may be worse than some single feature (color or texture) based ones if the consistency constraints are taken into account inappropriately. To address the problem, a graph cuts based color-texture cosegmentation method is proposed in this paper, where just the similarity constrains are considered rather than the global consistencies, and a penalty term is included to adaptively balance the possible local inconsistencies. Additionally, in order to extract the texture features effectively, a comprehensive texture descriptor is designed by integrating the nonlinear compact multi-scale structure tensor (NCMSST) and total variation flow (TV-flow). A large number of segmentation comparison experiments using the synthesis color-texture images and real natural scene images verify the superiorities of our proposed texture descriptor and color-texture cosegmentation method.

Unsupervised multiphase color–texture image segmentation based on variational formulation and multilayer graph

This paper proposes an unsupervised variational segmentation approach of color–texture images. To improve the description ability, the compact multi-scale structure tensor, total variation flow, and color information are integrated to extract color–texture information. Since heterogeneous image object and nonlinear variation exist in color–texture image, it is not appropriate to use one single/multiple constant in the Chan and Vese (CV) model to describe each phase [1,2]. Therefore, a multiphase successive active contour model (MSACM) based on the multivariable Gaussian distribution is presented to describe each phase. As geodesic active contour (GAC) has a stronger ability in capturing boundary. To inherit the advantages of edge-based model and region-based model, we incorporate the GAC into the MSACM to enhance the detection ability for concave edge. Although multiphase optimization of our proposed MSACM is a NP hard problem, we can discretely and approximately solve it by a multilayer graph method. In addition, to segment the color–texture image automatically, an adaptive iteration convergence criterion is designed by incorporating the local Kullback–Leibler distance and global phase label, so that we can control the segmentation process converges. Comparing to state-of-the-art unsupervised segmentation methods on a substantial of color texture images, our approach achieves a significantly better performance on capture ability of homogeneous region/smooth boundary and accuracy.

Multilayer graph cuts based unsupervised color–texture image segmentation using multivariate mixed student's t-distribution and regional credibility merging

This paper proposes an unsupervised color–texture image segmentation method. In order to enhance the effects of segmentation, a new color–texture descriptor is designed by integrating the compact multi-scale structure tensor (MSST), total variation (TV) flow, and the color information. Due to the fact that MSST does not work well for separating regions with large-scale texture, the total variation flow is used to auxiliarily describe the texture feature by extracting local scale information. To segment the color–texture image in an unsupervised and multi-label way, the multivariate mixed student's t-distribution (MMST) is chosen for probability distribution modeling, as MMST can describe the distribution of color–texture features accurately. Since the valid class number is hard to adaptively determine in advance, a component-wise expectation–maximization for MMST (CEM3ST) algorithm is proposed, which can effectively initialize the valid class number. Then, we can build up the energy functional according to the valid class number, and optimize it by multilayer graph cuts method. However, the problem of over/error-segmentation often happens. To overcome this problem, a strategy of regional credibility merging (RCM) is presented by integrating the regional adjacency relationship, region size, common edge between regions, and regional color–texture dissimilarity. In order to terminate the whole segmentation process, an adaptive iteration convergence criterion is designed, which combines the negative logarithm of probability of all color–texture features with the Kullback–Leibler (KL) divergence for MMST. Experiments using a large number of synthesis color–texture images and real natural scene images demonstrate the superiority of our proposed method, such as the effective over/error-segmentation reduction, high segmentation accuracy, and outperforming visual entirety/consistency.