Nuclear Norm Constraint Research Articles

Cube is a good form: Hyperspectral band selection via multi-dimensional and high-order structure preserved clustering

As an effective strategy for reducing the noisy and redundant information for hyperspectral imagery (HSI), hyperspectral band selection intends to select a subset of original hyperspectral bands, which boosts the subsequent different tasks. In this paper, we introduce a multi-dimensional high-order structure preserved clustering method for hyperspectral band selection, referred to as MHSPC briefly. By regarding original hyperspectral images as a tensor cube, we apply the tensor CP (CANDECOMP/PARAFAC) decomposition on it to exploit the multi-dimensional structural information as well as generate a low-dimensional latent feature representation. In order to capture the local geometrical structure along the spectral dimension, a graph regularizer is imposed on the new feature representation in the lower dimensional space. In addition, since the low rankness of HSIs is an important global property, we utilize a nuclear norm constraint on the latent feature representation matrix to capture the global data structure information. Different to most of previous clustering based hyperspectral band selection methods which vectorize each band as a vector without considering the 2-D spatial information, the proposed MHSPC can effectively capture the spatial structure as well as the spectral correlation of original hyperspectral cube in both local and global perspectives. An efficient alternatively updating algorithm with theoretical convergence guarantee is designed to solve the resultant optimization problem, and extensive experimental results on four benchmark datasets validate the effectiveness of the proposed MHSPC over other state-of-the-arts.

NOODLE: Joint Cross-View Discrepancy Discovery and High-Order Correlation Detection for Multi-View Subspace Clustering

Benefiting from the effective exploration of the valuable topological pair-wise relationship of data points across multiple views, multi-view subspace clustering (MVSC) has received increasing attention in recent years. However, we observe that existing MVSC approaches still suffer from two limitations that need to be further improved to enhance the clustering effectiveness. Firstly, previous MVSC approaches mainly prioritize extracting multi-view consistency, often neglecting the cross-view discrepancy that may arise from noise, outliers, and view-inherent properties. Secondly, existing techniques are constrained by their reliance on pair-wise sample correlation and pair-wise view correlation, failing to capture the high-order correlations that are enclosed within multiple views. To address these issues, we propose a novel MVSC framework via joi N t cr O ss-view discrepancy disc O very an D high-order corre L ation d E tection ( NOODLE ), seeking an informative target subspace representation compatible across multiple features to facilitate the downstream clustering task. Specifically, we first exploit the self-representation mechanism to learn multiple view-specific affinity matrices, which are further decomposed into cohesive factors and incongruous factors to fit the multi-view consistency and discrepancy, respectively. Additionally, an explicit cross-view sparse regularization is applied to incoherent parts, ensuring the consistency and discrepancy to be precisely separated from the initial subspace representations. Meanwhile, the multiple cohesive parts are stacked into a three-dimensional tensor associated with a tensor-Singular Value Decomposition (t-SVD) based weighted tensor nuclear norm constraint, enabling effective detection of the high-order correlations implicit in multi-view data. Our proposed method outperforms state-of-the-art methods for multi-view clustering on six benchmark datasets, demonstrating its effectiveness.

Multi-View Enhanced Tensor Nuclear Norm and Local Constraint Model for Cancer Clustering and Feature Gene Selection.

The analysis of cancer data from multi-omics can effectively promote cancer research. The main focus of this article is to cluster cancer samples and identify feature genes to reveal the correlation between cancers and genes, with the primary approach being the analysis of multi-view cancer omics data. Our proposed solution, the Multi-View Enhanced Tensor Nuclear Norm and Local Constraint (MVET-LC) model, aims to utilize the consistency and complementarity of omics data to support biological research. The model is designed to maximize the utilization of multi-view data and incorporates a nuclear norm and local constraint to achieve this goal. The first step involves introducing the concept of enhanced partial sum of tensor nuclear norm, which significantly enhances the flexibility of the tensor nuclear norm. After that, we incorporate total variation regularization into the MVET-LC model to further augment its performance. It enables MVET-LC to make use of the relationship between tensor data structures and sparse data while paying attention to the feature details of the tensor data. To tackle the iterative optimization problem of MVET-LC, the alternating direction method of multipliers is utilized. Through experimental validation, it is demonstrated that our proposed model outperforms other comparison models.

Multilinear multitask learning by transformed tensor singular value decomposition

In this paper, we study the problem of multilinear multitask learning (MLMTL), in which all tasks are stacked into a third-order tensor for consideration. In contrast to conventional multitask learning, MLMTL can explore inherent correlations among multiple tasks in a better manner by utilizing multilinear low rank structure. Existing approaches about MLMTL are mainly based on the sum of singular values for approximating low rank matrices obtained by matricizing the third-order tensor. However, these methods are suboptimal in the Tucker rank approximation. In order to elucidate intrinsic correlations among multiple tasks, we present a new approach by the use of transformed tensor nuclear norm (TTNN) constraint in the objective function. The main advantage of the proposed approach is that it can acquire a low transformed multi-rank structure in a transformed tensor by applying suitable unitary transformations which is helpful to determine principal components in grouping multiple tasks for describing their intrinsic correlations more precisely. Furthermore, we establish an excess risk bound of the minimizer of the proposed TTNN approach. Experimental results including synthetic problems and real-world images, show that the mean-square errors of the proposed method is lower than those of the existing methods for different number of tasks and training samples in MLMTL.

Auto-weighted multiple kernel tensor clustering

Multiple kernel subspace clustering (MKSC) has attracted intensive attention since its powerful capability of exploring consensus information by generating a high-quality affinity graph from multiple base kernels. However, the existing MKSC methods still exist the following limitations: (1) they essentially neglect the high-order correlations hidden in different base kernels; and (2) they perform candidate affinity graph learning and consensus affinity graph learning in two separate steps, where suboptimal solution may be obtained. To alleviate these problems, a novel MKSC method, namely auto-weighted multiple kernel tensor clustering (AMKTC), is proposed. Specifically, AMKTC first integrates the consensus affinity graph learning and candidate affinity graph learning into a unified framework, where the optimal goal can be achieved by making these two learning processes negotiate with each other. Further, an auto-weighted fusion scheme with one-step manner is proposed to learn the final consensus affinity graph, where the reasonable weights will be automatically learned for each candidate graph. Finally, the essential high-order correlations between multiple base kernels can be captured by leveraging tensor-singular value decomposition (t-SVD)-based tensor nuclear norm constraint on a 3-order graph tensor. Experiments on seven benchmark datasets with eleven comparison methods demonstrate that our method achieves state-of-the-art clustering performance.

Collaborative structure and feature learning for multi-view clustering

Multi-view clustering divides similar objects into the same class through using the fused multiview information. Most multi-view clustering methods obtain clustering result by only analyzing structure relationship among samples, ignoring the analysis of intrinsic features of each sample, while a few methods operate the original feature on the corresponding high-dimensional kernel matrices. However, noisy and redundant features of samples are inevitably mixed in original multi-view data or high-dimensional kernel matrices. To address this problem, we propose a novel multiview clustering method, which unifies structure learning and feature learning to a framework. Specifically, we obtain a consensus structure information from multiple views via sparse subspace structure learning with weight tensor nuclear norm constraint. Then our feature learning seeks projection directions to obtain data representation by data pseudo labels, which are obtained via the fused consensus structure information. Furthermore, we use the manifold regularization term to establish the relationship between data structure information and learnt data presentation. At last, the two subtasks are alternately iterated and optimized to acquire accurate structure and discriminative data presentation. Experimental results on different datasets validate the proposed method is superior to the state-of-the-art methods.

Tensor Robust Principal Component Analysis From Multilevel Quantized Observations

We consider Quantized Tensor Robust Principal Component Analysis (Q-TRPCA), which aims to recover a low-rank tensor and a sparse tensor from noisy, quantized, and sparsely corrupted measurements. A nonconvex constrained maximum likelihood (ML) estimation method is proposed for Q-TRPCA. We provide an upper bound on the Frobenius norm of tensor estimation error under this method. Making use of tools in information theory, we derive a theoretical lower bound on the best achievable estimation error from unquantized measurements. Compared with the lower bound, the upper bound on the estimation error is nearly order-optimal. We further develop an efficient convex ML estimation scheme for Q-TRPCA based on the tensor nuclear norm (TNN) constraint. This method is more robust to sparse noises than the latter nonconvex ML estimation approach. Conducting experiments on both synthetic data and real-world data, we show the effectiveness of the proposed methods.

Image restoration via wavelet-based low-rank tensor regularization

Low-rank models have been widely applied for visual analysis. However, the conventional global low rank on a single whole image and the patch-level low rank have difficulty in perfectly preserving dependence (or correlation) and the latent structures in the image. Inspired by recent advances in low-rank tensor analysis, a wavelet-based low rank tensor regularization model (WLTR) is proposed in this work. Utilizing the fact that the coefficients of an image under wavelet transform exhibit sparsity, the low-rank regularization can be obtained by imposing the nuclear norm constraint on the tensor composed of wavelet subbands coefficients. Considering all structure information of intrascale and interscale coefficients under the tensor representation, the proposed method characterizes the local and global elements in a unified manner. To make the proposed WLTR tractable and robust, the alternative direction of multiplier method (ADMM) is adopted to efficiently and effectively solve this convex optimization problem. Extensive experiments on various types of image restoration problems, including inpainting, deblurring and denoising, validate WLTR outperforms several existing state-of-the-art approaches in terms of peak signal-to-noise ratio and visual quality.

Static and incremental robust kernel factorization embedding graph regularization supporting ill-conditioned industrial data recovery

Low-rank approximation algorithms aim to utilize convex nuclear norm constraint of linear matrices to recover ill-conditioned entries caused by multi-sampling rates, sensor drop-out. However, these existing algorithms are often limited in solving high-dimensionality and rank minimization relaxation. In this paper, a robust kernel factorization embedding graph regularization method is developed to statically impute missing measurements. Specifically, the implicit high-dimensional feature space of ill-conditioned data is factorized by kernel sparse dictionary. Then, a robust sparse-norm and graph regularization constraints are performed in the objective function to ensure the consistency of the spatial information. For the optimization of the parameters involved in the model, a distributed adaptive proximal Newton gradient descent learning strategy is proposed to accelerate the convergence. Furthermore, considering the dynamic time-series and potentially non-stationary structure of industrial data, we propose extended incremental versions to alleviate the complexity of the overall model computation. Extensive data recovery experiments are conducted on two real industrial processes to evaluate the proposed method in comparison with existing state-of-the-art restorers. The results show that the proposed methods can impute better with different missing rates and have strong competitiveness in practical application.

Low-rank tensor approximation with local structure for multi-view intrinsic subspace clustering

Among various multi-view clustering approaches, tensor-based multi-view subspace clustering methods aim to explore the high-order correlations across varying views and have achieved encouraging effects. Nevertheless, there are still some demerits in them: (1) View-specific information hinders the mining of global consensus. (2) The local structure inside individual view lacks consideration. (3) Clustering results are not utilized to reversely guide the low-rank tensor optimization. In order to tackle these drawbacks, we propose a unified model termed as Low-rank Tensor Approximation with Local Structure for Multi-view Intrinsic Subspace Clustering. Specifically, the proposed model learns multiple intrinsic subspace representations via the rank preserving decomposition, which is to mitigate the impact of view-specific information on enhancing the global consistency. Then, these intrinsic subspace representations are assembled into a 3-order target tensor with tensor nuclear norm constraint. To preserve the consistent locality, we adopt the manifold regularization to constrain each view when mapping into the intrinsic subspace. Furthermore, since the learned label indicator matrix implicitly characterizes the cluster structure, which is used to guide the optimization of the low-rank tensor representation. Finally, abundant experiments on six real-word datasets demonstrate that the proposed method is superior over other state-of-the-art clustering methods.

Hyper-Laplacian regularized multi-view subspace clustering with jointing representation learning and weighted tensor nuclear norm constraint

In recent years, tensor-Singular Value Decomposition (t-SVD) based tensor nuclear norm has achieved remarkable progress in multi-view subspace clustering. However, most existing clustering methods still have the following shortcomings: (a) It has no meaning in practical applications for singular values to be treated equally. (b) They often ignore that data samples in the real world usually exist in multiple nonlinear subspaces. In order to solve the above shortcomings, we propose a hyper-Laplacian regularized multi-view subspace clustering model that joints representation learning and weighted tensor nuclear norm constraint, namely JWHMSC. Specifically, in the JWHMSC model, firstly, in order to capture the global structure between different views, the subspace representation matrices of all views are stacked into a low-rank constrained tensor. Secondly, hyper-Laplace graph regularization is adopted to preserve the local geometric structure embedded in the high-dimensional ambient space. Thirdly, considering the prior information of singular values, the weighted tensor nuclear norm (WTNN) based on t-SVD is introduced to treat singular values differently, which makes the JWHMSC more accurately obtain the sample distribution of classification information. Finally, representation learning, WTNN constraint and hyper-Laplacian graph regularization constraint are integrated into a framework to obtain the overall optimal solution of the algorithm. Compared with the state-of-the-art method, the experimental results on eight benchmark datasets show the good performance of the proposed method JWHMSC in multi-view clustering.

Hyperspectral Remote Sensing Image Feature Representation Method Based on CAE-H with Nuclear Norm Constraint

Due to the high dimensionality and high data redundancy of hyperspectral remote sensing images, it is difficult to maintain the nonlinear structural relationship in the dimensionality reduction representation of hyperspectral data. In this paper, a feature representation method based on high order contractive auto-encoder with nuclear norm constraint (CAE-HNC) is proposed. By introducing Jacobian matrix in the CAE of the nuclear norm constraint, the nuclear norm has better sparsity than the Frobenius norm and can better describe the local low dimension of the data manifold. At the same time, a second-order penalty term is added, which is the Frobenius norm of the Hessian matrix expressed in the hidden layer of the input, encouraging a smoother low-dimensional manifold geometry of the data. The experiment of hyperspectral remote sensing image shows that CAE-HNC proposed in this paper is a compact and robust feature representation method, which provides effective help for the ground object classification and target recognition of hyperspectral remote sensing image.

Consensus Graph Learning for Multi-View Clustering

Multi-view clustering, which exploits the multi-view information to partition data into their clusters, has attracted intense attention. However, most existing methods directly learn a similarity graph from original multi-view features, which inevitably contain noises and redundancy information. The learned similarity graph is inaccurate and is insufficient to depict the underlying cluster structure of multi-view data. To address this issue, we propose a novel multi-view clustering method that is able to construct an essential similarity graph in a spectral embedding space instead of the original feature space. Concretely, we first obtain multiple spectral embedding matrices from the view-specific similarity graphs, and reorganize the gram matrices constructed by the inner product of the normalized spectral embedding matrices into a tensor. Then, we impose a weighted tensor nuclear norm constraint on the tensor to capture high-order consistent information among multiple views. Furthermore, we unify the spectral embedding and low rank tensor learning into a unified optimization framework to determine the spectral embedding matrices and tensor representation jointly. Finally, we obtain the consensus similarity graph from the gram matrices via an adaptive neighbor manner. An efficient optimization algorithm is designed to solve the resultant optimization problem. Extensive experiments on six benchmark datasets are conducted to verify the efficacy of the proposed method. The code is implemented by using MATLAB R2018a and MindSpore library <xref ref-type="bibr" rid="ref1">[1]</xref>: <uri>https://github.com/guanyuezhen/CGL</uri>.

Error-robust low-rank tensor approximation for multi-view clustering

Multi-view clustering has been widely developed to improve the clustering performance over that of single-view clustering. However, the types of errors vary and behave inconsistently in each view, which results in performance degradation in real applications. To address this limitation, in this paper, we propose a novel Markov chain-based spectral clustering method for multi-view clustering to handle different types of errors. Unlike most of the existing self-representation-based subspace clustering methods, which process each view separately, ignoring complementary information among views, our method first computes the transition probability matrices of all views, then forms each transition probability matrix as a frontal slice of a third-order tensor to capture the multi-view information, and finally decomposes the tensor into an ideal tensor with the tensor nuclear norm constraint and an error term. Furthermore, the proposed method imposes the group l1 and l2,1 norms on the error matrix for error learning such that various errors can be clearly characterized and processed to improve performance. To solve the challenging optimization model, we propose an efficient algorithm using the augmented Lagrangian multiplier method. Experimental results on three real-world datasets show that the proposed method is superior to the state-of-the art methods in various evaluation metrics.

Hyperspectral-Multispectral Image Fusion via Tensor Ring and Subspace Decompositions

Fusion from a spatially low resolution hyperspectral image (LR-HSI) and a spectrally low resolution multispectral image (MSI) to produce a high spatial-spectral HSI (HR-HSI), known as hyperspectral super resolution, has risen to a preferred topic for reinforcing the spatial-spectral resolution of HSI in recent years. In this work, we propose a new model, namely, low-rank tensor ring decomposition based on tensor nuclear norm (LRTRTNN), for HSI-MSI fusion. Specifically, for each spectrally subspace cube, similar patches are grouped to exploit both the global low-rank property of LR-HSI and the nonlocal similarity of HR-MSI. Afterward, a joint optimization of all groups via the presented LRTRTNN approximation is implemented in a unified cost function. With the introduced tensor nuclear norm (TNN) constraint, all 3D tensor ring factors are no longer unfolded to suit the matrix nuclear norm used in conventional methods, and the internal tensor structure can be naturally retained. The alternating direction method of multipliers is introduced for coefficients update. Numerical and visual experiments on real data show that our LRTRTNN method outperforms most state-of-the-art algorithms in terms of fusing performance.

Multi-trace post-stack seismic data sparse inversion with nuclear norm constraint

Among many seismic inversion methods, the sparse spike inversion for post-stack seismic data uses the migrated and stacked seismic data which is regarded as zero offset reflection seismic data in the case of normal incidence to extract reflectivity and impedance of underground rocks. The seismic reflectivity and impedance can reflect underground rocks’ lithology, petrophysical property, oil–gas possibility, and so forth. However, the common used post-stack seismic inversion adopts single trace in the process of inversion and completes the whole data cube’s inversion through trace by trace. It cannot use lateral regularization. Hence, the lateral continuity of single trace inversion result is poor. It is difficult to represent the lateral variation features of underground rocks. Based on the conventional sparse spike inversion, the nuclear norm of matrix in the matrix completion theory is introduced in the process of post-stack seismic inversion. At the same time, the strategy of multi-trace seismic data simultaneous inversion is used to carry out lateral regularization constraint. Numerical tests on 2D model indicate that the inversion results obtained from the proposed method can clearly represent not only the vertical variation features but also the lateral variation features of underground rocks. At last, the inversion results of real seismic data further show the feasibility and superiority of the proposed method in practical application.

Manifold Transfer Learning via Discriminant Regression Analysis

In transfer learning, how to effectively transfer useful information from the source domain to the target domain is crucial. In this paper, we propose a novel transfer learning method for image classification, named manifold transfer learning via discriminant regression analysis (MTL-DRA), to transfer the local geometry structure information from the source domain to the target domain and ensure that the transform matrix is robust or sparse so that samples from different domains can be well combined. In MTL-DRA, we encode discriminant information of the source domain to the target domain by introducing between- and within-class graphs to preserve within-class similarity and reduce between-class similarity. With different norms as constraints, MTL-DRA overcomes the disturbance of noise and avoids negative transfer learning. To improve the robustness of MTL-DRA, we encode a nuclear norm constraint and propose robust MTL-DRA (RMTL-DRA). We analyzed the convergence and complexity of the two proposed methods. To verify the performance of the proposed methods, we conducted extensive experiments on five public image benchmarks. The experimental results show that the proposed methods outperform state-of-the-art transfer learning methods.

Cross-resolution face recognition with pose variations via multilayer locality-constrained structural orthogonal procrustes regression

In real video surveillance scenes, the extracted face regions generally have low-resolution (LR) and are sensitive to pose and illumination variations; these flaws undoubtedly degrade the subsequent recognition task. To overcome these challenges, we propose an approach named multilayer locality-constrained structural orthogonal Procrustes regression (MLCSOPR). The proposed MLCSOPR not only learns the pose-robust discriminative representation features to reduce the resolution gap between the LR image space and the high-resolution (HR) one but also strengthens the consistency between the LR and HR image space. In particular, several contributions are made in this paper: (i) Inspired by the orthogonal Procrustes problem (OPP), a matrix approximation is exploited to find an optimal correction between two data matrices. (ii) The nuclear norm constraint is applied to the reconstruction error to maintain the structural property. (iii) Based on the abovementioned learned resolution-robust representation features, a linear regression-based classification strategy is adopted to recognize the LR input face images. Experiments on commonly used face databases have shown the effectiveness of the proposed method on cross-resolution face matching with pose variations.

Towards a Combination of Low Rank and Sparsity in EIT Imaging

Electrical impedance tomography (EIT) calculates the internal conductivity distribution of a body using electrical contact measurement and has become increasingly attractive in the biomedical field. However, the design of optimal tomography image reconstruction algorithms has not achieved an adequate level of progress and maturity. The spatial-temporal properties are crucial for the improvement of reconstruction quality and efficiency in dynamic EIT reconstruction. However, these properties have not been fully utilized in previous research. In this paper, a mathematical model for EIT reconstruction is built upon a combination of the low-rank and the sparsity theories. In addition to the low-rank method based on the nuclear norm constraint, the patch-based sparse method is also used to obtain the spatial features of a reconstructed image, according to the characteristic of an irregular boundary for the EIT image. The mathematical model of the new method is solved using the variable split (VS) algorithm. The imaging results are compared with the reconstruction results of the traditional algorithms. The experimental results demonstrate better performance of the new method compared with the traditional methods. The effectiveness of the proposed scheme is verified.

Face Hallucination with Weighted Nuclear Norm Constraint

Face hallucination via patch-pairs leaning based methods has been wildly used in the past several years. Some position-patch based face hallucination methods have been proposed to improve the representation power of image patch and obtain the optimal regressive weighted vector. The rationale behind the position-patch based face hallucination is the fact that human face is always highly structured and consequently positioned and it plays an increasingly important role in the reconstruction. However, in the existing position-patch based methods, the probe image patch is usually represented as a linear combination of the corresponding patches of some training images, and the reconstruction residual is usually measured using the vector norm such as 1-norm and 2-norm. Since the vector norms neglect two-dimensional structures inside the residual, the final reconstruction performance is not very satisfactory. To cope with this problem, we present a weighted nuclear-norm constrained sparse coding (WNCSC) model for position-patch based face hallucination. In addition, an efficient algorithm for the WNCSC is developed using the alternating direction method of multipliers (ADMM) and the method of augmented Lagrange multipliers (ALM). The advantages of the proposed model are twofold: in order to fully make use of low-rank structure information of the reconstruction residual, the weighted nuclear norm is applied to measure the residual matrix, which is able to alleviate the bias between input patches and training data, and it is more robust than the Euclidean distance (2-norm); the more flexible selection method for rank components can determine the optimal combination weights and adaptively choose the relevant and nearest hallucinated neighbors. Finally, experimental results prove that the proposed method outperforms the related state-of-the-art methods in both quantitative and visual comparisons.