Diagonal Representation Research Articles

Ensemble clustering by block diagonal representation

Ensemble clustering by block diagonal representation

Multi-geometric block diagonal representation subspace clustering with low-rank kernel

Multi-geometric block diagonal representation subspace clustering with low-rank kernel

A restarted large-scale spectral clustering with self-guiding and block diagonal representation

Spectral clustering, a prominent unsupervised machine learning method, involves a critical task of constructing a similarity matrix. In existing approaches, this matrix is either computed once for all or updated alternatively. However, the former struggles to capture comprehensive relationships among data points, and the latter is often impractical for large-scale problems. This study introduces a new clustering framework with self-guidance and a block diagonal representation, retaining valuable information from previous cycles. To our knowledge, this is the first application of such a framework to spectral clustering, with a key distinction being the reclassification of samples in each cycle. To reduce computational overhead, we employ a block diagonal representation with Nyström approximation for constructing the similarity matrix. Theoretical results justify the rationality of approximate computations in spectral clustering. Comprehensive experiments on benchmark datasets demonstrate the superiority of our proposed algorithms over state-of-the-art methods for large-scale clustering. Notably, our framework has the potential to enhance clustering algorithms, performing well even with a randomly chosen initial guess.

Block diagonal representation learning with local invariance for face clustering

Block diagonal representation learning with local invariance for face clustering

A novel graph-based multiple kernel learning framework for hyperspectral image classification

ABSTRACT Multiple kernel learning (MKL) is an efficient way to improve hyperspectral image classification with few training samples by integrating spectral and spatial features. Nonetheless, presenting a MKL algorithm to capture significant information and eliminate redundancy is indispensable. In this paper, a novel graph-based MKL method (GMKL) is introduced to enhance the performance of the MKL algorithm and achieve high discriminative features, resulting in maximum separability. To do so, an informative low-rank graph is determined via a new algorithm called block diagonal representation that takes into account both local and global data structures. Then, in lieu of a time-consuming search for finding the optimal combination of the basic kernels, unlike the prior works, an optimal projective direction is acquired using graph theory. Various basic kernels generated from superpixels and attribute profiles are projected to produce a discriminative combination of kernels. Eventually, the optimum kernel is given to the support vector machine classifier. Experiments have been conducted on two popular hyperspectral datasets (Indian Pines and Pavia University). For the datasets, the proposed algorithm gained 90.43% and 99.28% overall accuracy, respectively, using only 1% training samples, which is much better than the state-of-the-art MKL algorithm and deep learning-based approaches with lower computational time.

Consensus Affinity Graph Learning via Structure Graph Fusion and Block Diagonal Representation for Multiview Clustering

Learning a robust affinity graph is fundamental to graph-based clustering methods. However, some existing affinity graph learning methods have encountered the following problems. First, the constructed affinity graphs cannot capture the intrinsic structure of data well. Second, when fusing all view-specific affinity graphs, most of them obtain a fusion graph by simply taking the average of multiple views, or directly learning a common graph from multiple views, without considering the discriminative property among diverse views. Third, the fusion graph does not maintain an explicit cluster structure. To alleviate these problems, the adaptive neighbor graph learning approach and the data self-expression approach are first integrated into a structure graph fusion framework to obtain a view-specific structure affinity graph to capture the local and global structures of data. Then, all the structural affinity graphs are weighted dynamically into a consensus affinity graph, which not only effectively incorporates the complementary affinity structure of important views but also has the capability of preserving the consensus affinity structure that is shared by all views. Finally, a k–block diagonal regularizer is introduced for the consensus affinity graph to encourage it to have an explicit cluster structure. An efficient optimization algorithm is developed to tackle the resultant optimization problem. Extensive experiments on benchmark datasets validate the superiority of the proposed method.

Convex Subspace Clustering by Adaptive Block Diagonal Representation.

Subspace clustering is a class of extensively studied clustering methods where the spectral-type approaches are its important subclass. Its key first step is to desire learning a representation coefficient matrix with block diagonal structure. To realize this step, many methods were successively proposed by imposing different structure priors on the coefficient matrix. These impositions can be roughly divided into two categories, i.e., indirect and direct. The former introduces the priors such as sparsity and low rankness to indirectly or implicitly learn the block diagonal structure. However, the desired block diagonalty cannot necessarily be guaranteed for noisy data. While the latter directly or explicitly imposes the block diagonal structure prior such as block diagonal representation (BDR) to ensure so-desired block diagonalty even if the data is noisy but at the expense of losing the convexity that the former's objective possesses. For compensating their respective shortcomings, in this article, we follow the direct line to propose adaptive BDR (ABDR) which explicitly pursues block diagonalty without sacrificing the convexity of the indirect one. Specifically, inspired by Convex BiClustering, ABDR coercively fuses both columns and rows of the coefficient matrix via a specially designed convex regularizer, thus naturally enjoying their merits and adaptively obtaining the number of blocks. Finally, experimental results on synthetic and real benchmarks demonstrate the superiority of ABDR to the state-of-the-arts (SOTAs).

Kernel Block Diagonal Representation Subspace Clustering with Similarity Preservation

Subspace clustering methods based on the low-rank and sparse model are effective strategies for high-dimensional data clustering. However, most existing low-rank and sparse methods with self-expression can only deal with linear structure data effectively, but they cannot handle data with complex nonlinear structure well. Although kernel subspace clustering methods can efficiently deal with nonlinear structure data, some similarity information between samples may be lost when the original data are reconstructed in the kernel space. Moreover, these kernel subspace clustering methods may not obtain an affinity matrix with an optimal block diagonal structure. In this paper, we propose a novel subspace clustering method termed kernel block diagonal representation subspace clustering with similarity preservation (KBDSP). KBDSP contains three contributions: (1) an affinity matrix with block diagonal structure is generated by introducing a block diagonal representation term; (2) a similarity-preserving regularizer is constructed and embedded into our model by minimizing the discrepancy between inner products of original data and inner products of reconstructed data in the kernel space, which better preserve the similarity information between original data; (3) the KBDSP model is proposed by integrating the block diagonal representation term and similarity-preserving regularizer into the kernel self-expressing frame. The optimization of our proposed model is solved efficiently by utilizing the alternating direction method of multipliers (ADMM). Experimental results on nine datasets demonstrate the effectiveness of the proposed method.

Numerical Simulation of a Space-Fractional Molecular Beam Epitaxy Model without Slope Selection

In this paper, we introduce a space-fractional version of the molecular beam epitaxy (MBE) model without slope selection to describe super-diffusion in the model. Compared to the classical MBE equation, the spatial discretization is an important issue in the space-fractional MBE equation because of the nonlocal nature of the fractional operator. To approximate the fractional operator, we employ the Fourier spectral method, which gives a full diagonal representation of the fractional operator and achieves spectral convergence regardless of the fractional power. And, to combine with the Fourier spectral method directly, we present a linear, energy stable, and second-order method. Then, it is possible to simulate the dynamics of the space-fractional MBE equation efficiently and accurately. By using the numerical method, we investigate the effect of the fractional power in the space-fractional MBE equation.

New Gradient Correction Scheme for Electronically Nonadiabatic Dynamics Involving Multiple Spin States

It has been recommended that the best representation to use for trajectory surface hopping (TSH) calculations is the fully adiabatic basis in which the Hamiltonian is diagonal. Simulations of intersystem crossing processes with conventional TSH methods require an explicit computation of nonadiabatic coupling vectors (NACs) in the molecular-Coulomb-Hamiltonian (MCH) basis, also called the spin-orbit-free basis, in order to compute the gradient in the fully adiabatic basis (also called the diagonal representation). This explicit requirement destroys some of the advantages of the overlap-based algorithms and curvature-driven algorithms that can be used for the most efficient TSH calculations. Therefore, although these algorithms allow one to perform NAC-free simulations for internal conversion processes, one still requires NACs for intersystem crossing. Here, we show that how the NAC requirement is circumvented by a new computation scheme called the time-derivative-matrix scheme.

Robust Subspace Clustering with Block Diagonal Representation for Noisy Image Datasets

As a relatively advanced method, the subspace clustering algorithm by block diagonal representation (BDR) will be competent in performing subspace clustering on a dataset if the dataset is assumed to be noise-free and drawn from the union of independent linear subspaces. Unfortunately, this assumption is far from reality, since the real data are usually corrupted by various noises and the subspaces of data overlap with each other, the performance of linear subspace clustering algorithms, including BDR, degrades on the real complex data. To solve this problem, we design a new objective function based on BDR, in which l2,1 norm of the reconstruction error is introduced to model the noises and improve the robustness of the algorithm. After optimizing the objective function, we present the corresponding subspace clustering algorithm to pursue a self-expressive coefficient matrix with a block diagonal structure for a noisy dataset. An affinity matrix is constructed based on the coefficient matrix, and then fed to the spectral clustering algorithm to obtain the final clustering results. Experiments on several artificial noisy image datasets show that the proposed algorithm has robustness and better clustering performance than the compared algorithms.

Extension procedures for lattice Lipschitz operators on Euclidean spaces

We present a new class of Lipschitz operators on Euclidean lattices that we call lattice Lipschitz maps, and we prove that the associated McShane and Whitney formulas provide the same extension result that holds for the real valued case. Essentially, these maps satisfy a (vector-valued) Lipschitz inequality involving the order of the lattice, with the peculiarity that the usual Lipschitz constant becomes a positive real function. Our main result shows that, in the case of Euclidean space, being lattice Lipschitz is equivalent to having a diagonal representation, in which the coordinate coefficients are real-valued Lipschitz functions. We also show that in the linear case the extension of a diagonalizable operator from the values in their eigenvectors coincide with the operator obtained both from the McShane and the Whitney formulae. Our work on such extension/representation formulas is intended to follow current research on the design of machine learning algorithms based on the extension of Lipschitz functions.

Joint Dual-Structural Constrained and Non-negative Analysis Representation Learning for Pattern Classification

ABSTRACT In recent years, analysis dictionary learning (ADL) model has attracted much attention from researchers, owing to its scalability and efficiency in representation-based classification. Despite the supervised label information embedding, the classification performance of analysis representation suffers from the redundant and noisy samples in real-world datasets. In this paper, we propose a joint Dual-Structural constrained and Non-negative Analysis Representation (DSNAR) learning model. First, the supervised latent structural transformation term is considered implicitly to generate a roughly block diagonal representation for intra-class samples. However, this discriminative structure is fragile and weak in the presence of noisy and redundant samples. To highlight both intra-class similarity and inter-class separation for class-oriented representation, we then explicitly incorporate an off-block suppressing term on the ADL model, together with a non-negative representation constraint, to achieve a well-structured and meaningful interpretation of the contributions from all class-oriented atoms. Moreover, a robust classification scheme in latent space is proposed to avoid accidental incorrect predictions with noisy information. Finally, the DSNAR model is alternatively solved by the K-SVD method, iterative re-weighted method and gradient method efficiently. Extensive classification results on five benchmark datasets validate the performance superiority of our DSNAR model compared to other state-of-the-art DL models.

Multi-view subspace clustering network with block diagonal and diverse representation

Learning multi-view data, and especially multi-view data clustering, is a popular area in computer vision and pattern recognition. The multi-view subspace clustering has achieved a better clustering quality than the single-view subspace clustering, mainly because of the complementarity of multi-view information. First, for not directly pursuing a block diagonal representation matrix of previous ℓ1 or ℓ2 regularizers in a deep subspace clustering network, a k-block diagonal regularizer is proposed to replace traditional regularizers. This block diagonal representation module is integrated into this multi-view subspace clustering network, and it can improve the clustering quality. Secondly, there exists some redundancy among the representation matrices, and a diverse representation module can be introduced into this network. This can boost the diversity of representation matrices, and make learned representation matrices more discriminative and help improve the clustering performance. In this paper, based on the deep subspace clustering network, we integrate the block diagonal and diverse representation into the network, and a multi-view subspace clustering network with the block diagonal and the diverse representation is proposed. The experimental results on the UCI Digit, Caltech101-20, COIL100 and Caltech101-7 datasets have demonstrated the superior performance of the proposed algorithm over other popular multi-view algorithms.

Block Diagonal Representation Learning for Hyperspectral Band Selection

Hyperspectral band selection is viewed as an effective dimension reduction method. Recently, researchers present graph-based clustering for hyperspectral image (HSI) processing. However, most of them conduct clustering on a fixed data matrix so that it is sensitive to the quality of initial matrix. Moreover, these algorithms apply spectral clustering to obtain the final clustering result in increasing the time consumption. Based on these facts, we propose a block diagonal representation learning algorithm (BDRLA) in this paper. BDRLA generates a high-quality similarity matrix by approximating the initial affinity matrix. Meanwhile, motivated to the spectral bands distance similarity matrix has a clear diagonal structure, a block diagonal similarity matrix with ordered partition points based upon the ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> -norm is constructed. By doing so, the obtained similarity matrix is directly applied to subsequent processing without extracting the clustering indicators. Additionally, in order to estimate the importance of bands, dictionary learning is adopted to select the representative band in each cluster. Extensive experiment results on three public datasets indicate that the bands selected by the proposed method achieve satisfactory performance.

Fast subspace clustering by learning projective block diagonal representation

Block Diagonal Representation (BDR) has attracted massive attention in subspace clustering, yet the high computational cost limits its widespread application. To address this issue, we propose a novel approach called Projective Block Diagonal Representation (PBDR), which rapidly pursues a representation matrix with the block diagonal structure. Firstly, an effective sampling strategy is utilized to select a small subset of the original large-scale data. Then, we learn a projection mapping to match the block diagonal representation matrix on the selected subset. After training, we employ the learned projection mapping to quickly generate the representation matrix with an ideal block diagonal structure for the original large-scale data. Additionally, we further extend the proposed PBDR model (i.e., PBDRℓ1 and PBDR*) by capturing the global or local structure of the data to enhance block diagonal coding capability. This paper also proves the effectiveness of the proposed model theoretically. Especially, this is the first work to directly learn a representation matrix with a block diagonal structure to handle the large-scale subspace clustering problem. Finally, experimental results on publicly available datasets show that our approaches achieve faster and more accurate clustering results compared to the state-of-the-art block diagonal-based subspace clustering approaches, which demonstrates its practical usefulness.

Prediction of brain age based on the community structure of functional networks

Studies have demonstrated that brain functional networks (BFNs) and machine learning are promising methods for predicting brain age. However, many studies ignore the important structural properties of BFNs and the feature alterations of community structure in healthy aging brains. Given this issue, we developed a novel scheme to predict brain age based on resting-state functional magnetic resonance imaging (rs-fMRI). Firstly, motivated by the fact that the brain is organized with sparsity, modular and other properties, we established a novel BFNs model using the nonnegative block diagonal representation (NBDR) with block diagonal matrix-induced regularization. Secondly, the collective sparse symmetric nonnegative matrix factorization (cssNMF) was adopted to detect the individual-level community strengths as a mesoscale network topology structure, which combined with small-scale and large-scale network topologies, were used as features. Finally, brain age was predicted by using six different models with a machine learning framework based on the extracted features. Experimental results on both simulated and real data revealed that the novel BFNs method was superior to other compared methods in connectivity, and the feature extraction scheme proposed performed well in predicting brain age. This research provides a novel insight into the construction of BFNs and the feature extraction of brain signals, which is useful to understand the mechanism of normal brain aging.

Latent block diagonal representation for subspace clustering

Latent block diagonal representation for subspace clustering

Block Diagonal Least Squares Regression for Subspace Clustering

Least squares regression (LSR) is an effective method that has been widely used for subspace clustering. Under the conditions of independent subspaces and noise-free data, coefficient matrices can satisfy enforced block diagonal (EBD) structures and achieve good clustering results. More importantly, LSR produces closed solutions that are easier to solve. However, solutions with block diagonal properties that have been solved using LSR are sensitive to noise or corruption as they are fragile and easily destroyed. Moreover, when using actual datasets, these structures cannot always guarantee satisfactory clustering results. Considering that block diagonal representation has excellent clustering performance, the idea of block diagonal constraints has been introduced into LSR and a new subspace clustering method, which is named block diagonal least squares regression (BDLSR), has been proposed. By using a block diagonal regularizer, BDLSR can effectively reinforce the fragile block diagonal structures of the obtained matrices and improve the clustering performance. Our experiments using several real datasets illustrated that BDLSR produced a higher clustering performance compared to other algorithms.

Enforced block diagonal subspace clustering with closed form solution

Subspace clustering aims to fit each category of data points by learning an underlying subspace and then conduct clustering according to the learned subspace. Ideally, the learned subspace is expected to be block diagonal such that the similarities between clusters are zeros. In this paper, we provide the explicit theoretical connection between spectral clustering and the subspace clustering based on block diagonal representation. We propose Enforced Block Diagonal Subspace Clustering (EBDSC) and show that the spectral clustering with the Radial Basis Function kernel can be regarded as EBDSC. Compared with the exiting subspace clustering methods, an analytical, nonnegative and symmetrical solution can be obtained by EBDSC. An important difference with respect to the existing ones is that our model is a more general case. EBDSC directly uses the obtained solution as the similarity matrix, which can avoid the complex computation of the optimization program. Then the solution obtained by the proposed method can be used for the final clustering. Finally, we provide the experimental analysis to show the efficiency and effectiveness of our method on the synthetic data and several benchmark data sets in terms of different metrics.