Block Diagonal Representation Research Articles

Block diagonal representation learning with local invariance for face clustering

Block diagonal representation learning with local invariance for face clustering

A novel graph-based multiple kernel learning framework for hyperspectral image classification

ABSTRACT Multiple kernel learning (MKL) is an efficient way to improve hyperspectral image classification with few training samples by integrating spectral and spatial features. Nonetheless, presenting a MKL algorithm to capture significant information and eliminate redundancy is indispensable. In this paper, a novel graph-based MKL method (GMKL) is introduced to enhance the performance of the MKL algorithm and achieve high discriminative features, resulting in maximum separability. To do so, an informative low-rank graph is determined via a new algorithm called block diagonal representation that takes into account both local and global data structures. Then, in lieu of a time-consuming search for finding the optimal combination of the basic kernels, unlike the prior works, an optimal projective direction is acquired using graph theory. Various basic kernels generated from superpixels and attribute profiles are projected to produce a discriminative combination of kernels. Eventually, the optimum kernel is given to the support vector machine classifier. Experiments have been conducted on two popular hyperspectral datasets (Indian Pines and Pavia University). For the datasets, the proposed algorithm gained 90.43% and 99.28% overall accuracy, respectively, using only 1% training samples, which is much better than the state-of-the-art MKL algorithm and deep learning-based approaches with lower computational time.

Consensus Affinity Graph Learning via Structure Graph Fusion and Block Diagonal Representation for Multiview Clustering

Learning a robust affinity graph is fundamental to graph-based clustering methods. However, some existing affinity graph learning methods have encountered the following problems. First, the constructed affinity graphs cannot capture the intrinsic structure of data well. Second, when fusing all view-specific affinity graphs, most of them obtain a fusion graph by simply taking the average of multiple views, or directly learning a common graph from multiple views, without considering the discriminative property among diverse views. Third, the fusion graph does not maintain an explicit cluster structure. To alleviate these problems, the adaptive neighbor graph learning approach and the data self-expression approach are first integrated into a structure graph fusion framework to obtain a view-specific structure affinity graph to capture the local and global structures of data. Then, all the structural affinity graphs are weighted dynamically into a consensus affinity graph, which not only effectively incorporates the complementary affinity structure of important views but also has the capability of preserving the consensus affinity structure that is shared by all views. Finally, a k–block diagonal regularizer is introduced for the consensus affinity graph to encourage it to have an explicit cluster structure. An efficient optimization algorithm is developed to tackle the resultant optimization problem. Extensive experiments on benchmark datasets validate the superiority of the proposed method.

Brain connectivity patterns derived from aging-related alterations in dynamic brain functional networks and their potential as features for brain age classification.

Recent studies have demonstrated that the analysis of brain functional networks (BFN) is a powerful tool for exploring brain aging and age-related neurodegenerative diseases. However, investigating the mechanism of brain aging associated with dynamic BFN is still limited. The purpose of this study is to develop a novel scheme to explore brain aging patterns by constructing dynamic BFN using resting-state functional magnetic resonance imaging (rs-fMRI) data. A dynamic sliding-windowed non-negative block-diagonal representation (dNBDR) method is proposed for constructing dynamic BFN, based on which a collection of dynamic BFN measures are suggested for examining age-related differences at the group level and used as features for brain age classification at the individual level. The experimental results reveal that the dNBDR method is superior to the sliding time window with Pearson correlation (SWPC) method in terms of dynamic network structure quality. Additionally, significant alterations in dynamic BFN structures exist across the human lifespan. Specifically, average node flexibility and integration coefficient increase with age, while the recruitment coefficient shows a decreased trend. The proposed feature extraction scheme based on dynamic BFN achieved the highest accuracy of 78.7% in classifying three brain age groups. These findings suggest that dynamic BFN measures, dynamic community structure metrics in particular, play an important role in quantitatively assessing brain aging.

Convex Subspace Clustering by Adaptive Block Diagonal Representation.

Subspace clustering is a class of extensively studied clustering methods where the spectral-type approaches are its important subclass. Its key first step is to desire learning a representation coefficient matrix with block diagonal structure. To realize this step, many methods were successively proposed by imposing different structure priors on the coefficient matrix. These impositions can be roughly divided into two categories, i.e., indirect and direct. The former introduces the priors such as sparsity and low rankness to indirectly or implicitly learn the block diagonal structure. However, the desired block diagonalty cannot necessarily be guaranteed for noisy data. While the latter directly or explicitly imposes the block diagonal structure prior such as block diagonal representation (BDR) to ensure so-desired block diagonalty even if the data is noisy but at the expense of losing the convexity that the former's objective possesses. For compensating their respective shortcomings, in this article, we follow the direct line to propose adaptive BDR (ABDR) which explicitly pursues block diagonalty without sacrificing the convexity of the indirect one. Specifically, inspired by Convex BiClustering, ABDR coercively fuses both columns and rows of the coefficient matrix via a specially designed convex regularizer, thus naturally enjoying their merits and adaptively obtaining the number of blocks. Finally, experimental results on synthetic and real benchmarks demonstrate the superiority of ABDR to the state-of-the-arts (SOTAs).

Discriminative Dictionary Pair Learning With Scale-Constrained Structured Representation for Image Classification.

The dictionary pair learning (DPL) model aims to design a synthesis dictionary and an analysis dictionary to accomplish the goal of rapid sample encoding. In this article, we propose a novel structured representation learning algorithm based on the DPL for image classification. It is referred to as discriminative DPL with scale-constrained structured representation (DPL-SCSR). The proposed DPL-SCSR utilizes the binary label matrix of dictionary atoms to project the representation into the corresponding label space of the training samples. By imposing a non-negative constraint, the learned representation adaptively approximates a block-diagonal structure. This innovative transformation is also capable of controlling the scale of the block-diagonal representation by enforcing the sum of within-class coefficients of each sample to 1, which means that the dictionary atoms of each class compete to represent the samples from the same class. This implies that the requirement of similarity preservation is considered from the perspective of the constraint on the sum of coefficients. More importantly, the DPL-SCSR does not need to design a classifier in the representation space as the label matrix of the dictionary can also be used as an efficient linear classifier. Finally, the DPL-SCSR imposes the l2,p -norm on the analysis dictionary to make the process of feature extraction more interpretable. The DPL-SCSR seamlessly incorporates the scale-constrained structured representation learning, within-class similarity preservation of representation, and the linear classifier into one regularization term, which dramatically reduces the complexity of training and parameter tuning. The experimental results on several popular image classification datasets show that our DPL-SCSR can deliver superior performance compared with the state-of-the-art (SOTA) dictionary learning methods. The MATLAB code of this article is available at https://github.com/chenzhe207/DPL-SCSR.

Kernel Block Diagonal Representation Subspace Clustering with Similarity Preservation

Subspace clustering methods based on the low-rank and sparse model are effective strategies for high-dimensional data clustering. However, most existing low-rank and sparse methods with self-expression can only deal with linear structure data effectively, but they cannot handle data with complex nonlinear structure well. Although kernel subspace clustering methods can efficiently deal with nonlinear structure data, some similarity information between samples may be lost when the original data are reconstructed in the kernel space. Moreover, these kernel subspace clustering methods may not obtain an affinity matrix with an optimal block diagonal structure. In this paper, we propose a novel subspace clustering method termed kernel block diagonal representation subspace clustering with similarity preservation (KBDSP). KBDSP contains three contributions: (1) an affinity matrix with block diagonal structure is generated by introducing a block diagonal representation term; (2) a similarity-preserving regularizer is constructed and embedded into our model by minimizing the discrepancy between inner products of original data and inner products of reconstructed data in the kernel space, which better preserve the similarity information between original data; (3) the KBDSP model is proposed by integrating the block diagonal representation term and similarity-preserving regularizer into the kernel self-expressing frame. The optimization of our proposed model is solved efficiently by utilizing the alternating direction method of multipliers (ADMM). Experimental results on nine datasets demonstrate the effectiveness of the proposed method.

State space model-based harmonic decomposition of pseudo-periodic non-stationary multivariate signals

This work is concerned with the harmonic decomposition of pseudo-periodic non-stationary multivariate signals. In this framework, a signal component corresponds to a single sinusoid with time-dependent amplitude and frequency. Thus, in the proposed harmonic decomposition, all the signal components are bound to share the same fundamental time-dependent frequency. Here, we establish a procedure to yield such decomposition by casting the multivariate signal into a block-diagonal state-space representation with multiple outputs. The harmonic components and instantaneous frequency are estimated using Kalman filtering. In addition, we provide an optimisation framework for the hyperparameters of the state space representation based on the Expectation-Maximisation algorithm. The proposed method provides an accurate, compact and physically explainable procedure for analysing pseudo-periodic non-stationary multiple-output systems, as demonstrated in the case of a simulated unbalanced rotor system and on real electrocardiographic records.

Naimark dilations of qubit POVMs and joint measurements

Measurement incompatibility is one of the cornerstones of quantum theory. This phenomenon appears in many forms, of which the concept of non-joint measurability has received considerable attention in the recent years. In order to characterise this non-classical phenomenon, various analytical and numerical methods have been developed. The analytical approaches have mostly concentrated on the qubit case, as well as to scenarios involving sets of measurements with symmetries, such as position and momentum or sets of mutually unbiased bases. The numerical methods can, in principle, decide any finite-dimensional and discrete joint measurability problem, but they naturally have practical limitations in terms of computational power. These methods exclusively start from a given set of measurements and ask whether the set possesses incompatibility. Here, we take a complementary approach by asking which measurements are compatible with a given measurement. It turns out, that this question can be answered in full generality through a minimal Naimark dilation of the given measurement: the set of interest is exactly those measurements that have a block-diagonal representation in such dilation. We demonstrate the use of the technique through various qubit examples, leading to an alternative characterisation of all compatible pairs of binary qubit measurements, which retrieves the celebrated Busch criterion. We further apply the technique to special examples of trinary and continuous qubit measurements.

Robust Subspace Clustering with Block Diagonal Representation for Noisy Image Datasets

As a relatively advanced method, the subspace clustering algorithm by block diagonal representation (BDR) will be competent in performing subspace clustering on a dataset if the dataset is assumed to be noise-free and drawn from the union of independent linear subspaces. Unfortunately, this assumption is far from reality, since the real data are usually corrupted by various noises and the subspaces of data overlap with each other, the performance of linear subspace clustering algorithms, including BDR, degrades on the real complex data. To solve this problem, we design a new objective function based on BDR, in which l2,1 norm of the reconstruction error is introduced to model the noises and improve the robustness of the algorithm. After optimizing the objective function, we present the corresponding subspace clustering algorithm to pursue a self-expressive coefficient matrix with a block diagonal structure for a noisy dataset. An affinity matrix is constructed based on the coefficient matrix, and then fed to the spectral clustering algorithm to obtain the final clustering results. Experiments on several artificial noisy image datasets show that the proposed algorithm has robustness and better clustering performance than the compared algorithms.

G-Circulant Quantum Markov Semigroups

We broaden the study of circulant Quantum Markov Semigroups (QMS). First, we introduce the notions of [Formula: see text]-circulant GKSL generator and [Formula: see text]-circulant QMS from the circulant case, corresponding to [Formula: see text], to an arbitrary finite group [Formula: see text]. Second, we show that each [Formula: see text]-circulant GKSL generator has a block-diagonal representation [Formula: see text], where [Formula: see text] is a [Formula: see text]-circulant matrix determined by some [Formula: see text]. Denoting by [Formula: see text] the subgroup of [Formula: see text] generated by the support of [Formula: see text], we prove that [Formula: see text] has its own block-diagonal matrix representation [Formula: see text] where [Formula: see text] is an irreducible [Formula: see text]-circulant matrix and [Formula: see text] is the index of [Formula: see text] in [Formula: see text]. Finally, we exploit such block representations to characterize the structure, steady states, and asymptotic evolution of [Formula: see text]-circulant QMSs.

Projection-preserving block-diagonal low-rank representation for subspace clustering

In this paper, a novel model named projection-preserving block-diagonal low-rank representation (PBDIR) is proposed and can obtain a more distinguishable representation matrix for clustering. PBDIR acquires a more advantageous representation by extracting the essential features. Specifically, we introduce a projection matrix to our model to learn a new feature space that can capture more significant features. Therefore, our model learns a more robust representation, which can reduce noise interference. Meanwhile, we introduce a block-diagonal regularization to ensure that the obtained representation matrix involves a k-block diagonal, where k denotes the number of clusters. This term brings more benefits for clustering tasks. Experimental results on real datasets show that our model can significantly improve the clustering performance and the proposed approach is robust against Gaussian noise, Multiplicative noise, and Salt-and-Pepper noise.

Multi-view subspace clustering network with block diagonal and diverse representation

Learning multi-view data, and especially multi-view data clustering, is a popular area in computer vision and pattern recognition. The multi-view subspace clustering has achieved a better clustering quality than the single-view subspace clustering, mainly because of the complementarity of multi-view information. First, for not directly pursuing a block diagonal representation matrix of previous ℓ1 or ℓ2 regularizers in a deep subspace clustering network, a k-block diagonal regularizer is proposed to replace traditional regularizers. This block diagonal representation module is integrated into this multi-view subspace clustering network, and it can improve the clustering quality. Secondly, there exists some redundancy among the representation matrices, and a diverse representation module can be introduced into this network. This can boost the diversity of representation matrices, and make learned representation matrices more discriminative and help improve the clustering performance. In this paper, based on the deep subspace clustering network, we integrate the block diagonal and diverse representation into the network, and a multi-view subspace clustering network with the block diagonal and the diverse representation is proposed. The experimental results on the UCI Digit, Caltech101-20, COIL100 and Caltech101-7 datasets have demonstrated the superior performance of the proposed algorithm over other popular multi-view algorithms.

Block Diagonal Representation Learning for Hyperspectral Band Selection

Hyperspectral band selection is viewed as an effective dimension reduction method. Recently, researchers present graph-based clustering for hyperspectral image (HSI) processing. However, most of them conduct clustering on a fixed data matrix so that it is sensitive to the quality of initial matrix. Moreover, these algorithms apply spectral clustering to obtain the final clustering result in increasing the time consumption. Based on these facts, we propose a block diagonal representation learning algorithm (BDRLA) in this paper. BDRLA generates a high-quality similarity matrix by approximating the initial affinity matrix. Meanwhile, motivated to the spectral bands distance similarity matrix has a clear diagonal structure, a block diagonal similarity matrix with ordered partition points based upon the ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> -norm is constructed. By doing so, the obtained similarity matrix is directly applied to subsequent processing without extracting the clustering indicators. Additionally, in order to estimate the importance of bands, dictionary learning is adopted to select the representative band in each cluster. Extensive experiment results on three public datasets indicate that the bands selected by the proposed method achieve satisfactory performance.

Fast subspace clustering by learning projective block diagonal representation

Block Diagonal Representation (BDR) has attracted massive attention in subspace clustering, yet the high computational cost limits its widespread application. To address this issue, we propose a novel approach called Projective Block Diagonal Representation (PBDR), which rapidly pursues a representation matrix with the block diagonal structure. Firstly, an effective sampling strategy is utilized to select a small subset of the original large-scale data. Then, we learn a projection mapping to match the block diagonal representation matrix on the selected subset. After training, we employ the learned projection mapping to quickly generate the representation matrix with an ideal block diagonal structure for the original large-scale data. Additionally, we further extend the proposed PBDR model (i.e., PBDRℓ1 and PBDR*) by capturing the global or local structure of the data to enhance block diagonal coding capability. This paper also proves the effectiveness of the proposed model theoretically. Especially, this is the first work to directly learn a representation matrix with a block diagonal structure to handle the large-scale subspace clustering problem. Finally, experimental results on publicly available datasets show that our approaches achieve faster and more accurate clustering results compared to the state-of-the-art block diagonal-based subspace clustering approaches, which demonstrates its practical usefulness.

Prediction of brain age based on the community structure of functional networks

Studies have demonstrated that brain functional networks (BFNs) and machine learning are promising methods for predicting brain age. However, many studies ignore the important structural properties of BFNs and the feature alterations of community structure in healthy aging brains. Given this issue, we developed a novel scheme to predict brain age based on resting-state functional magnetic resonance imaging (rs-fMRI). Firstly, motivated by the fact that the brain is organized with sparsity, modular and other properties, we established a novel BFNs model using the nonnegative block diagonal representation (NBDR) with block diagonal matrix-induced regularization. Secondly, the collective sparse symmetric nonnegative matrix factorization (cssNMF) was adopted to detect the individual-level community strengths as a mesoscale network topology structure, which combined with small-scale and large-scale network topologies, were used as features. Finally, brain age was predicted by using six different models with a machine learning framework based on the extracted features. Experimental results on both simulated and real data revealed that the novel BFNs method was superior to other compared methods in connectivity, and the feature extraction scheme proposed performed well in predicting brain age. This research provides a novel insight into the construction of BFNs and the feature extraction of brain signals, which is useful to understand the mechanism of normal brain aging.

Latent block diagonal representation for subspace clustering

Latent block diagonal representation for subspace clustering

Block Diagonal Least Squares Regression for Subspace Clustering

Least squares regression (LSR) is an effective method that has been widely used for subspace clustering. Under the conditions of independent subspaces and noise-free data, coefficient matrices can satisfy enforced block diagonal (EBD) structures and achieve good clustering results. More importantly, LSR produces closed solutions that are easier to solve. However, solutions with block diagonal properties that have been solved using LSR are sensitive to noise or corruption as they are fragile and easily destroyed. Moreover, when using actual datasets, these structures cannot always guarantee satisfactory clustering results. Considering that block diagonal representation has excellent clustering performance, the idea of block diagonal constraints has been introduced into LSR and a new subspace clustering method, which is named block diagonal least squares regression (BDLSR), has been proposed. By using a block diagonal regularizer, BDLSR can effectively reinforce the fragile block diagonal structures of the obtained matrices and improve the clustering performance. Our experiments using several real datasets illustrated that BDLSR produced a higher clustering performance compared to other algorithms.

Autoencoder-Based Latent Block-Diagonal Representation for Subspace Clustering.

Block-diagonal representation (BDR) is an effective subspace clustering method. The existing BDR methods usually obtain a self-expression coefficient matrix from the original features by a shallow linear model. However, the underlying structure of real-world data is often nonlinear, thus those methods cannot faithfully reflect the intrinsic relationship among samples. To address this problem, we propose a novel latent BDR (LBDR) model to perform the subspace clustering on a nonlinear structure, which jointly learns an autoencoder and a BDR matrix. The autoencoder, which consists of a nonlinear encoder and a linear decoder, plays an important role to learn features from the nonlinear samples. Meanwhile, the learned features are used as a new dictionary for a linear model with block-diagonal regularization, which can ensure good performances for spectral clustering. Moreover, we theoretically prove that the learned features are located in the linear space, thus ensuring the effectiveness of the linear model using self-expression. Extensive experiments on various real-world datasets verify the superiority of our LBDR over the state-of-the-art subspace clustering approaches.

Enforced block diagonal subspace clustering with closed form solution

Subspace clustering aims to fit each category of data points by learning an underlying subspace and then conduct clustering according to the learned subspace. Ideally, the learned subspace is expected to be block diagonal such that the similarities between clusters are zeros. In this paper, we provide the explicit theoretical connection between spectral clustering and the subspace clustering based on block diagonal representation. We propose Enforced Block Diagonal Subspace Clustering (EBDSC) and show that the spectral clustering with the Radial Basis Function kernel can be regarded as EBDSC. Compared with the exiting subspace clustering methods, an analytical, nonnegative and symmetrical solution can be obtained by EBDSC. An important difference with respect to the existing ones is that our model is a more general case. EBDSC directly uses the obtained solution as the similarity matrix, which can avoid the complex computation of the optimization program. Then the solution obtained by the proposed method can be used for the final clustering. Finally, we provide the experimental analysis to show the efficiency and effectiveness of our method on the synthetic data and several benchmark data sets in terms of different metrics.