Block Diagonal Property Research Articles

Enforced Block Diagonal Graph Learning for Multikernel Clustering

The existing multikernel graph clustering (MKGC) methods have emerged with notable success on nonlinear clustering tasks since the graph learning can effectively capture graph structure similarity between sample pair. Ideally, a high-quality graph should enjoy the good block diagonal property, i.e. the intercluster similarities correspond to zeros, while intracluster similarities represent nonzeros. Meanwhile, the number of diagonal blocks for a graph equals to the number of clusters on the dataset. However, most of the existing MKGC methods design a corpulent three-parts graph leaning process that poses challenges for hyperparameter tuning, time cost, and clustering performance. To overcome these challenging issues, we propose an enforced block diagonal graph learning for multikernel clustering (EBDGL-MKC) method, where we pursue a high-quality block diagonal graph via well-designed one-part graph leaning scheme rather than three parts. Inspired by symmetric matrix factorization (SMF), we first design a one-part block diagonal graph learning scheme to learn multiple block diagonal graphs, by exploring an explicit theoretical connection between the clustering partition of kernel <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$</tex-math> </inline-formula> -means and the excellent block diagonal graph. Then, these block diagonal graphs are stacked into a low-rank tensor for exploiting the high-order structure information hidden in the nonlinear data. After that, an effective alternate algorithm with convergence proof is performed on extensive experiments to demonstrate the superiority of EBDGL compared with the state-of-the-art multikernel clustering (MKC) methods.

Multiple kernel k-means clustering with block diagonal property

Multiple kernel k-means clustering with block diagonal property

Multiple kernel clustering with structure-preserving and block diagonal property

Multiple kernel clustering with structure-preserving and block diagonal property

Maximum Block Energy Guided Robust Subspace Clustering.

Subspace clustering is useful for clustering data points according to the underlying subspaces. Many methods have been presented in recent years, among which Sparse Subspace Clustering (SSC), Low-Rank Representation (LRR) and Least Squares Regression clustering (LSR) are three representative methods. These approaches achieve good results by assuming the structure of errors as a prior and removing errors in the original input space by modeling them in their objective functions. In this paper, we propose a novel method from an energy perspective to eliminate errors in the projected space rather than the input space. Since the block diagonal property can lead to correct clustering, we measure the correctness in terms of a block in the projected space with an energy function. A correct block corresponds to the subset of columns with the maximal energy. The energy of a block is defined based on the unary column, pairwise and high-order similarity of columns for each block. We relax the energy function of a block and approximate it by a constrained homogenous function. Moreover, we propose an efficient iterative algorithm to remove errors in the projected space. Both theoretical analysis and experiments show the superiority of our method over existing solutions to the clustering problem, especially when noise exists.

Maximum Entropy Subspace Clustering Network

Deep subspace clustering networks have attracted much attention in subspace clustering, in which an auto-encoder non-linearly maps the input data into a latent space, and a fully connected layer named self-expressiveness module is introduced to learn the affinity matrix via a typical regularization term (e.g., sparse or low-rank). However, the adopted regularization terms ignore the connectivity within each subspace, limiting their clustering performance. In addition, the adopted framework suffers from the coupling issue between the auto-encoder module and the self-expressiveness module, making the network training non-trivial. To tackle these two issues, we propose a novel deep subspace clustering method named Maximum Entropy Subspace Clustering Network (MESC-Net). Specifically, MESC-Net maximizes the entropy of the affinity matrix to promote the connectivity within each subspace, in which its elements corresponding to the same subspace are uniformly and densely distributed. Meanwhile, we design a novel framework to explicitly decouple the auto-encoder module and the self-expressiveness module. Besides, we also theoretically prove that the learned affinity matrix satisfies the block-diagonal property under the assumption of independent subspaces. Extensive quantitative and qualitative results on commonly used benchmark datasets validate MESC-Net significantly outperforms state-of-the-art methods. The code is publicly available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/ZhihaoPENG-CityU/MESC</uri> .

Robust multiple kernel subspace clustering with block diagonal representation and low-rank consensus kernel

Multiple kernel learning (MKL) is widely used to deal with nonlinear clustering problem, because MKL can obtain the optimal consensus kernel by designing kernel weighting strategy, which avoids the limitation of single-kernel learning restricted by the selected kernel function. However, existing MKL subspace clustering algorithms ignore noise and the low-rank structure of the data in the feature space. We propose a new robust multiple kernel subspace clustering algorithm (LRMKSC) with block diagonal representation (BDR) and low-rank consensus kernel (LRCK), which is more systematic and comprehensive for data processing. In particular, (1) to learn the optimal consensus kernel, we design an automatic weighting strategy using Mixture Correntropy Induced Metric (MCIM), which not only sets the optimal weight for each kernel, but also improves the robustness of LRMKSC by suppressing noise; (2) to explore low-rank structure of input data in feature space, we propose the Weighted Truncated Schatten p-Norm (WTSN) and apply it to the low-rank constraint of the optimal consensus kernel; (3) considering the block diagonal property of the affinity matrix, we apply block diagonal constraint to the coefficient matrix. LRMKSC combines MKL, LRCK and BDR to solve these problems simultaneously. Through the interaction of three technologies, the results of other technologies are used in the overall optimal solution to iteratively improve the efficiency of each technology, and finally form an overall optimal algorithm for processing nonlinear structural data. Compared with the most advanced MKL subspace clustering algorithms, extensive experiments on image and text datasets verify the competitiveness of LRMKSC.

Structured block diagonal representation for subspace clustering

The aim of the subspace clustering is to segment the high-dimensional data into the corresponding subspace. The structured sparse subspace clustering and the block diagonal representation clustering are quite advanced spectral-type subspace clustering algorithms when handling to the linear subspaces. In this paper, the respective advantages of these two algorithms are fully exploited, and the structured block diagonal representation (SBDR) subspace clustering is proposed. In many classical spectral-type subspace clustering algorithms, the affinity matrix which obeys the block diagonal property can not necessarily bring satisfying clustering results. However, the k-block diagonal regularizer of the SBDR algorithm directly pursues the block diagonal matrix, and this regularizer is obviously more effective. On the other hand, the general procedure of the spectral-type subspace clustering algorithm is to get the affinity matrix firstly and next perform the spectral clustering. The SBDR algorithm considers the intrinsic relationship of the two seemingly separate steps, the subspace structure matrix obtained by the spectral clustering is used iteratively to facilitate a better initialization for the representation matrix. The experimental results on the synthetic dataset and the real dataset have demonstrated the superior performance of the proposed algorithm over other prevalent subspace clustering algorithms.

Joint correntropy metric weighting and block diagonal regularizer for robust multiple kernel subspace clustering

Nonlinear kernel-based subspace clustering methods that can reveal the multi-cluster nonlinear structure of samples are an emerging research topic. However, the existing kernel subspace clustering methods have the following three flaws: 1) their clustering performance is largely determined by the chosen kernel function; 2) they may lack robustness in the presence of non-Gaussian noise and impulsive noise; and 3) their learned affinity matrix can not hold the desired block diagonal property for clustering purpose, which possibly leads to incorrect clustering when using spectral clustering. In this paper, we propose a Joint Robust Multiple Kernel Subspace Clustering (JMKSC) method for data clustering, which has two primary innovations. First, our multiple kernel weighting strategy introduces the correntropy metric weighting instead of a fixed, or inappropriately assigned weighting, which is more robust to the non-Gaussian noise and contributes to learning the optimal consensus kernel. Second, our method encourages acquiring an affinity matrix with the optimal block diagonal property based on the block diagonal regularizer (BDR) and the self-expressiveness property. Experiments on several different types of datasets confirm that the proposed JMKSC significantly outperforms several state-of-the-art single kernel and multiple kernel subspace clustering methods in terms of accuracy, NMI and purity.

Multi-View Subspace Clustering With Block Diagonal Representation

Self-representation model has made good progress for a single view subspace clustering. This paper proposed the multi-view subspace clustering model based on self-representation. This model assumes that the samples from different classes are embedded in independent subspaces. Thus, the fused multi-view self-representation feature should be block diagonal, and a block diagonal regularizer with the complementarity of multi-view information is given. The model optimization algorithm by alternating minimization is proposed and its convergence without any additional assumption is proved. With the complementarity of multi-view information and the block diagonal property, our model will depict data more comprehensively than single view independently. The extensive experiments on public datasets demonstrate the effectiveness of our proposed model.

Ensemble Subspace Segmentation Under Blockwise Constraints

The graph-based subspace segmentation technique has garnered a lot of attention in the visual data representation problem. In general, data (e.g., tracks of moving objects) are drawn from multiple linear subspaces. Thus, how to build a block-diagonal affinity matrix is the critical problem. In this paper, we propose a novel graph-based method, Ensemble Subspace Segmentation under Blockwise constraints (ESSB), which unifies least squares regression and a locality preserving graph regularizer into an ensemble learning framework. Specifically, compact encoding using least squares regression coefficients helps achieve a block-diagonal representation matrix among all samples. Meanwhile, the locality preserving regularizer tends to capture the intrinsic local structure, which further enhances the block-diagonal property. Both the blockwise efforts, i.e., least squares regression and the sparse regularizer, work jointly and are formulated in the ensemble learning framework, making ESSB more robust and efficient, especially when handling high-dimensional data. Finally, an efficient optimization solution based on inexact augmented Lagrange multiplier is derived with theoretical time complexity analysis. To demonstrate the effectiveness of the proposed method, we consider three different applications: face clustering, object clustering, and motion segmentation. Extensive results of both accuracy and normalized mutual information on four benchmarks, i.e., YaleB, ORL, COIL and Hopkins155, are reported. Also, the evaluations of computational cost are provided, based on which the superiority of our proposed method in both accuracy and efficiency is demonstrated compared with 12 baseline algorithms.

Robust discriminant low-rank representation for subspace clustering

For the low-rank representation-based subspace clustering, the affinity matrix is block diagonal. In this paper, a novel robust discriminant low-rank representation (RDLRR) algorithm is proposed to enhance the block diagonal property to explore the multiple subspace structures of samples. In order to cluster samples into their corresponding subspace and remove outliers, the proposed RDLRR considers both the within-class and the between-class distance during seeking the lowest-rank representation of samples. RDLRR could well indicate the global structure of samples, when the labeling is available. We conduct experiments on several datasets, including the Extended Yale B, AR and Hopkins 155, to show that our approach outperforms all the other state-of-the-art approaches.

Subspace Clustering by Block Diagonal Representation.

This paper studies the subspace clustering problem. Given some data points approximately drawn from a union of subspaces, the goal is to group these data points into their underlying subspaces. Many subspace clustering methods have been proposed and among which sparse subspace clustering and low-rank representation are two representative ones. Despite the different motivations, we observe that many existing methods own the common block diagonal property, which possibly leads to correct clustering, yet with their proofs given case by case. In this work, we consider a general formulation and provide a unified theoretical guarantee of the block diagonal property. The block diagonal property of many existing methods falls into our special case. Second, we observe that many existing methods approximate the block diagonal representation matrix by using different structure priors, e.g., sparsity and low-rankness, which are indirect. We propose the first block diagonal matrix induced regularizer for directly pursuing the block diagonal matrix. With this regularizer, we solve the subspace clustering problem by Block Diagonal Representation (BDR), which uses the block diagonal structure prior. The BDR model is nonconvex and we propose an alternating minimization solver and prove its convergence. Experiments on real datasets demonstrate the effectiveness of BDR.

Block-diagonalization of the variational nodal response matrix using the symmetry group theory

To further improve the efficiency of the Variational Nodal Method (VNM) for solving the neutron transport equation in hexagonal-z geometry, the nodal response matrix is further block-diagonalized by utilizing the symmetry group theory to decompose the surface basis functions into irreducible components. The block-diagonal property of the nodal response matrix is determined by the symmetry properties of the hexagonal node in geometry, material and basis functions, including both reflection and rotation symmetries. To fully utilize those properties, the symmetry group theory is employed to analyze the symmetry property of the nodal response matrices. It is mathematically proved that the nodal response matrix can be further block-diagonalized into 16 diagonal blocks instead of the current 4 ones by using the symmetry group theory. Numerical comparisons demonstrate that the new approach can reduce the memory storage and computing time by a factor of 2∼3 for P7 angular approximation, compared with the currently employed variables transformation algorithm.

A 3-D Continuous–Discontinuous Galerkin Finite-Element Time-Domain Method for Maxwell's Equations

A 3-D continuous-discontinuous Galerkin (CDG) finite-element time-domain method for Maxwell's equations is proposed to analyze transient electromagnetic problems, which is based on the field variables E and B. This method is aimed at exploiting the advantages of reduced number of unknowns for continuous Galerkin (CG) method and block-diagonal property of discontinuous Galerkin (DG) method. The whole domain is divided into several clusters. The CG is used for the elements in the clusters, and they exchange information with adjacent clusters through traditional numerical fluxes in a DG manner. Moreover, compared to the conventional method based on field variables E and H, the proposed method can eliminate spurious modes. Numerical results demonstrate that the proposed CDG method is superior to the DG method in terms of memory and simulation time.

MIMO Radar Space–Time Adaptive Processing Using Prolate Spheroidal Wave Functions

In the traditional transmitting beamforming radar system, the transmitting antennas send coherent waveforms which form a highly focused beam. In the multiple-input multiple-output (MIMO) radar system, the transmitter sends noncoherent (possibly orthogonal) broad (possibly omnidirectional) waveforms. These waveforms can be extracted at the receiver by a matched filterbank. The extracted signals can be used to obtain more diversity or to improve the spatial resolution for clutter. This paper focuses on space-time adaptive processing (STAP) for MIMO radar systems which improves the spatial resolution for clutter. With a slight modification, STAP methods developed originally for the single-input multiple-output (SIMO) radar (conventional radar) can also be used in MIMO radar. However, in the MIMO radar, the rank of the jammer-and-clutter subspace becomes very large, especially the jammer subspace. It affects both the complexity and the convergence of the STAP algorithm. In this paper, the clutter space and its rank in the MIMO radar are explored. By using the geometry of the problem rather than data, the clutter subspace can be represented using prolate spheroidal wave functions (PSWF). A new STAP algorithm is also proposed. It computes the clutter space using the PSWF and utilizes the block-diagonal property of the jammer covariance matrix. Because of fully utilizing the geometry and the structure of the covariance matrix, the method has very good SINR performance and low computational complexity.