Low-rank Embedding Research Articles

Unsupervised motor incipient fault detection using lightweight network and orthogonal low-rank embedding

Abstract Electrical motor is a key component in industrial systems. Detecting incipient fault of motor is critical to system reliability. However, the characteristics of faults and normal stages are difficult to distinguish, Meanwhile, the lightweight requirements of the model and the imbalance of the samples also make incipient fault detection challenged. To address these problems, this paper proposes an unsupervised fault detection method combines lightweight network and orthogonal low-rank embedding (OLE). The raw signals are firstly transformed into time-frequency images and fed into lightweight convolution network for feature extraction. Then, the features are clustered into orthogonal subspaces to enhance inter-class separability. Finally, a detection module based on distance metric is designed to identify the incipient fault of motor. The effectiveness of the proposed method is validated on four industrial motor dataset and compared with other methods.

Discriminative low-rank embedding with manifold constraint for image feature extraction and classification

Discriminative low-rank embedding with manifold constraint for image feature extraction and classification

Enhanced Multimodal Low-rank Embedding based Feature Selection Model for Multimodal Alzheimer’s Disease Diagnosis

Enhanced Multimodal Low-rank Embedding based Feature Selection Model for Multimodal Alzheimer’s Disease Diagnosis

Structure preserving projections learning via low-rank embedding for image classification

Subspace mapping is a key step and an important tool for feature extraction and selection. Although subspace mapping methods are successful, there are still some common defects: 1) The final subspace does not possess global properties, only local properties; 2) most models fail to retain the low-rank structural information; and 3) the projection matrix contains considerable redundant and irrelevant information. Accordingly, we develop a novel method, i.e., structure preserving projections learning via low-rank embedding (SPPL-LRE), to address the above issues. We first extract the principal component information by seeking the projection directions of maximum variance, which makes the final subspace equipped with global properties. Then, SPPL-LRE regresses the principal component information to a classwise block-diagonal structure. In this way, the low-rank information and main energy of the data are held in the subspace so that the obtained projection can extract more salient and informative features. Meanwhile, introducing low-rank information can also enhance the robustness of the model. Moreover, a strong L2 norm constraint is imposed on the projection to avoid model overfitting and exclude the interference of redundant information, which can enhance the interpretability of the model. Finally, we introduce the graph smoothness of the representation matrix to better preserve the manifold geometry of the original data. Extensive experiments indicate that our method is more robust and effective than other state-of-the-art methods.

Edge Structure Learning via Low Rank Residuals for Robust Image Classification

Traditional low-rank methods overlook residuals as corruptions, but we discovered that low-rank residuals actually keep image edges together with corrupt components. Therefore, filtering out such structural information could hamper the discriminative details in images, especially in heavy corruptions. In order to address this limitation, this paper proposes a novel method named ESL-LRR, which preserves image edges by finding image projections from low-rank residuals. Specifically, our approach is built in a manifold learning framework where residuals are regarded as another view of image data. Edge preserved image projections are then pursued using a dynamic affinity graph regularization to capture the more accurate similarity between residuals while suppressing the influence of corrupt ones. With this adaptive approach, the proposed method can also find image intrinsic low-rank representation, and much discriminative edge preserved projections. As a result, a new classification strategy is introduced, aligning both modalities to enhance accuracy. Experiments are conducted on several benchmark image datasets, including MNIST, LFW, and COIL100. The results show that the proposed method has clear advantages over compared state-of-the-art (SOTA) methods, such as Low-Rank Embedding (LRE), Low-Rank Preserving Projection via Graph Regularized Reconstruction (LRPP_GRR), and Feature Selective Projection (FSP) with more than 2% improvement, particularly in corrupted cases.

Unsupervised dimensionality reduction by jointing dynamic hypergraph and low-rank embedding for classification and clustering

Hypergraph learning is largely used to handle dimensional disaster problem of high-dimensional data in view of its flexible ability to model complex data correlations. However, the following three problems limit the performance of unsupervised hypergraph-based dimensionality reduction methods: (1) most of them are unable to obtain consistently good performance for different data sets; (2) noise or outliers in the data can lead to unreliability of the fixed hypergraph; and (3) the simple K-nearest neighbor strategy generates hypergraph with a fixed number of vertices in each hyperedge. To solve the above problems, an unsupervised dimensionality reduction method named joint dynamic hypergraph and low-rank embedding (DHLRE) is proposed. Specifically, DHLRE unifies hypergraph learning and low-rank learning into a single objective function. By doing so, both global and higher-order local structural information of the data can be extracted by DHLRE, and the two structural information complement each other to make the projection matrix have stronger discriminative power. In addition, the weight of hyperedge and hypergraph in DHLRE are dynamically generated in dimension-reduced space, which suppresses the ability of noise or outlier to affect the projection matrix. To divide hyperedge in a flexible way, a new approach without neighborhood parameter is proposed. By using this approach, the number of vertices of the hyperedge can be adaptively decided for different data sets. Further, an iterative algorithm is designed to solve the objective function of DHLRE. Finally, experimental results on several real-world data sets show that DHLRE outperforms others related methods for both classification and clustering tasks.

Multiple Graphs and Low-Rank Embedding for Multi-Source Heterogeneous Domain Adaptation

Multi-source domain adaptation is a challenging topic in transfer learning, especially when the data of each domain are represented by different kinds of features, i.e., Multi-source Heterogeneous Domain Adaptation (MHDA). It is important to take advantage of the knowledge extracted from multiple sources as well as bridge the heterogeneous spaces for handling the MHDA paradigm. This article proposes a novel method named Multiple Graphs and Low-rank Embedding (MGLE), which models the local structure information of multiple domains using multiple graphs and learns the low-rank embedding of the target domain. Then, MGLE augments the learned embedding with the original target data. Specifically, we introduce the modules of both domain discrepancy and domain relevance into the multiple graphs and low-rank embedding learning procedure. Subsequently, we develop an iterative optimization algorithm to solve the resulting problem. We evaluate the effectiveness of the proposed method on several real-world datasets. Promising results show that the performance of MGLE is better than that of the baseline methods in terms of several metrics, such as AUC, MAE, accuracy, precision, F1 score, and MCC, demonstrating the effectiveness of the proposed method.

Community Detection in Fully-Connected Multi-layer Networks Through Joint Nonnegative Matrix Factorization

Modern data analysis and processing tasks typically involve large sets of structured data. Graphs provide a powerful tool to describe the structure of such data, where the entities and the relationships between them are modeled as the nodes and edges of the graph. Traditional single layer network models are insufficient for describing the multiple entity types and modes of interaction encountered in real-world applications. Recently, multi-layer network models, which consider the different types of interactions both within and across layers, have emerged to model these systems. One of the important tools in understanding the topology of these high-dimensional networks is community detection. In this paper, a joint nonnegative matrix factorization approach is proposed to detect the community structure in multi-layer networks. The proposed approach models the multi-layer network as the union of a multiplex and bipartite network and formulates community detection as a regularized optimization problem. This optimization problem simultaneously finds the nonnegative low-rank embedding of the intra- and inter-layer adjacency matrices while minimizing the distance between the two to guarantee pair-wise similarity across embeddings. The proposed approach can detect the community structure for both homogeneous and heterogeneous multi-layer networks and is robust to noise and sparsity. The performance of the proposed approach is evaluated for both simulated and real networks and compared to state-of-the-art methods.

Multi-view fuzzy clustering of deep random walk and sparse low-rank embedding

Multi-view clustering aims to improve the learning performance by exploiting discriminative information from heterogeneous data sources. It has been capturing growing research attention due to its wide utilization in the areas of data mining and computer vision. However, the full use of complementarity and consistency from multi-view data is still a challenging task. Here, we propose a novel multi-view fuzzy clustering, which adopts the joint learning of deep random walk and sparse low-rank embedding. First, a deep random walk is employed to acquire a robust similarity matrix of data points and convert fuzzy membership matrix learning to adaptive graph learning. Second, the adaptive graph is restricted with sparse low-rank constraints, which ensures its strong discriminative ability and effective cluster assignments. Third, by exploiting the sparse low-rank property, the multi-view fuzzy clustering problem is formulated as optimizing a regularized graph adjacency matrix with distance metric learning and spectral norm minimization simultaneously. The distance metric learning is embedded to scatter all data points so that any two samples are projected onto a low-dimensional subspace with a large margin. Finally, an efficient algorithm is developed for the formulated problem and its convergence is also guaranteed. In order to demonstrate the superiority of the proposed method, extensive experiments are conducted by comparing state-of-the-arts on real-world databases.

Robust sparse low-rank embedding for image dimension reduction

Many methods based on matrix factorization have recently been proposed and achieve good performance in many practical applications. Latent low-rank representation (LatLRR) is a marvelous feature extraction method, and it has shown a powerful ability in extracting robust data features. However, LatLRR and the variants of LRR have some shortcomings as follows: (1) The label information of the original data are not considered, and they are usually unsupervised learning methods. (2) The local structure information is not preserved in the projected space. (3) The dimension of projection space is not reduced, and the extracted features do not have good and distinct interpretability. In order to solve the above problems, a new dimensionality reduction method based on low-rank representation termed robust sparse low-rank embedding (RSLRE) is proposed. Especially, by introducing the L2,1 norm constraint into the projected matrix, RSLRE algorithm can adaptively select the most discriminative and robust data features. In addition, two different matrices are introduced to ensure that projected feature dimensions can be reduced, and the obtained features can simultaneously maintain most of the energy of the observed samples. A large number of experiments on five public image datasets show that the proposed method can achieve very encouraging results compared with some classical feature extraction methods.

Dynamic coarse‐to‐fine ISAR image blind denoising using active joint prior learning

Most existing nonblind denoising approaches assumed the noise to be homogeneous white Gaussian distribution with known intensity. However, it is difficult to know beforehand or model accurately real-world noises with complex hybrid distribution and noise intensity. In this paper, active joint prior learning (JPL) is proposed for real-world ISAR image blind denoising. (1) To explore strong model hierarchy and components relationship automatically, a novel graphical Dirichlet mixture process (GDMP) model is developed, where the latent representations and component hyperparameters are jointly learned from each other. (2) A multiscale joint learning strategy (MJLS) is proposed to take advantage of both the optimization- and discriminative learning-based capabilities. The external noiseless, internal noisy image information and their relationships are jointly explored simultaneously. (3) Low-rank weighted sparse learning (LWSL) is proposed to learn sparse discriminative correlation components for robust prior learning, and latent low-rank embedding for GDMP patterns self-adaptive inference. Extensive experimental results on ISAR image datasets demonstrate the effectiveness of the proposed model for both synthesis and real-world noisy ISAR images, and the proposed method outperforms the state-of-the-art denoising methods.

Robust Image Feature Extraction via Approximate Orthogonal Low-Rank Embedding

Feature extraction (FE) plays an important role in machine learning. In order to handle the “dimensionality disaster”problem, the usual approach is to transform the original sample into a low-dimensional target space, in which the FE task is performed. However, the data in reality will always be corrupted by various noises or interfered by outliers, making the task of feature extraction extremely challenging. Thence, we propose a novel image FE method via approximate orthogonal low-rank embedding (AOLRE), which adopts an orthogonal matrix to reserve the major energy of the samples, and the introduction of the ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> , <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> -norm makes the features more compact, differentiated and interpretable. In addition, the weighted Schatten p-norm is adopted in this model for fully exploring the effects of different ranks while approaching the original hypothesis of low rank. Meanwhile, as a robust measure, the correntropy is applied in AOLRE. This can effectively suppress the adverse influences of contaminated data and enhance the robustness of the algorithm. Finally, the introduction of the classification loss item allows our model to effectively fit the supervised scene. Five common datasets are used to evaluate the performance of AOLRE. The results show that the recognition accuracy and robustness of AOLRE are significantly better than those of several advanced FE algorithms, and the improvement rate ranges from 2% to 15%.

Neither global nor local: A hierarchical robust subspace clustering for image data

In this study, we consider the problem of subspace clustering in the presence of spatially contiguous noise, occlusion, and disguise. We argue that self-expressive representation of data, which is a key characteristic of current state-of-the-art approaches, is severely sensitive to occlusions and complex real-world noises. To alleviate this problem, we highlight the importance of previously neglected local representations in improving robustness and propose a hierarchical framework that combines the robustness of local-patch-based representations and the discriminative property of global representations. This approach consists of two main steps: 1) A top-down stage, in which the input data are subject to repeated division to smaller patches and 2) a bottom-up stage, in which the low rank embedding of representation matrices of local patches in the field of view of a corresponding patch in the upper level are merged on a Grassmann manifold. This unified approach provides two key pieces of information for neighborhood graph of the corresponding patch on the upper level: cannot-links and recommended-links. This supplies a robust summary of local representations which is further employed for computing self-expressive representations using a novel weighted sparse group lasso optimization problem. Numerical results for several data sets confirm the efficiency of our approach.

Multi-view Locality Low-rank Embedding for Dimension Reduction

During the last decades, we have witnessed a surge of interest in learning a low-dimensional space with discriminative information from one single view. Even though most of them can achieve satisfactory performance in certain situations, they fail to fully consider the information from multiple views which are highly relevant but sometimes look different from each other. Besides, correlations between features from multiple views always vary greatly, which challenges the capability of multi-view subspace learning methods. Therefore, how to learn an appropriate subspace which could maintain valuable information from multi-view features is of vital importance but challenging. To tackle this problem, this paper proposes a novel multi-view dimension reduction method named Multi-view Locality Low-rank Embedding for Dimension Reduction (MvL2E). MvL2E mainly focuses on capturing a common low-dimensional embedding among multiple different views, which makes full use of correlations between multi-view features by adopting low-rank representations. Meanwhile, it aims to maintain the correlations and construct a suitable manifold structure to capture the low-dimensional embedding for multi-view features. A centroid based scheme is designed to get one common low-dimensional manifold space and force multiple views to learn from each other. And an iterative alternating strategy is developed to obtain the optimal solution of MvL2E. The proposed method is evaluated on 5 benchmark datasets. Comprehensive experiments show that our proposed MvL2E can achieve comparable performance with previous approaches proposed in recent works of literature.

Learning Latent Low-Rank and Sparse Embedding for Robust Image Feature Extraction

To defy the curse of dimensionality, the inputs are always projected from the original high-dimensional space into the target low-dimension space for feature extraction. However, due to the existence of noise and outliers, the feature extraction task for corrupted data is still a challenging problem. Recently, a robust method called low rank embedding (LRE) was proposed. Despite the success of LRE in experimental studies, it also has many disadvantages: 1) The learned projection cannot quantitatively interpret the importance of features. 2) LRE does not perform data reconstruction so that the features may not be capable of holding the main energy of the original "clean" data. 3) LRE explicitly transforms error into the target space. 4) LRE is an unsupervised method, which is only suitable for unsupervised scenarios. To address these problems, in this paper, we propose a novel method to exploit the latent discriminative features. In particular, we first utilize an orthogonal matrix to hold the main energy of the original data. Next, we introduce an l2,1 -norm term to encourage the features to be more compact, discriminative and interpretable. Then, we enforce a columnwise l2,1 -norm constraint on an error component to resist noise. Finally, we integrate a classification loss term into the objective function to fit supervised scenarios. Our method performs better than several state-of-the-art methods in terms of effectiveness and robustness, as demonstrated on six publicly available datasets.

Feature Selective Projection with Low-Rank Embedding and Dual Laplacian Regularization

Feature extraction and feature selection have been regarded as two independent dimensionality reduction methods in most of the existing literature. In this paper, we propose to integrate both approaches into a unified framework and design an unsupervised linear feature selective projection (FSP) for feature extraction with low-rank embedding and dual Laplacian regularization, with the aim to exploit the intrinsic relationship among data and suppress the impact of noise. Specifically, a projection matrix with an $l_{2,1}$ l 2 , 1 -norm regularization is introduced to project original high dimensional data points into a new subspace with lower dimension, where the $l_{2,1}$ l 2 , 1 -norm regularization can endow the projection with good interpretability. We deploy a coefficient matrix with low rank constraint to reconstruct the data points and the $l_{2,1}$ l 2 , 1 -norm is imposed to regularize the data reconstruction errors in the low-dimensional subspace and make FSP robust to noise. Furthermore, a dual graph Laplacian regularization term is imposed on the low dimensional data and data reconstruction matrix for preserving the local manifold geometrical structure of data. Finally, an alternatively iterative algorithm is carefully designed for solving the proposed optimization model. Theoretical convergence and computational complexity analysis of the algorithm are also provided. Comprehensive experiments on various benchmark datasets have been carried out to evaluate the performance of the proposed FSP. As indicated, our algorithm significantly outperforms other state-of-the-art methods for feature extraction.

Tensor Low-Rank Discriminant Embedding for Hyperspectral Image Dimensionality Reduction

Recently, low-rank embedding (LRE) has yielded satisfactory results in dimensionality reduction (DR), for which low-rank representation and projection learning are integrated into one model to generate robust low-dimensional features. However, LRE requires to convert samples into vectors even if the data naturally appear in high-order form. Furthermore, LRE fails to take the label information into consideration. To address these problems, this paper proposes a novel supervised DR method based on multilinear algebra, i.e., the algebra of tensors. By the motivation of extending LRE into tensor space and simultaneously combining the tensor discriminant analysis, we establish tensor low-rank discriminant embedding (TLRDE) model for hyperspectral image (HSI) DR. The model of TLRDE is solved by an alternative iteration algorithm, whose convergence is also mathematically proven. The proposed TLRDE method employs the tensor representation to preserve the intrinsic geometrical structure, uses low-rank reconstruction to uncover the potential relationship among the data points, and combines label information to enhance the discriminability of features. Moreover, the proposed TLRDE does not suffer from the small sample size problem. The experimental results on three real HSI data sets validate the effectiveness of our proposed TLRDE method.

Generative Zero-Shot Learning via Low-Rank Embedded Semantic Dictionary.

Zero-shot learning for visual recognition, which approaches identifying unseen categories through a shared visual-semantic function learned on the seen categories and is expected to well adapt to unseen categories, has received considerable research attention most recently. However, the semantic gap between discriminant visual features and their underlying semantics is still the biggest obstacle, because there usually exists domain disparity across the seen and unseen classes. To deal with this challenge, we design two-stage generative adversarial networks to enhance the generalizability of semantic dictionary through low-rank embedding for zero-shot learning. In detail, we formulate a novel framework to simultaneously seek a two-stage generative model and a semantic dictionary to connect visual features with their semantics under a low-rank embedding. Our first-stage generative model is able to augment more semantic features for the unseen classes, which are then used to generate more discriminant visual features in the second stage, to expand the seen visual feature space. Therefore, we will be able to seek a better semantic dictionary to constitute the latent basis for the unseen classes based on the augmented semantic and visual data. Finally, our approach could capture a variety of visual characteristics from seen classes that are "ready-to-use" for new classes. Extensive experiments on four zero-shot benchmarks demonstrate that our proposed algorithm outperforms the state-of-the-art zero-shot algorithms.

Scaling Up Kernel SVM on Limited Resources: A Low-Rank Linearization Approach.

Kernel support vector machines (SVMs) deliver state-of-the-art results in many real-world nonlinear classification problems, but the computational cost can be quite demanding in order to maintain a large number of support vectors. Linear SVM, on the other hand, is highly scalable to large data but only suited for linearly separable problems. In this paper, we propose a novel approach called low-rank linearized SVM to scale up kernel SVM on limited resources. Our approach transforms a nonlinear SVM to a linear one via an approximate empirical kernel map computed from efficient kernel low-rank decompositions. We theoretically analyze the gap between the solutions of the approximate and optimal rank- k kernel map, which in turn provides guidance on the sampling scheme of the Nyström approximation. Furthermore, we extend it to a semisupervised metric learning scenario in which partially labeled samples can be exploited to further improve the quality of the low-rank embedding. Our approach inherits rich representability of kernel SVM and high efficiency of linear SVM. Experimental results demonstrate that our approach is more robust and achieves a better tradeoff between model representability and scalability against state-of-the-art algorithms for large-scale SVMs.

Incorporating network structure with node contents for community detection on large networks using deep learning

Abstract Community detection is an important task in social network analysis. In community detection, in general, there exist two types of the models that utilize either network topology or node contents. Some studies endeavor to incorporate these two types of models under the framework of spectral clustering for a better community detection. However, it was not successful to obtain a big achievement since they used a simple way for the combination. To reach a better community detection, it requires to realize a seamless combination of these two methods. For this purpose, we re-examine the properties of the modularity maximization and normalized-cut models and fund out a certain approach to realize a seamless combination of these two models. These two models seek for a low-rank embedding to represent of the community structure and reconstruct the network topology and node contents, respectively. Meanwhile, we found that autoencoder and spectral clustering have a similar framework in their low- rank matrix reconstruction. Based on this property, we proposed a new approach to seamlessly combine the models of modularity and normalized-cut via the autoencoder. The proposed method also utilized the advantages of the deep structure by means of deep learning. The experiment demonstrated that the proposed method can provide a nonlinearly deep representation for a large-scale network and reached an efficient community detection. The evaluation results showed that our proposed method outperformed the existing leading methods on nine real-world networks.