Low-rank Transformation Research Articles

Node proximity preserved dynamic network embedding via matrix perturbation

In recent years, network embedding has attracted extensive interests, which aims at representing nodes of an original network in a low-dimensional vector space while preserving the inherent topological structures of the network. Despite the remarkable advantages of complex networks, most existing network embedding methods are mainly focused on static networks while ignoring the evolving characteristic, which is proved to be an essential property of real-world networks. In this paper, we propose Node Proximity Preserved Dynamic Network Embedding via Matrix Perturbation (NPDNE) to tackle the dilemma. Specifically, our method implements a low-rank transformation on the normalized Laplacian matrix of the given networks and then derives the embedding vectors through generalized SVD. Subsequently, the node proximities are preserved in the embedding vectors by exploiting the eigen-decomposition reweighing theorem, which reveals the intrinsic relationship among different-order proximities. Moreover, a generalized eigen perturbation is adopted to update the embedding vectors so that the evolution of given networks can be captured over time. Finally, we conduct experiments of multi-label classification, link prediction, and visualization on several real-world datasets. The experimental results demonstrate the superiority of the proposed NPDNE model compared with state-of-the-art baselines.

Low-Rank RNN Adaptation for Context-Aware Language Modeling

A context-aware language model uses location, user and/or domain metadata (context) to adapt its predictions. In neural language models, context information is typically represented as an embedding and it is given to the RNN as an additional input, which has been shown to be useful in many applications. We introduce a more powerful mechanism for using context to adapt an RNN by letting the context vector control a low-rank transformation of the recurrent layer weight matrix. Experiments show that allowing a greater fraction of the model parameters to be adjusted has benefits in terms of perplexity and classification for several different types of context.

Active learning support vector machines with low-rank transformation

Active learning has proven to be quite effective in a vast array of machine learning tasks. Despite the lower labeling cost of active learning, it has been shown that active learning still can not reach state-of-the-art performance on several classification tasks and is sensitive to initial state. In this work, we propose a novel algorithm to improve the performance of active learning and it’s robustness to initial state. More specifically, we integrate low-rank transformation (LRT) with active learning. In each iteration, LRT is applied to project original high dimensional data to a feature space where data are easier to be classified and then support vector machines classifier is updated in this feature space. As iteration goes on, active learning’s propriety of labeling data improves the performance of LRT, which further promotes the accuracy of SVM classifier. Experiment on several benchmark binary classification datasets results showed the proposed algorithm outperforms other active learning methods in accuracy and robustness.

Sparse subspace clustering with low-rank transformation

In order to solve the problem that sparse subspace clustering cannot effectively cluster the dataset under non-independent assumption, this paper proposes the sparse subspace clustering with low-rank transformation, which merges the low-rank transformation into sparse subspace clustering. The low-rank transformation can not only reduce variations within the subspaces, but also increase separations between the subspaces. This guarantees that property of these datasets is altered from inconsistent independent assumption to asymptotic consistent independent assumption, ultimately to consistent independent assumption. Sparse subspace clustering with low-rank transformation is formulated as the minimization optimizations with two variables, which can be solved by an iterative strategy including the clustering and computing the optimal transformation. Experimental results on the synthetic datasets show that sparse subspace clustering with low-rank transformation is robust to synthetic dataset with noise and its classification error is approximately 75% lower than that of sparse subspace clustering. Experimental results on face clustering on the Yale Extend B dataset and motion segmentation on Hopkins 155 dataset show that the proposed approach significantly outperforms sparse subspace clustering.

Towards effective codebookless model for image classification

The bag-of-features (BoF) model for image classification has been thoroughly studied over the last decade. Different from the widely used BoF methods which model images with a pre-trained codebook, the alternative codebook-free image modeling method, which we call codebookless model (CLM), attracts little attention. In this paper, we present an effective CLM that represents an image with a single Gaussian for classification. By embedding Gaussian manifold into a vector space, we show that the simple incorporation of our CLM into a linear classifier achieves very competitive accuracy compared with state-of-the-art BoF methods (e.g., Fisher Vector). Since our CLM lies in a high-dimensional Riemannian manifold, we further propose a joint learning method of low-rank transformation with support vector machine (SVM) classifier on the Gaussian manifold, in order to reduce computational and storage cost. To study and alleviate the side effect of background clutter on our CLM, we also present a simple yet effective partial background removal method based on saliency detection. Experiments are extensively conducted on eight widely used databases to demonstrate the effectiveness and efficiency of our CLM method.

Method to Facilitate High-Dimensional Design Space Exploration Using Computationally Expensive Analyses

The emergence of adjoint design methods has been a breakthrough in the field of aerodynamic shape optimization. The strength of such methods lies in the fact that the adjoint formulation effectively removes the curse of dimensionality by drastically reducing the cost of computing the gradient. However, the curse of dimensionality is still very much alive when it comes to design space exploration, where gradient-free methods cannot be avoided. In attempt to solve this problem, this work presents a method called latent space gradient transformation, which facilitates gradient-free exploration for cases where function evaluations are expensive and dimensionality is high. The method builds upon previous work by applying principal component analysis to gradient information, something that had not been done before. This yields a low-rank transformation that maps a high-dimensional design space onto an equivalent, low-dimensional space in which gradient-free methods become more affordable. Concepts and limitations are discussed through analytical examples, and the method is tested on an overwing nacelle integration problem. Results showed that the method is cost-effective, requiring only 150 gradient samples to successfully reduce the problem from 145 design variables to less than 20, with an error of 10 drag counts on average and 25 drag counts at worst with 95% confidence.

Learning transformations for clustering and classification

A low-rank transformation learning framework for subspace clustering and classification is proposed here. Many high-dimensional data, such as face images and motion sequences, approximately lie in ...

Adaptive Widely Linear Reduced-Rank Interference Suppression Based on the Multistage Wiener Filter

We propose a widely linear multistage Wiener filter (WL-MSWF) receiver to suppress inter-/intra-symbol interference, multiuser interference, and narrowband interference in a high data rate direct-sequence ultra wideband (DS-UWB) system. The proposed WL receiver fully exploits the second-order statistics of the received signal, yielding a smaller Minimum Mean Square Error (MMSE) than the linear receiver. The WL-MSWF receiver mainly consists of a low-rank transformation and an adaptive reduced-rank after. The rank-reduction is achieved via a transformation matrix. Based on the linear MSWF concept, two constructions of this rank-reduction matrix, namely total WL (TWL) and quasi WL (QWL), are proposed. We develop stochastic gradient (SG) and recursive least squares (RLS) adaptive versions of the proposed TWL/QWL-MSWF and theoretically analyze their convergence behavior. The comparison of the proposed TWL/QWL-MSWF and the existing algorithms is carried out in terms of the computational complexity and the resulting MMSE performance. Extensive simulation results show that the proposed TWL/QWL-MSWF schemes outperform the existing schemes in both convergence and steady-state performance under various conditions.