Tensor Dimensionality Reduction Research Articles

ANTLER: Bayesian Nonlinear Tensor Learning and Modeler for Unstructured, Varying-Size Point Cloud Data

Unstructured point clouds of varying sizes are increasingly acquired in a variety of environments through laser triangulation or Light Detection and Ranging (LiDAR). Predicting a vector response based on unstructured point clouds is a common problem that arises in a wide variety of applications. The current literature relies on several pre-processing steps such as structured subsampling and feature extraction to analyze the point cloud data. Those techniques lead to quantization artifacts and do not consider the relationship between the regression response and the point cloud during pre-processing. Therefore, we propose a general and holistic “Bayesian Nonlinear Tensor Learning and Modeler” (ANTLER) to model the relationship of unstructured, varying-size point cloud data with a vector response. The proposed ANTLER simultaneously optimizes a nonlinear tensor dimensionality reduction and a nonlinear regression model with a 3D point cloud input and a regression response. ANTLER can consider the complex data representation, high-dimensionality, and inconsistent size of the 3D point cloud data. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This paper is motivated by a real-world case study concerning the prediction of the transmission error and eccentricity based on unstructured point clouds of varying sizes in gear manufacturing. In the current state-of-the-art method, those characteristics can only be obtained via expensive and time-consuming Finite Element Analysis (FEA) or test benches. The proposed ANTLER framework can directly link the measurement point clouds with a vector response and serves as a guiding example for the immense potential of the ANTLER.

Tensor factorization-based particle swarm optimization for large-scale many-objective problems

• A tensor factorization-based particle swarm optimization is proposed to directly dived large-scale decision space into small-scale sub-spaces, alleviating the dimensionality cures of large-scale problems. • An circular algorithm of sub-population is designed to asymptotically push the whole population to the true Pareto front. • The proposed tensor factorization-based model can be applied to other large-scale and multi-objective evolutionary algorithms. In this paper, a tensor factorization-based circular optimization algorithm of subpopulations is proposed to solve multi- and many-objective optimization problems with large-scale decision variables. Various intermediated procedures, such as variable analysis, clustering, and grouping, to divide large-scale decision variables are needed by most existing algorithms, seriously affecting the optimization performance. The tensor factorization, which is excellent in decomposition and dimensionality reduction of tensors, is originally introduced to the proposed algorithm to decompose the large-scale decision space into small-scale sub-spaces. Then an evolutionary strategy is designed to optimize these sub-populations by speed-constrained multi-objective particle swarm optimization circularly, searching for ideal solutions in each newly constructed lower-scale sub-space. Finally, tensor factorization is used to reconstruct the whole population for asymptotically optimizing the population by all the subpopulations. Using the proposed algorithm on some well-known multi-objective and many-objective benchmarks with 1000 decision variables, it outperformed other state-of-the-art algorithms both in solution quality and convergence rate. The main weakness of the proposed algorithm is the reconstruction error caused by the tensor factorization, which slows down the convergence rate of optimization. Hence our future work is to construct a constrained objective function with the restricted condition of the sparsity of the tensor, which affects the reconstruction error, and seek the solving algorithm.

Tensor local linear embedding with global subspace projection optimisation

In this paper, a novel tensor dimensionality reduction (TDR) approach is proposed, which maintains the local geometric structure of tensor data by tensor local linear embedding and explores the global feature by optimising global subspace projection. Firstly, we analyse the local linear feature of tensor data for learning the linear separable embedding of the tenor data. Furthermore, a global subspace projection distance minimisation strategy is introduced to extract the global characteristic of the tensor data. The aim of this strategy is to find an optimal low-dimensional subspace for TDR. In particular, two novel TDR algorithms are developed by the ensemble of tensor local feature preservation and global subspace projection distance minimisation, which express the subspace projection optimisation as an iteration optimisation problem and a Rayleigh quotient problem, respectively. The extensive experimental results on tensor data classification and clustering have demonstrated the proposed algorithms performed well.

Dimensionality reduction of tensor data based on local linear embedding and mode product

There are three contributions in this paper. (1) A tensor version of LLE (short for Local Linear Embedding algorithm) is deduced and presented. LLE is the most famous manifold learning algorithm. Since its proposal, various improvements to LLE have kept emerging without interruption. However, all these achievements are only suitable for vector data, not tensor data. The proposed tensor LLE can also be used a bridge for various improvements to LLE to transfer from vector data to tensor data. (2) A framework of tensor dimensionality reduction based on tensor mode product is proposed, in which the mode matrices can be determined according to specific criteria. (3) A novel dimensionality reduction algorithm for tensor data based on LLE and mode product (LLEMP-TDR) is proposed, in which LLE is used as a criterion to determine the mode matrices. Benefiting from local LLE and global mode product, the proposed LLEMP-TDR can preserve both local and global features of high-dimensional tenser data during dimensionality reduction. The experimental results on data clustering and classification tasks demonstrate that our method performs better than 5 other related algorithms published recently in top academic journals.

Dimensionality reduction of tensors based on manifold-regularized tucker decomposition and its iterative solution

Dimensionality reduction of tensors based on manifold-regularized tucker decomposition and its iterative solution

Tensor dimensionality reduction via mode product and HSIC

Tensor dimensionality reduction (TDR) is a hot research topic in machine learning, which learns data representations by preserving the original data structure while avoiding convert samples into vectors and solving the problem of the curse of dimensionality of tensor data. In the work, a novel TDR approach based on mode product and Hilbert–Schmidt Independence criterion (HSIC) is proposed. The contributions of authors' work is described as following: (1) HSIC measures the statistical correlation of two random variables. However, instead of measuring the statistical correlation of two random variables directly, HSIC first transforms the two random variables into two reproducing kernel Hilbert spaces (RKHSs), and then measures the statistical correlation of transformed random variables by using Hilbert–Schmidt operators between the two RKHSs. The exploitation of RKHS increases the flexibility and applicability of HSIC. Although HSIC is widely used in machine learning, the authors have not seen its application to dimensionality reduction (DR)(except for authors' previous work). (2) A novel HSIC-based TDR approach is proposed, which first applies HSIC to capture statistical information of tensor data set for DR. The authors give the mathematical derivation of HSIC for tensor data and establish a framework of TDR based on HSIC, named HSIC-TDR for short, which aims to improve the DR results of tensor by exploring and preserving the statistical information of original data set. (3) Furthermore, to solve the out-of-sample problem, the authors learn an explicit expression between the dimensionality-reduced tensors and the higher-dimensional tensors by introducing mode product to HSIC-TDR. The experimental results between the proposed method and other state-of-the-art algorithm on various datasets demonstrate the well performance of the proposed method.

Short-Term Rental Forecast of Urban Public Bicycle Based on the HOSVD-LSTM Model in Smart City.

As a kind of transportation in a smart city, urban public bicycles have been adopted by major cities and bear the heavy responsibility of the “last mile” of urban public transportation. At present, the main problem of the urban public bicycle system is that it is difficult for users to rent a bike during peak h, and real-time monitoring cannot be solved adequately. Therefore, predicting the demand for bicycles in a certain period and performing redistribution in advance is of great significance for solving the lag of bicycle system scheduling with the help of IoT. Based on the HOSVD-LSTM prediction model, a prediction model of urban public bicycles based on the hybrid model is proposed by transforming the source data (multiple time series) into a high-order tensor time series. Furthermore, it uses the tensor decomposition technology (HOSVD decomposition) to extract new features (kernel tenor) from higher-order tensors. At the same time, these kernel tenors are directly used to train tensor LSTM models to obtain new kernel tenors. The inverse tensor decomposition and high-dimensional, multidimensional, and tensor dimensionality reduction were introduced. The new kernel tenor obtains the predicted value of the source sequence. Then the bicycle rental amount is predicted.

Dimensionality Reduction of Tensors Based on Local Homeomorphism and Global Subspace Projection Distance Minimum

Tensors are powerful tools for representing and processing multidimensional data. In this paper, an algorithm of dimension reduction for multidimensional data is proposed. We consider both the global and local structures of the data to fully extract important features. Multiplying a tensor by a matrix in mode-n can change the size of a certain dimension of the tensor. Therefore, for the global structure, we use a matrix to span a subspace to minimize the distance between the tensor data and its projection on the subspace for each mode. For local structure, we construct a dimensionality reduction algorithm for tensor data based on local homeomorphism to keep the continuous dependency relationship of the original high-dimensional tensor data unchanged after dimensionality reduced in each locality. The proposed algorithm achieves dimensionality reduction by combining the global subspace projection distance minimum and local homeomorphism, which realizes global variance maximization and maintains the local non-linear geometric structure at the same time. The proposed algorithm performs good feasibility while compared with other advanced algorithms in classification and clustering experiments.

FERLrTc: 2D+3D facial expression recognition via low-rank tensor completion

In this paper, a 4D tensor model is firstly constructed to explore efficient structural information and correlations from multi-modal data (both 2D and 3D face data). As the dimensionality of the generated 4D tensor is high, a tensor dimensionality reduction technique is in need. Since many real-world high-order data often reside in a low dimensional subspace, Tucker decomposition as a powerful technique is utilized to capture multilinear low-rank structure and to extract useful information from the generated 4D tensor data. Our goal is to use Tucker decomposition to obtain a set of core tensors with smaller sizes and factor matrices which are projected into the 4D tensor data for classification prediction. To characterize the involved similarities of the 4D tensor, the low-rank and sparse representation is built in terms of the low-rank structure of factor matrices and the sparsity of the core tensor in the Tucker decomposition of the generated 4D tensor. A tensor completion (TC) framework is embedded to recover the missing information in the 4D tensor modeling process. Thus, a novel tensor dimensionality reduction approach for 2D+3D facial expression recognition via low-rank tensor completion (FERLrTC) is proposed to solve the factor matrices in a majorization–minimization manner by using a rank reduction strategy. Numerical experiments are conducted with a full implementation on the BU-3DFE and Bosphorus databases and synthetic data to illustrate the effectiveness of the proposed approach.

Generalized notions of sparsity and restricted isometry property. Part I: a unified framework

AbstractThe restricted isometry property (RIP) is an integral tool in the analysis of various inverse problems with sparsity models. Motivated by the applications of compressed sensing and dimensionality reduction of low-rank tensors, we propose generalized notions of sparsity and provide a unified framework for the corresponding RIP, in particular when combined with isotropic group actions. Our results extend an approach by Rudelson and Vershynin to a much broader context including commutative and non-commutative function spaces. Moreover, our Banach space notion of sparsity applies to affine group actions. The generalized approach in particular applies to high-order tensor products.

Efficient Tensor-Based 2D+3D Face Verification

We propose a novel approach for face verification by encoding 2D and 3D face images as a high order tensor. To perform tensor dimensionality reduction for both the unsupervised and supervised cases, we propose multilinear whitened principal component analysis (MWPCA) and tensor exponential discriminant analysis (TEDA), respectively. MWPCA is utilized to solve the small sample size problem in the high-dimensional space and to improve the discrimination power achieved by classical MPCA. In the supervised case, we extend multilinear discriminant analysis to TEDA in order to emphasize the discriminant data included in the null space of the within-class scatter matrix of each tensor’s mode. Additionally, TEDA enlarges the margin between samples belonging to different classes via distance diffusion mappings. Our proposed approach can be seen as a novel data fusion method based on tensor representation. Indeed, the histograms of different local descriptors extracted from both 2D and 3D face modalities are combined through different tensor modes. The extensive experimental evaluation carried out on FRGC v2.0, Bosphorus, and CASIA 2D and 3D face databases indicates that the proposed approach performs significantly better than the state-of-the-art approaches.

Ranking Tensor Subspaces in Weighted Multilinear Principal Component Analysis

Multilinear principal component analysis (MPCA) has been applied for tensor decomposition and dimensionality reduction in image databases modeled through higher order tensors. Despite the well-known attractive properties of MPCA, the traditional approach does not incorporate prior information in order to steer its subspace computation. In this paper, we propose a method to explicitly incorporate such semantics in the MPCA framework to allow an automatic selective treatment of the variables that compose the patterns of interest. The method relies on spatial weights calculated, in this work, by separating hyperplanes and Fisher criterion. In this way, we can perform feature extraction and dimensionality reduction taking advantage of high level information in the form of labeled data. Besides, the corresponding tensor components are ranked in order to identify the principal weighted tensor subspaces for classification tasks. In the computational results we consider gender and facial expression experiments to illustrate the capabilities of the method for dimensionality reduction, classification and reconstruction of face images.

Large Margin Low Rank Tensor Analysis

We present a supervised model for tensor dimensionality reduction, which is called large margin low rank tensor analysis (LMLRTA). In contrast to traditional vector representation-based dimensionality reduction methods, LMLRTA can take any order of tensors as input. And unlike previous tensor dimensionality reduction methods, which can learn only the low-dimensional embeddings with a priori specified dimensionality, LMLRTA can automatically and jointly learn the dimensionality and the low-dimensional representations from data. Moreover, LMLRTA delivers low rank projection matrices, while it encourages data of the same class to be close and of different classes to be separated by a large margin of distance in the low-dimensional tensor space. LMLRTA can be optimized using an iterative fixed-point continuation algorithm, which is guaranteed to converge to a local optimal solution of the optimization problem. We evaluate LMLRTA on an object recognition application, where the data are represented as 2D tensors, and a face recognition application, where the data are represented as 3D tensors. Experimental results show the superiority of LMLRTA over state-of-the-art approaches.

Dimensionality reduction and topographic mapping of binary tensors

In this paper, a decomposition method for binary tensors, generalized multi-linear model for principal component analysis (GMLPCA) is proposed. To the best of our knowledge at present there is no other principled systematic framework for decomposition or topographic mapping of binary tensors. In the model formulation, we constrain the natural parameters of the Bernoulli distributions for each tensor element to lie in a sub-space spanned by a reduced set of basis (principal) tensors. We evaluate and compare the proposed GMLPCA technique with existing real-valued tensor decomposition methods in two scenarios: (1) in a series of controlled experiments involving synthetic data; (2) on a real-world biological dataset of DNA sub-sequences from different functional regions, with sequences represented by binary tensors. The experiments suggest that the GMLPCA model is better suited for modelling binary tensors than its real-valued counterparts. Furthermore, we extended our GMLPCA model to the semi-supervised setting by forcing the model to search for a natural parameter subspace that represents a user-specified compromise between the modelling quality and the degree of class separation.

Semi-supervised Tensor Graph-optimized Linear Discriminant Analysis for Two-dimensional Face Recognition

A new semi-supervised dimensionality reduction method called Semi-supervised Tensor Graph-optimized Linear Discriminant Analysis (STGLDA) is proposed for two-dimensional face recognition. Unlike recent proposed Tensor Locally Linear Discriminative Analysis (TLLDA), STGLDA offers some advantages over TLLDA. 1) STGLDA is originated from Graph-based Fisher Analysis (GbFA) while TLLDA is originated from the Local Fisher Discriminant Analysis (LFDA). In contrast to LFDA, GbFA encodes the richer discriminating information and there is no assumption that data obeys Gaussian distribution. 2) With the linear weighted way of fusing Principal Component Analysis (PCA) and GbFA, STGLDA preserves global scatter structure information and Graph-based discriminant information capturing local structure feature. TLLDA only pays attentions to preserve local structure information and discriminant information while ignores preserving global structure information. Experimental results on real face databases demonstrate that STGLDA is highly competitive with TLLDA and other tensor dimensionality reduction algorithm.

Orthogonal tensor rank one differential graph preserving projections with its application to facial expression recognition

In this paper, a new tensor dimensionality reduction algorithm is proposed based on graph preserving criterion and tensor rank-one projections. In the algorithm, a novel, effective and converged orthogonalization process is given based on a differential-form objective function. A set of orthogonal rank-one basis tensors are obtained to preserve the intra-class local manifolds and enhance the inter-class margins. The algorithm is evaluated by applying to the basic facial expressions recognition.

Orthogonal Tensor Neighborhood Preserving Embedding for facial expression recognition

In this paper a generalized tensor subspace model is concluded from the existing tensor dimensionality reduction algorithms. With this model, we investigate the orthogonality of the bases of the high-order tensor subspace, and propose the Orthogonal Tensor Neighborhood Preserving Embedding (OTNPE) algorithm. We evaluate the algorithm by applying it to facial expression recognition, where both the 2nd-order gray-level raw pixels and the encoded 3rd-order tensor-formed Gabor features of facial expression images are utilized. The experiments show the excellent performance of our algorithm for the dimensionality reduction of the tensor-formed data especially when they lie on some smooth and compact manifold embedded in the high dimensional tensor space.