High-order Tensor Data Research Articles

Accelerating regularized tensor decomposition using the alternating direction method of multipliers with multiple Nesterov’s extrapolations

Tensor decomposition is an essential tool for multiway signal processing. At present, large-scale high-order tensor data require fast and efficient decomposing algorithms. In this paper, we propose accelerated regularized tensor decomposition algorithms using the alternating direction method of multipliers with multiple Nesterov’s extrapolations in the block coordinate descent framework. We implement the acceleration in three cases: only in the inner loop, only in the outer loop, and in both the inner and outer loops. Adaptive safeguard strategies are developed following the acceleration to guarantee monotonic convergence. Afterwards, we utilize the proposed algorithms to accelerate two types of conventional decomposition: nonnegative CANDECOMP/PARAFAC (NCP) and sparse CANDECOMP/PARAFAC (SCP). The experimental results on synthetic and real-world tensors demonstrate that the proposed algorithms achieve significant acceleration effects and outperform state-of-the-art algorithms. The accelerated algorithm with extrapolations in both the inner and outer loops has the fastest convergence speed and takes almost one-third of the running time of typical algorithms.

Multilinear subspace learning for Person Re-Identification based fusion of high order tensor features

Video surveillance image analysis and processing is an important field in computer vision and among the challenging tasks for Person Re-Identification (PRe-ID). The latter aims at finding a target person who has already been identified and appeared on a camera network using a powerful description of their pedestrian images. The success of recent research on person PRe-ID is largely backed into the effective features extraction and representation with a powerful learning of these features to correctly discriminate pedestrian images. To this end, two powerful features, Convolutional Neural Network (CNN) and Local Maximal Occurrence (LOMO) are modeled on a multidimensional data in the proposed method, High-Dimensional Feature Fusion (HDFF). Specifically, a new tensor fusion scheme is introduced to take advantage and combine two types of features in the same tensor data even if its dimensions are not the same. To improve the accuracy, we use Tensor Cross-View Quadratic Analysis (TXQDA) to perform multilinear subspace learning followed by the Cosine similarity for matching. TXQDA efficiently ensures the learning ability and reduces the high dimensionality resulting from high-order tensor data. The effectiveness of the proposed method is verified through experiments on three challenging widely-used PRe-ID datasets namely, VIPeR, GRID, and PRID450S. Extensive experiments show that the proposed method performs very well when compared with recent state-of-the-art methods.

Probabilistic Linear Discriminant Analysis With Vectorial Representation for Tensor Data.

Linear discriminant analysis (LDA) has been a widely used supervised feature extraction and dimension reduction method in pattern recognition and data analysis. However, facing high-order tensor data, the traditional LDA-based methods take two strategies. One is vectorizing original data as the first step. The process of vectorization will destroy the structure of high-order data and result in high dimensionality issue. Another is tensor LDA-based algorithms that extract features from each mode of high-order data and the obtained representations are also high-order tensor. This paper proposes a new probabilistic LDA (PLDA) model for tensorial data, namely, tensor PLDA. In this model, each tensorial data are decomposed into three parts: the shared subspace component, the individual subspace component, and the noise part. Furthermore, the first two parts are modeled by a linear combination of latent tensor bases, and the noise component is assumed to follow a multivariate Gaussian distribution. Model learning is conducted through a Bayesian inference process. To further reduce the total number of model parameters, the tensor bases are assumed to have tensor CandeComp/PARAFAC (CP) decomposition. Two types of experiments, data reconstruction and classification, are conducted to evaluate the performance of the proposed model with the convincing result, which is superior or comparable against the existing LDA-based methods.

Tensor Discriminant Analysis with Partial Label

Small Sample Size (SSS) and overfitting problems will be encountered if using vector space methods by vectorize high order tensor data. In addition, the structure information will be lost, So thensor based methods or multilinear algorithm will be more approriate for this type of data. For medical data expecially electrocardiograms (ECGs), achieving the diagnosed disease label is time sonsuming and expensive. To overcome this type of problem, this paper propose a algorithm for data with partial label, in other words, our algorithm take labeled data and unlabel data as input data, Anotoher issue is the spaseity of effective features. Here we propose a Sparse Semisupervised Sparse Multilinear Discriminant Analysis (SSSMDA) for electrocardiograms (ECGs), our method consider the distirbution of unlabeled and labeled data, we also consider the label achieved by label propagation algorithm. We convert original 12 lead electrocardiograms (ECGs) to 3-order tensor by Short Time Fourier Transformation (STFT). The effectiveness of our approach is proved in the experiment section.

Support high-order tensor data description for outlier detection in high-dimensional big sensor data

The various high-dimensional sensor data can be collected by wireless sensor networks, video monitoring systems and multimedia sensor networks, while High-dimensional sensor data is inherently large-scale because each sensor node has spatial attributes and may also be associated with large amounts of measurement data evolving over time. Detecting outlier in high-dimensional big sensor data is a challenging task. Most of existing outlier detection methods is based on vector representation. However, high-dimensional sensor data is naturally described by tensor representations. The vector-based methods can lead to destroy original structural information and correlation for high-dimensional sensors data, result in the problem of curse of dimensionality, and some outliers cannot be detected. To solve this problem, support high-order tensor data description (STDD) and kernel support high-order tensor data description (KSTDD) are proposed to detect outliers for tensor data. STDD and KSTDD extend support vector data description from vector space to tensor space. KSTDD maintains the structural information of data, avoids the problem caused by the vectorization of tensor data, and improves the performance of outlier detection. Experiments on four sensor datasets show that the proposed method is superior to the traditional vectorized data analysis method.

Multilinear Spatial Discriminant Analysis for Dimensionality Reduction

In the last few years, great efforts have been made to extend the linear projection technique (LPT) for multidimensional data (i.e., tensor), generally referred to as the multilinear projection technique (MPT). The vectorized nature of LPT requires high-dimensional data to be converted into vector, and hence may lose spatial neighborhood information of raw data. MPT well addresses this problem by encoding multidimensional data as general tensors of a second or even higher order. In this paper, we propose a novel multilinear projection technique, called multilinear spatial discriminant analysis (MSDA), to identify the underlying manifold of high-order tensor data. MSDA considers both the nonlocal structure and the local structure of data in the transform domain, seeking to learn the projection matrices from all directions of tensor data that simultaneously maximize the nonlocal structure and minimize the local structure. Different from multilinear principal component analysis (MPCA) that aims to preserve the global structure and tensor locality preserving projection (TLPP) that is in favor of preserving the local structure, MSDA seeks a tradeoff between the nonlocal (global) and local structures so as to drive its discriminant information from the range of the non-local structure and the range of the local structure. This spatial discriminant characteristic makes MSDA have more powerful manifold preserving ability than TLPP and MPCA. Theoretical analysis shows that traditional MPTs, such as multilinear linear discriminant analysis, TLPP, MPCA, and tensor maximum margin criterion, could be derived from the MSDA model by setting different graphs and constraints. Extensive experiments on face databases (ORL, CMU PIE, and the extended Yale-B) and the Weizmann action database demonstrate the effectiveness of the proposed MSDA method.

A novel method for classification of matrix data using Twin Multiple Rank SMMs

It is known that most high-order tensor data are linear inseparable and all of them can be transformed into matrix data through tucker tensor decomposition and the matrices involved are always multiple rank. Hence, how to effectively and efficiently classify matrix data becomes an important research topic. However, up to now most known classifiers for matrix data are linear and a few nonlinear classifiers are only for rank-one matrices. In order to classify multiple rank matrix data, in this paper, a novel supervised classification method called Linear Twin Multiple Rank Support Matrix Machine (LTMRSMM) is developed, which is also an improvement of TSVM and an extension of MRMLSVM. For linear inseparable matrix data, a nonlinear classification method named Nonlinear Twin Multiple Rank Support Matrix Machine (NTMRSMM) is provided by means of a new matrix kernel function introduced recently by authors of the paper. Experimental results show that the presented methods are effective and competitive classification methods for multiple rank matrix data.

Multiple rank multi-linear kernel support vector machine for matrix data classification

High-order tensors especially matrices are one of the common forms of data in real world. How to classify tensor data is an important research topic. We know that all high-order tensor data can be transformed into matrix data through tucker tensor decomposition and most of them are linear inseparable and the matrices involved are multiple rank. However, up to now most known classifiers for matrix data are linear and a few nonlinear classifiers are only for rank-one matrices. In order to classify linear inseparable multiple rank matrix data, in this paper, a novel nonlinear classifier named as multiple rank multi-linear kernel SVM (MRMLKSVM) is proposed, which is also an extension of MRMLSVM and an improvement of NLS-TSTM. For verifying the effectiveness of the proposed method, a series of comparative experiments are performed on four data sets taken from different databases. Experiment results indicate that MRMLKSVM is an effective and efficient nonlinear classifier.

Semisupervised Sparse Multilinear Discriminant Analysis

Various problems are encountered when adopting ordinary vector space algorithms for high-order tensor data input. Namely, one must overcome the Small Sample Size (SSS) and overfitting problems. In addition, the structural information of the original tensor signal is lost during the vectorization process. Therefore, comparable methods using a direct tensor input are more appropriate. In the case of electrocardiograms (ECGs), another problem must be overcome; the manual diagnosis of ECG data is expensive and time consuming, rendering it difficult to acquire data with diagnosis labels. However, when effective features for classification in the original data are very sparse, we propose a semisupervised sparse multilinear discriminant analysis (SSSMDA) method. This method uses the distribution of both the labeled and the unlabeled data together with labels discovered through a label propagation algorithm. In practice, we use 12-lead ECGs collected from a remote diagnosis system and apply a short-time-fourier transformation (STFT) to obtain third-order tensors. The experimental results highlight the sparsity of the ECG data and the ability of our method to extract sparse and effective features that can be used for classification.