Higher Order Singular Value Decomposition Research Articles

Approximate Subspace-Based Iterative Adaptive Approach for Fast Two-Dimensional Spectral Estimation

In this paper, we devise a new approach for fast implementation of two-dimensional (2-D) iterative adaptive approach (IAA) using single or multiple snapshots. Our underlying idea is to apply the subspace methodology in this nonparametric technique by performing the IAA on the dominant singular vectors extracted from the singular value decomposition (SVD) or higher-order SVD of the multidimensional observations. In doing so, 2-D IAA is approximately realized by multiple steps of 1-D IAA, implying that computational attractiveness is achieved particularly for large data size, number of grid points and/or snapshot number. Algorithms based on matrix and tensor operations are developed, and their implementation complexities are analyzed. Computer simulations are also included to compare the proposed approach with the state-of-the-art techniques in terms of resolution probability, spectral estimation performance and computational requirement.

Modeling individual HRTF tensor using high-order partial least squares

A tensor is used to describe head-related transfer functions (HRTFs) depending on frequencies, sound directions, and anthropometric parameters. It keeps the multi-dimensional structure of measured HRTFs. To construct a multi-linear HRTF personalization model, an individual core tensor is extracted from the original HRTFs using high-order singular value decomposition (HOSVD). The individual core tensor in lower-dimensional space acts as the output of the multi-linear model. Some key anthropometric parameters as the inputs of the model are selected by Laplacian scores and correlation analyses between all the measured parameters and the individual core tensor. Then, the multi-linear regression model is constructed by high-order partial least squares (HOPLS), aiming to seek a joint subspace approximation for both the selected parameters and the individual core tensor. The numbers of latent variables and loadings are used to control the complexity of the model and prevent overfitting feasibly. Compared with the partial least squares regression (PLSR) method, objective simulations demonstrate the better performance for predicting individual HRTFs especially for the sound directions ipsilateral to the concerned ear. The subjective listening tests show that the predicted individual HRTFs are approximate to the measured HRTFs for the sound localization.

Revisiting the TP Model Transformation: Interpolation and Rule Reduction

AbstractThe tensor‐product (TP) model transformation is a numerical technique that finds a convex representation, akin to a Takagi‐Sugeno (TS) fuzzy model, from a given linear parameter varying (LPV) model of a system. It samples the LPV model over a limited domain, which allows the use of the higher order singular value decomposition (HOSVD) and convex transformations that leads to the TS representation of the LPV model. In this paper, we discuss different strategies that could be used on the sampling step of the TP model transformation (which in turn lead to different membership function properties of a TS fuzzy model). Additionally, this paper discusses how the other steps could be used to reduce the number of rules of a given TS fuzzy model. In cases where nonzero singular values were discarded in the rule reduction, we also show how to obtain an uncertain model that covers the original.

Tensor Renormalization of Quantum Many-Body Systems Using Projected Entangled Simplex States

We propose a new class of tensor-network states, which we name projected entangled simplex states (PESS), for studying the ground-state properties of quantum lattice models. These states extend the pair-correlation basis of projected entangled pair states (PEPS) to a simplex. PESS are an exact representation of the simplex solid states and provide an efficient trial wave function that satisfies the area law of entanglement entropy. We introduce a simple update method for evaluating the PESS wave function based on imaginary-time evolution and the higher-order singular-value decomposition of tensors. By applying this method to the spin-1/2 antiferromagnetic Heisenberg model on the kagome lattice, we obtain accurate and systematic results for the ground-state energy, which approach the lowest upper bounds yet estimated for this quantity.

A Tensor-Based Subspace Approach for Bistatic MIMO Radar in Spatial Colored Noise

In this paper, a new tensor-based subspace approach is proposed to estimate the direction of departure (DOD) and the direction of arrival (DOA) for bistatic multiple-input multiple-output (MIMO) radar in the presence of spatial colored noise. Firstly, the received signals can be packed into a third-order measurement tensor by exploiting the inherent structure of the matched filter. Then, the measurement tensor can be divided into two sub-tensors, and a cross-covariance tensor is formulated to eliminate the spatial colored noise. Finally, the signal subspace is constructed by utilizing the higher-order singular value decomposition (HOSVD) of the cross-covariance tensor, and the DOD and DOA can be obtained through the estimation of signal parameters via rotational invariance technique (ESPRIT) algorithm, which are paired automatically. Since the multidimensional inherent structure and the cross-covariance tensor technique are used, the proposed method provides better angle estimation performance than Chen's method, the ESPRIT algorithm and the multi-SVD method. Simulation results confirm the effectiveness and the advantage of the proposed method.

Doubling of entanglement spectrum in tensor renormalization group

We investigate the entanglement spectrum in HOTRG ---tensor renormalization group (RG) method combined with the higher order singular value decomposition--- for two-dimensional (2D) classical vertex models. In the off-critical region, it is explained that the entanglement spectrum associated with the RG transformation is described by `doubling' of the spectrum of a corner transfer matrix. We then demonstrate that the doubling actually occurs for the square-lattice Ising model by HOTRG calculations up to $D = 64$, where $D$ is the cut-off dimension of tensors. At the critical point, we also find that a non-trivial $D$ scaling behavior appears in the entanglement entropy. We mention about the HOTRG for the 1D quantum system as well.

Direction of Arrival Estimation Based on a Tensor Approach

In this letter, we put forward a novel tensor-based Multiple Signal Classification (TB-MUSIC) applicable to a vector hydrophone array. For this purpose, the signal subspace is derived from the higher order singular value decomposition (HOSVD) of the third order tensor of the output model. Then the proposed method is achieved by signal subspace projection weighted with the reciprocal of principal singular values multiplying by the spatial spectrum based on TB-MUSIC. The synthetic spatial spectrum shows higher resolution and robustness under no-ideal scenarios. Monte Carlo experimental results are provided to illustrate the better performance of the proposed method.

Multilinear Graph Embedding: Representation and Regularization for Images

Given a set of images, finding a compact and discriminative representation is still a big challenge especially when multiple latent factors are hidden in the way of data generation. To represent multifactor images, although multilinear models are widely used to parameterize the data, most methods are based on high-order singular value decomposition (HOSVD), which preserves global statistics but interprets local variations inadequately. To this end, we propose a novel method, called multilinear graph embedding (MGE), as well as its kernelization MKGE to leverage the manifold learning techniques into multilinear models. Our method theoretically links the linear, nonlinear, and multilinear dimensionality reduction. We also show that the supervised MGE encodes informative image priors for image regularization, provided that an image is represented as a high-order tensor. From our experiments on face and gait recognition, the superior performance demonstrates that MGE better represents multifactor images than classic methods, including HOSVD and its variants. In addition, the significant improvement in image (or tensor) completion validates the potential of MGE for image regularization.

Tensor renormalization group study of classicalXYmodel on the square lattice

Using the tensor renormalization group method based on the higher-order singular value decomposition, we have studied the thermodynamic properties of the continuous XY model on the square lattice. The temperature dependence of the free energy, the internal energy, and the specific heat agree with the Monte Carlo calculations. From the field dependence of the magnetic susceptibility, we find the Kosterlitz-Thouless transition temperature to be 0.8921(19), consistent with the Monte Carlo as well as the high temperature series expansion results. At the transition temperature, the critical exponent δ is estimated as 14.5, close to the analytic value by Kosterlitz.

Multi-criteria collaborative filtering with high accuracy using higher order singular value decomposition and Neuro-Fuzzy system

Collaborative Filtering (CF) is the most widely used prediction technique in recommender systems. It makes recommendations based on ratings that users have assigned to items. Most of the current CF recommender systems maintain only single user ratings inside the user-item ratings matrix. Multi-criteria based CF presents a possibility of providing accurate recommendations by considering the user preferences in multi aspects of items. However, in the multi-criteria CF, the user behavior about items’ features is frequently subjective, imprecise and vague. These in turn induce uncertainty in reasoning and representation of items’ features that exactly cannot be solved using crisp machine learning techniques. In contrast, approaches such as fuzzy methods instead of crisp methods can better solve the issue of uncertainty. In addition, fuzzy methods can predict the users’ preference more accurately and even better alleviate the sparsity problem in overall rating by considering user perception about items’ features. Apart from this, in the multi-criteria CF, users provide the ratings on different aspects (criteria) of an item in new dimensions; thereby, increasing the scalability problem. Appropriate dimensionality reduction techniques are thus needed to capture the high dimensions all together without reducing them into lower dimensions to reveal the latent associations among the components. This study presents a new model for multi-criteria CF using Adaptive Neuro-Fuzzy Inference System (ANFIS) combined with subtractive clustering and Higher Order Singular Value Decomposition (HOSVD). HOSVD is used for dimensionality reduction for improving the scalability problem and ANFIS is used for extracting fuzzy rules from the experimental dataset, alleviating the sparsity problems in overall ratings and representing and reasoning the users’ behavior on items’ features. Experimental results on real-world dataset show that combination of two techniques remarkably improves the predictive accuracy and recommendation quality of multi-criteria CF.

An Efficient Dimensionality Reduction Approach for Small-sample Size and High-dimensional Data Modeling

As for massive multidimensional data are being generated in a wide range of emerging applications, this paper introduces two new methods of dimension reduction to conduct small-sample size and high-dimensional data processing and modeling. Through combining the support vector machine (SVM) and recursive feature elimination (RFE), SVM-RFE algorithm is proposed to select features, and further, adding the higher order singular value decomposition (HOSVD) to the feature extraction which involves successfully organizing the data into high order tensor pattern. The validation of simulation experiment data shows that the proposed novel feature selection and feature extraction methods can be effectively applied to the research work for analyzing and modeling the data of atmospheric corrosion. The feature selection method pledges that the remaining feature subset is optimal; feature extraction method reserves the original structure, discriminate information, and the integrity of data, etc. Finally, this paper proposes a complete data dimensionality reduction solution that can effectively solve the high-dimensional small sample data problem, and code programming for this solution has been implemented.

A Small Target Detection Method for the Hyperspectral Image Based on Higher Order Singular Value Decomposition (HOSVD)

This letter proposes a small target detection method for the hyperspectral image based on higher order statistics. This method first calculates the coskewness tensor of the hyperspectral image, followed by the orthogonal decomposition using higher order singular value decomposition. The obtained singular vectors are then used to perform the orthogonal transform to the centralized image. Compared to the popular blind source separation techniques, the presented method keeps clear of nonconvergence. Experiments with a real hyperspectral image show that the interested small target will be presented in the first few bands (even in the first band) very clearly after the transformation.

Calculation of Fourth-Order Tensor Product Expansion by Power Method and Comparison of it with Higher-Order Singular Value Decomposition

Higher-order singular value decomposition (HOSVD) is known as an effective technique to reduce the dimension of multidimensional data. We have proposed a method to perform third-order tensor product expansion (3OTPE) by using the power method for the same purpose as HOSVD, and showed that our method had a better accuracy property than HOSVD, and furthermore, required fewer computation time than that. Since our method could not be applied to the tensors of fourth-order (or more) in spite of having those useful properties, we extend our algorithm of 3OTPE calculation to forth-order tensors in this paper. The results of newly developed method are compared to those obtained by HOSVD. We show that the results follow the same trend as the case of 3OTPE.

Calculation of Fourth-Order Tensor Product Expansion by Power Method and Comparison of it with Higher-Order Singular Value Decomposition

Higher-order singular value decomposition (HOSVD) is known as an effective technique to reduce the dimension of multidimensional data. We have proposed a method to perform third-order tensor product expansion (3OTPE) by using the power method for the same purpose as HOSVD, and showed that our method had a better accuracy property than HOSVD, and furthermore, required fewer computation time than that. Since our method could not be applied to the tensors of fourth-order (or more) in spite of having those useful properties, we extend our algorithm of 3OTPE calculation to forth-order tensors in this paper. The results of newly developed method are compared to those obtained by HOSVD. We show that the results follow the same trend as the case of 3OTPE.

Robust HOSVD-Based Higher-Order Data Indexing and Retrieval

Higher-order singular value decomposition (HOSVD), a natural multilinear extension of the matrix SVD, computes the orthonormal spaces associated with different modes of the tensor. It is widely employed for feature extraction, dimensionality reduction etc. However, due to the vast quantities of tensor entries involved in calculation, it inevitably suffers from high computational cost, especially when recalculation of HOSVD is frequently required. To address the problem, we prove theoretically that the set of HOSVD unitary matrices of a sub-tensor is equivalent to the corresponding subset of HOSVD unitary matrices of the original tensor. Therefore, if we first arrange all tensors in the database compactly as a higher-order tensor, then we only need to conduct HOSVD once on the total tensor. We subsequently propose a robust HOSVD-based multilinear approach for efficiently indexing and retrieving multifactor data, in responding to various query structures. We also apply the proposed method for indexing and retrieval of multi-camera multi-object motion trajectory. Simulation results demonstrate the superior performance of the proposed approach in terms of both robustness and efficiency.

Surrogate model for viscous drag in aircraft empennage conceptual design

A surrogate model is presented to predict viscous drag in the context of conceptual design activities in aeronautics, where the norm is the use of low fidelity methods. The surrogate is developed with the requirement that its computational cost has to be comparable to that of its lower fidelity counterparts. The specific problem that has been chosen to test the method capabilities is the optimization of an unconventional aircraft empennage. The surrogate model is based on the high order singular value decomposition of a multidimensional tensor that contains a limited amount of information on viscous drag as a function of various design parameters defining the configuration of the empennage. This limited amount of information is generated using a medium level fidelity method, namely a panel method plus a boundary layer formulation, but the surrogate only uses the drag information and is independent of both the method used to generate this information and the specific problem that is considered for illustration. Thus, the proposed surrogate model is generic and can be seen as a means to bring information from higher fidelity models into the lower fidelity methods that are commonly used in the conceptual design phases of aeronautic design.

Dimensionality Reduction of Dynamic Mesh Animations Using HO-SVD

Abstract This work presents an analysis of Higher Order Singular Value Decomposition (HOSVD) applied to reduction of dimensionality of 3D mesh animations. Compression error is measured using three metrics (MSE, Hausdorff, MSDM). Results are compared with a method based on Principal Component Analysis (PCA) and presented on a set of animations with typical mesh deformations.

Hybrid computer vision system for drivers' eye recognition and fatigue monitoring

This paper presents a hybrid visual system for monitoring driver's states of fatigue, sleepiness and inattention based on driver's eye recognition. Safe operation in car conditions and processing in daily and night conditions are obtained thanks to the custom setup of two cameras operating in the visible and near infra-red spectra, respectively. In each of these spectra image processing is performed by a cascade of two classifiers. The first classifier in a cascade is responsible for detection of eye regions based on the proposed eye models specific to each spectrum. The second classifier in each cascade is responsible for eye verification. It is based on the higher order singular value decomposition of the tensors of geometrically deformed versions of real eye prototypes, specific to the visible and NIR spectra. Experiments were performed in real car conditions in which four volunteer drivers participated. The obtained results show high recognition accuracy and real-time processing in software implementation. Thanks to these the system can become a part of the advanced driver’s assisting system.

Tensor Approach for Eigenvector-Based Multi-Dimensional Harmonic Retrieval

In this paper, we propose an eigenvector-based frequency estimator for R-dimensional (R-D) sinusoids with R ≥ 2 in additive white Gaussian noise. Our underlying idea is to utilize the tensorial structure of the received data and then apply higher-order singular value decomposition (HOSVD) and structure least squares (SLS) to perform estimation. After obtaining the tensor-based signal subspace from HOSVD, we decompose it into a set of single-tone tensors from which single-tone vectors can be constructed by another HOSVD. In doing so, the R-D multiple sinusoids are converted to a set of single-tone sequences whose frequencies are individually estimated according to SLS. The mean and variance of the frequency estimator are also derived. Computer simulations are also included to compare the proposed approach with conventional R -D harmonic retrieval schemes in terms of mean square error performance and computational complexity particularly in the presence of identical frequencies.

Partial Order in Potts Models on the Generalized Decorated Square Lattice

We explore the Potts model on the generalized decorated square lattice, with both nearest (J1) and next-nearest (J2) neighbor interactions. Using the tensor renormalization-group method augmented by higher order singular value decompositions, we calculate the spontaneous magnetization of the Potts model with q = 2, 3, and 4. The results for q = 2 allow us to benchmark our numerics using the exact solution. For q = 3, we find a highly degenerate ground state with partial order on a single sublattice, but with vanishing entropy per site, and we obtain the phase diagram as a function of the ratio J2/J1. There is no finite-temperature transition for the q = 4 case when J1 = J2, whereas the magnetic susceptibility diverges as the temperature goes to zero, showing that the model is critical at T = 0.