Higher-order Tensors Research Articles

Tensor slow feature analysis and its applications for batch process monitoring

How to establish efficient monitoring models for complex batch processes is a challenging problem because of the three-dimensional (3-D) data array and serious dynamics. The unfolding operation in most methods would destroy the batch data structure and increase the dimensions of modeling data obviously. Meanwhile, it is more practical to perform within-batch detection rather than end-of-batch detection. In this work, a novel high-order tensor slow feature analysis model is proposed to handle the 3-D and dynamical issues simultaneously. The proposed model can be solved by iteratively tackling two sub-optimal problems. Within-batch detection is performed based on the properly defined monitoring statistics to timely detect the abnormal situation. The multi-phase property is further considered in this work to improve detection accuracy. A simulated case and an industrial case are taken to show the merits of the proposed method.

Rank-Adaptive Time Integration of Tree Tensor Networks

A rank-adaptive integrator for the approximate solution of high-order tensor differential equations by tree tensor networks is proposed and analyzed. In a recursion from the leaves to the root, the integrator updates bases and then evolves connection tensors by a Galerkin method in the augmented subspace spanned by the new and old bases. This is followed by rank truncation within a specified error tolerance. The memory requirements are linear in the order of the tensor and linear in the maximal mode dimension. The integrator is robust to small singular values of matricizations of the connection tensors. Up to the rank truncation error, which is controlled by the given error tolerance, the integrator preserves norm and energy for Schrödinger equations, and it dissipates the energy in gradient systems. Numerical experiments with a basic quantum spin system illustrate the behavior of the proposed algorithm.

Imputed quantile tensor regression for near-sited spatial-temporal data

Modern spatial temporal data are collected from sensor networks. Missing data problems are common for this kind of data. Making robust and accurate imputation is important in many applications. There are complex correlations in both spatial and temporal dimensions. Thus, it is even a challenge to model missing spatial-temporal data. In this article, the imputation of missing values is with the help of related covariates. First, we transform the original sensor × time observational matrix to a high order tensor by adding an extra temporal dimension. Then we integrate quantile tensor regression with tensor completion. The objective function includes check loss and nuclear norm penalty. An alternating update algorithm combined with alternating direction method of multipliers (ADMM) is developed to solve the objective function. Theoretical properties of the proposed estimator are investigated. Simulation studies show our proposed method is more robust and can get more accurate imputation results. Real data analysis about Beijing's PM2.5 concentration level is conducted to verify the efficiency of the estimation procedure.

Imputed mean tensor regression for near-sited spatial temporal data

Modern spatial temporal data are often collected from sensor networks. Missing data problems are common to this kind of data. Making accurate imputation is important for many applications. In the unsupervised setting, one technique is to minimize the rank of a tensor or matrix. If we add related covariates, can we get more accurate imputation results? To address this, we transform the original sensor×time measurements to high order tensors by adding additional temporal dimensions and then integrate tensor regression with tensor completion using nuclear norm penalty. One advantage is we can simultaneously estimate parameters and impute missing values due to clear spatial consistency for near-sited spatial-temporal data. The proposed method doesn't assume missing mechanism of the response. Theoretical properties of the proposed estimator are investigated. Simulation studies and real data analysis are conducted to verify the efficiency of the estimation procedure.

Improved local truncation schemes for the higher-order tensor renormalization group method

The higher-order tensor renormalization group is a tensor-network method providing estimates for the partition function and thermodynamical observables of classical and quantum systems in thermal equilibrium. At every step of the iterative blocking procedure, the coarse-grid tensor is truncated to keep the tensor dimension under control. For a consistent tensor blocking procedure, it is crucial that the forward and backward tensor modes are projected on the same lower-dimensional subspaces. In this paper we present two methods, the SuperQ and the iterative SuperQ method, to construct tensor truncations that reduce or even minimize the local approximation errors, while satisfying this constraint.

Adaptive tensor networks decomposition for high-order tensor recovery and compression

Tensor recovery and tensor compression, mainly relying on tensor decomposition (TD) techniques, find extensive applications in many visual applications. However, existing TD models neglect the inherent correlation within data modes, and there are problems with rank determination and latent factors arrangement. In this paper, we propose a data-adaptive TD model called adaptive tensor networks (ATN) decomposition, which constructs an optimal topological structure for TD according to the intrinsic properties of the data. Specifically, we leverage a generalized tensor rank to measure the correlation between two data modes, and then establish a multilinear connection among the corresponding latent factors with an adaptive rank. Moreover, ATN possesses the merits of permutation invariance, strong robustness, and less storage cost for representing high-order data. Beyond the above, we propose a unifying structured Schatten norm to bypass the poor local minima of TD-based applications and develop a scalable algorithm with a convergence guarantee that builds upon the linearized alternating direction method (LADM) framework. The effectiveness and superiority of ATN are verified sufficiently on four typical tasks: tensor completion, image denoising, neural network compression, and infrared small target detection. Experiments on synthetic and real datasets show that ATN outperforms state-of-the-art TD methods.

Tensor Ring decomposition for context-aware recommendation

As a powerful tool for processing high-dimensional data, tensor decomposition has been widely used in context-aware recommendation. Most of the current tensor decomposition-based recommendation models use CP decomposition or Tucker decomposition. In practical recommendation applications, fewer parameters limit the performance of CP decomposition, and the high-order tensor kernel makes the computational complexity of Tucker decomposition exponential. In order to improve the performance of recommendation while maintaining high computational efficiency, we propose a bias Tensor Ring decomposition framework for context-aware recommendation, which employs Tensor Ring decomposition to extract user-item-context latent factors. Comparing to current tensor decomposition-based recommendation models, our framework achieves a better balance between recommendation performance and computational complexity by Tensor Ring decomposition. This is a highly scalable recommendation framework that can easily further integrate additional information such as social trust and implicit feedback, thus used for more complex recommendation tasks. To the best of our knowledge, this work is the first to formulate context-aware recommendation as Tensor Ring decomposition process. Our in-depth analyses confirm that Tensor Ring decomposition has certain advantages over CP decomposition and Tucker decomposition. Multiple experiments on three real datasets also show that the proposed framework helps to improve the recommendation performance.

Hyperspectral image restoration via hybrid smoothness regularized auto-weighted low-rank tensor ring factorization

Restoration of hyperspectral images (HSI) is a crucial step in many potential applications as a preprocessing step. Recently, low-rank tensor ring factorization was applied for HSI reconstruction, which has high-order tensors’ powerful and generalized representation ability. Although low-rank TR-based approaches with nuclear norm regularization achieved successful results for restoring hyperspectral images, there is still room for improved tensor low-rank approximation. In this article, we propose a novel Auto-weighted low-rank Tensor Ring Factorization with Hybrid Smoothness regularization (ATRFHS) for mixed noise removal in HSI. Nonlocal Cuboid Tensorization (NCT) is leveraged to transform HSI data into high-order tensors. TR factorization using latent factors rank minimization removes the mixed noise in HSI data. To highlight nuclear norms of factor tensors differently effective, an auto-weighted strategy is employed to reduce the more prominent factors while shrinking the smaller ones. A hybrid regularization combining total variation (TV) and phase congruency (PC) is incorporated into a low-rank tensor ring factorization model for the HSI noise removal problem. This efficient combination yields sharper edge preservation and resolves this weakness of existing pure TV regularization. Moreover, we develop an efficient algorithm for solving the resulting optimization problem using the framework of alternating minimization. Extensive experimental results demonstrate that our proposed method can significantly outperform existing approaches for mixed noise removal in HSI. The proposed algorithm is validated on synthetic and natural HSI data.

An ESPRIT-Based Supervised Channel Estimation Method Using Tensor Train Decomposition for mmWave 3-D MIMO-OFDM Systems

This paper focuses on the downlink supervised channel estimation problem for the millimeter wave threedimensional multiple-input multiple-output orthogonal frequency division multiplexing (mmWave 3D MIMO-OFDM) systems, where both the transmitter and the receiver are equipped with uniform rectangular arrays (URAs). Based on the sparse scattering nature, the mmWave channel is modeled as a low-rank higher-order tensor. By formulating the channel and the received training signal as tensors, a fast ESPRIT-based Vandermondestructured tensor decomposition method is proposed to estimate the channel parameters involving angles of arrival and departure (AoAs/AoDs), delays and path gains. In specific, we first establish the relationship of the tensor ranks between the tensor train (TT) model and the higher-order tensor CANDECOMP/PARAFAC (CP) signal model. Then the TT decomposition is used to estimate the signal subspace and derive the shift invariance equation, which exploits the higher-order tensor low-rankness of the signal and the Vandermonde structure in the frequency domain. Based on the derived analytical estimation errors, theoretical justifications are provided to unveil the advantage of using TT decomposition. Moreover, we extend the proposed method in an iterative manner to pursue higher accuracy. According to our simulation results, compared with the state-of-the-art, the proposed method achieves higher estimation accuracy on both the channel parameters and the entire channel. In addition, up to 87% of computing time can be saved in comparison to the current best iterative algorithm in the experiments.

Higher-Order Tensor Independent Component Analysis for MIMO Remote Sensing of Respiration and Heartbeat Signals

This paper proposes a novel method of independent component analysis (ICA), which we name higher-order tensor ICA (HOT-ICA). We newly develop a robust microwave multiple-input multiple-output (MIMO) radar system, in which HOT-ICA performs separation of multiple-target signals to detect respiration and heartbeat. In comparison with millimeter waves, microwaves spread wider with diffraction and propagate even in an environment with obstacles to reach targets. However, it often requires more powerful signal separation because of its lower resolution. HOT-ICA realizes high robustness in self-organization of a separation tensor by utilizing channel information, i.e., the information of physical-measurement circumstances concerning, e.g., which transmitting/receiving antennas are used. In numerical and living-human experiments, our HOT-ICA system effectively separates the bio-signals successfully even in an obstacle-affecting environment, which has been a difficult task. The results demonstrate the significance of HOT-ICA in remote sensing. It fully utilizes the high dimensionality of the separation tensor by keeping the tensor structure unchanged to take advantage of the measurement-circumstances information.

Tensor Singular Spectrum Decomposition: Multisensor Denoising Algorithm and Application

Realising multi-sensor signal fusion and weak feature adaptive extraction is a challenging task. Therefore, a new algorithm called tensor singular spectrum decomposition is proposed in this study for the adaptive decomposition of multi-sensor time series. Traditional tensor decomposition algorithms, such as CP, HOSVD and Tucker decomposition, are derived from <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</i> -mode product. The <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</i> -mode product essentially uses the idea of matrices to deal with tensors given that it defines the multiplication between matrix and higher-order tensor, thereby creating problems of non-pseudodiagonal core tensor and nonunique decomposition results in traditional tensor decomposition algorithms. To this end, decomposition of the original tensor signal and reconstruction of multi-sensor component signals are realised in this study by combining the trajectory tensor construction, superposition of the Gaussian function spectral model, adaptive iterative optimisation of embedding dimension and diagonal average method on the basis of the principle of tensor–tensor order-preserving multiplication. The proposed algorithm inherits the perfect mathematical theory and excellent properties of matrix singular value decomposition in processing single-sensor signals whilst retaining the inherent structure and coupling relationship between multi-sensor data and realising the organic fusion and adaptive decomposition of multi-sensor signals. The analysis results of simulation, experimental and engineering signals showed that the proposed method can effectively extract weak fault quantification features hidden in original multi-sensor signals compared with existing methods.

Global rate optimality of integral curve estimators in high order tensor models

Вдохновленные применениями в нейровизуализации, мы рассматриваем проблему установления глобальной минимаксной нижней границы в модели тензора высокого порядка. В частности, описываемая нами методология позволяет получить глобальную минимаксную границу для оценок интегральных кривых, предложенных в работе О. Кармайкла и второго автора 2015 г., при полупараметрической постановке задачи. Теоретические результаты настоящей работы гарантируют, что оценки, используемые для отслеживания сложной структуры волокон внутри живого человеческого мозга и построенные по данным, полученным из диффузионной тензорной МРТ с высоким угловым разрешением (HARDI), оптимальны не только локально, но и глобально. Таким образом, глобальная минимаксная граница асимптотического риска оценок предоставит квантификацию неопределенности для метода оценки во всей области изображения. В дополнение к теоретическим результатам проводится подробное эмпирическое исследование с целью определить оптимальное число градиентных направлений для протоколов нейровизуализации, которые мы далее иллюстрируем анализом сканирования мозга живого человека по реальным данным.

High-order tensor flow processing using integrated photonic circuits

Tensor analytics lays the mathematical basis for the prosperous promotion of multiway signal processing. To increase computing throughput, mainstream processors transform tensor convolutions into matrix multiplications to enhance the parallelism of computing. However, such order-reducing transformation produces data duplicates and consumes additional memory. Here, we propose an integrated photonic tensor flow processor (PTFP) without digitally duplicating the input data. It outputs the convolved tensor as the input tensor ‘flows’ through the processor. The hybrid manipulation of optical wavelengths, space dimensions, and time delay steps, enables the direct representation and processing of high-order tensors in the optical domain. In the proof-of-concept experiment, an integrated processor manipulating wavelengths and delay steps is implemented for demonstrating the key functionalities of PTFP. The multi-channel images and videos are processed at the modulation rate of 20 Gbaud. A convolutional neural network for video action recognition is demonstrated on the processor, which achieves an accuracy of 97.9%.

A Normal Form Algorithm for Tensor Rank Decomposition

We propose a new numerical algorithm for computing the tensor rank decomposition or canonical polyadic decomposition of higher-order tensors subject to a rank and genericity constraint. Reformulating this computational problem as a system of polynomial equations allows us to leverage recent numerical linear algebra tools from computational algebraic geometry. We characterize the complexity of our algorithm in terms of an algebraic property of this polynomial system—the multigraded regularity. We prove effective bounds for many tensor formats and ranks, which are of independent interest for overconstrained polynomial system solving. Moreover, we conjecture a general formula for the multigraded regularity, yielding a (parameterized) polynomial time complexity for the tensor rank decomposition problem in the considered setting. Our numerical experiments show that our algorithm can outperform state-of-the-art numerical algorithms by an order of magnitude in terms of accuracy, computation time, and memory consumption.

Image-Based Feedback Control Using Tensor Analysis

In manufacturing systems, many quality measurements are in the form of images, including overlay measurements in the semiconductor manufacturing and dimensional deformation profiles of fuselages in an aircraft assembly process. To reduce the process variability and ensure on-target quality, process control strategies should be deployed, in which the high-dimensional image output is controlled by one or more input variables. To design an effective control strategy, the process model should be first estimated via relationship exploration between the image output and inputs, off-line. Next, the control law is formulated by minimizing the control objective function online. The main challenges of achieving such a control strategy include (i) the high dimensional output of a regression model, (ii) the integrated analysis of both the spatial structure of image outputs and the temporal structure of the image sequence, and (iii) non-iid noises. To address these challenges, we propose a novel tensor-based process control approach by incorporating the tensor time series and regression techniques. Based on the process model, we can then obtain the control law by minimizing a control objective function. Although our proposed approach is motivated by a 2D image case, it can be extended to higher-order tensors such as point clouds. Simulation and case studies show that our proposed method is more effective than benchmarks in terms of relative mean square error.

Grassmann higher-order tensor renormalization group approach for two-dimensional strong-coupling QCD

We present a tensor-network approach for two-dimensional strong-coupling QCD with staggered quarks at nonzero chemical potential. After integrating out the gauge fields at infinite coupling, the partition function can be written as a full contraction of a tensor network consisting of coupled local numeric and Grassmann tensors. To evaluate the partition function and to compute observables, we develop a Grassmann higher-order tensor renormalization group method, specifically tailored for this model. During the coarsening procedure, the blocking of adjacent Grassmann tensors is performed analytically, and the total number of Grassmann variables in the tensor network is reduced by a factor of two at each coarsening step. The coarse-site numeric tensors are truncated using higher-order singular value decompositions. The method is validated by comparing the partition function, the chiral condensate and the baryon density computed with the tensor method with exact analytical results on small lattices up to volumes of 4×4. For larger volumes, we present first tensor results for the chiral condensate as a function of the mass and volume, and observe that the chiral symmetry is not broken dynamically in two dimensions. We also present tensor results for the number density as a function of the chemical potential, which hint at a first-order phase transition.

A uniformly-valid asymptotic plate theory of growth with numerical implementation

In this paper, we aim to develop a simplified and uniformly-valid finite-strain plate theory of growth. First, starting from a consistent finite-strain plate theory of growth proposed in our previous work, we specify the magnitudes of growth functions in five different cases and conduct systematic asymptotic analyses, from which the original plate theory can be reduced to some classical plate and membrane theories. Based on the asymptotic analyses, it is found that some terms in the original plate equations (which contain the high-order stress tensor S(2)) always have higher asymptotic orders. By dropping these terms, the plate equations can be simplified significantly. Then, a reduced finite-strain plate theory of growth is established, which is valid in a wide range of growth and mechanical loading conditions. The associated weak form of this uniformly-valid plate theory has also been derived for numerical implementation. To demonstrate the efficiency of this plate theory, it is implemented into a finite element software and applied to study four typical examples. To verify the accuracy of the 2D plate models, the corresponding 3D volume models have also been constructed for these examples. Through some comparisons, it is found that the numerical results obtained from the 2D and 3D models can fit each other quite well. Furthermore, the numerical calculations based on the plate models show obvious advantages in the aspects of computational efficiency, convergence rate and stability. In our opinion, the uniformly-valid asymptotic plate theory of growth proposed in the current work is adequate for studying the complex growth behaviors of thin hyperelastic plates.

DHOST gravity in ultra-diffuse galaxies – part I: the case of NGC1052-DF2

The Ultra-Diffuse galaxy NGC1052-DF2 has recently been revealed to be “extremely deficient” in dark matter, if not lacking it at all. This claim has raised many questions regarding the relationship between baryons and dark matter in Ultra-Diffuse galaxies. But there seems to be a quite unanimous belief that, if such very low dark matter content is confirmed and extended to other similar galactic objects, it might be a deathblow to theories which modify and extend General Relativity. Deficient dark matter galaxies thus represent a fertile ground to test both standard dark matter and modified gravity theories. In this work, we consider a specific Degenerate Higher-Order Scalar Tensor model to study the internal kinematics of NGC1052-DF2. Due to the partial breaking of the corresponding screening mechanism, this model can possibly have large cosmological scale effects influencing the dynamics of smaller structures like galaxies. We consider two scenarios: one in which the model only describes dark energy; and one in which it additionally entirely substitutes dark matter. We find that the best model to explain data is General Relativity with only stellar contribution. But while in the former scenario General Relativity is still statistically favoured, in the latter one the alternative model is as much successful and effective as General Relativity in matching observations. Thus, we can conclude that even objects like NGC1052-DF2 are not in contrast, and are not obstacles, to the study and the definition of a reliable alternative to General Relativity.

Detecting anomalous sequences in electronic health records using higher-order tensor networks

Detecting anomalous sequences is an integral part of building and protecting modern large-scale health information technology (HIT) systems. These HIT systems generate a large volume of records of patients’ state and significant events, which provide a valuable resource to help improve clinical decisions, patient care processes, and other issues. However, detecting anomalous sequences in electronic health records (EHR) remains a challenge in healthcare applications for several reasons, including imbalances in the data, complexity of relationships between events in the sequence, and the curse of dimensionality. Conventional anomaly detection methods use the finite sequence of events to discriminate sequences. They fail to incorporate salient event details under variable higher-order dependencies (e.g., duration between events) that can provide better discrimination of sequences in their models. To address this problem, we propose event sequence and subsequence anomaly detection algorithms that (1) use network-based representations of interactions in the data, (2) account for variable higher-order dependencies in the data, and (3) incorporate events duration for adequate discrimination of the data. The proposed approach identifies anomalies by monitoring the change in the graph after the test sequence is removed from the network. The change is quantified using graph distance metrics so that dramatic changes in the network can be attributed to the removed sequence. Furthermore, the proposed subsequence algorithm recommends plausible paths and salient information for the detected anomalous subsequences. Our results show that the proposed event sequence anomaly detection algorithm outperforms the baseline methods for both synthetic data and real-world EHR data.

The Real Eigenpairs of Symmetric Tensors and Its Application to Independent Component Analysis.

It has been proved that the determination of independent components (ICs) in the independent component analysis (ICA) can be attributed to calculating the eigenpairs of high-order statistical tensors of the data. However, previous works can only obtain approximate solutions, which may affect the accuracy of the ICs. In addition, the number of ICs would need to be set manually. Recently, an algorithm based on semidefinite programming (SDP) has been proposed, which utilizes the first-order gradient information of the Lagrangian function and can obtain all the accurate real eigenpairs. In this article, for the first time, we introduce this into the ICA field, which tends to further improve the accuracy of the ICs. Note that the number of eigenpairs of symmetric tensors is usually larger than the number of ICs, indicating that the results directly obtained by SDP are redundant. Thus, in practice, it is necessary to introduce second-order derivative information to identify local extremum solutions. Therefore, originating from the SDP method, we present a new modified version, called modified SDP (MSDP), which incorporates the concept of the projected Hessian matrix into SDP and, thus, can intellectually exclude redundant ICs and select true ICs. Some cases that have been tested in the experiments demonstrate its effectiveness. Experiments on the image/sound blind separation and real multi/hyperspectral image also show its superiority in improving the accuracy of ICs and automatically determining the number of ICs. In addition, the results on hyperspectral simulation and real data also demonstrate that MSDP is also capable of dealing with cases, where the number of features is less than the number of ICs.