Tensor Regression Model Research Articles

Improving Tensor Regression by Optimal Model Averaging

Tensors have broad applications in neuroimaging, data mining, digital marketing, etc. CANDECOMP/PARAFAC (CP) tensor decomposition can effectively reduce the number of parameters to gain dimensionality-reduction and thus plays a key role in tensor regression. However, in CP decomposition, there is uncertainty about which rank to use. In this article, we develop a model averaging method to handle this uncertainty by weighting the estimators from candidate tensor regression models with different ranks. When all candidate models are misspecified, we prove that the model averaging estimator is asymptotically optimal. When correct models are included in the set of candidate models, we prove the consistency of parameters and the convergence of the model averaging weight. Simulations and empirical studies illustrate that the proposed method has superiority over the competition methods and has promising applications. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

Tensor-based statistical learning methods for diagnosing product quality defects in multistage manufacturing processes

Multistage manufacturing processes with identical stages provide three-dimensional process data in which the first dimension represents the process (control/sensing) variable, the second is the stage, and the third is the measurement/sampling/data acquisition time point. Diagnosing quality faults in such processes often requires the simultaneous identification of crucial process variables and stages associated with product quality anomalies. Most existing diagnosis methods convert 3D data into a 2D matrix, resulting in loss of information and reduced diagnostic accuracy and stability. To address this challenge, we propose a penalized tensor regression model that regresses the product quality index against its 3D process data. For the estimation of high-dimensional regression coefficients with the limited amount of historical data, we apply the CANDECOMP/PARAFAC and Tucker decompositions to the coefficient tensor, which significantly reduces the number of parameters to be estimated. Based on the decompositions, a new regularization term is designed to enable the joint identification of critical process variables and stages. To estimate the parameters, we develop the block coordinate proximal descent algorithm and provide its convergence guarantee. Numerical studies demonstrate that the proposed methods can enhance diagnostic stability and on average improve the diagnostic accuracy by around 20% over existing benchmarks.

Change-point detection in a tensor regression model

Change-point detection in a tensor regression model

A tensor based varying-coefficient model for multi-modal neuroimaging data analysis.

All neuroimaging modalities have their own strengths and limitations. A current trend is toward interdisciplinary approaches that use multiple imaging methods to overcome limitations of each method in isolation. At the same time neuroimaging data is increasingly being combined with other non-imaging modalities, such as behavioral and genetic data. The data structure of many of these modalities can be expressed as time-varying multidimensional arrays (tensors), collected at different time-points on multiple subjects. Here, we consider a new approach for the study of neural correlates in the presence of tensor-valued brain images and tensor-valued covariates, where both data types are collected over the same set of time points. We propose a time-varying tensor regression model with an inherent structural composition of responses and covariates. Regression coefficients are expressed using the B-spline technique, and the basis function coefficients are estimated using CP-decomposition by minimizing a penalized loss function. We develop a varying-coefficient model for the tensor-valued regression model, where both covariates and responses are modeled as tensors. This development is a non-trivial extension of function-on-function concurrent linear models for complex and large structural data, where the inherent structures are preserved. In addition to the methodological and theoretical development, the efficacy of the proposed method based on both simulated and real data analysis (e.g., the combination of eye-tracking data and functional magnetic resonance imaging (fMRI) data) is also discussed.

Multi-faceted analysis and prediction for the outbreak of pediatric respiratory syncytial virus.

Respiratory syncytial virus (RSV) is a significant cause of pediatric hospitalizations. This article aims to utilize multisource data and leverage the tensor methods to uncover distinct RSV geographic clusters and develop an accurate RSV prediction model for future seasons. This study utilizes 5-year RSV data from sources, including medical claims, CDC surveillance data, and Google search trends. We conduct spatiotemporal tensor analysis and prediction for pediatric RSV in the United States by designing (i) a nonnegative tensor factorization model for pediatric RSV diseases and location clustering; (ii) and a recurrent neural network tensor regression model for county-level trend prediction using the disease and location features. We identify a clustering hierarchy of pediatric diseases: Three common geographic clusters of RSV outbreaks were identified from independent sources, showing an annual RSV trend shifting across different US regions, from the South and Southeast regions to the Central and Northeast regions and then to the West and Northwest regions, while precipitation and temperature were found as correlative factors with the coefficient of determination R2≈0.5, respectively. Our regression model accurately predicted the 2022-2023 RSV season at the county level, achieving R2≈0.3 mean absolute error MAE < 0.4 and a Pearson correlation greater than 0.75, which significantly outperforms the baselines with P-values <.05. Our proposed framework provides a thorough analysis of RSV disease in the United States, which enables healthcare providers to better prepare for potential outbreaks, anticipate increased demand for services and supplies, and save more lives with timely interventions.

Tensor Stein-rules in a generalized tensor regression model

In this paper, we consider an estimation problem in a generalized tensor regression model with multi-mode covariates. We generalize the main results in recent literature in four ways. First, we weaken assumptions underlying the main results of the previous works. In particular, the dependence structure of the error and covariates are as weak as an L2− mixingale array, and the error term does not need to be uncorrelated with regressors. Second, we consider a more general constraint than the one considered in previous works. Third, we establish the asymptotic properties of the tensor estimators. Specifically, we derive the joint asymptotic distribution of the unrestricted tensor estimator (UE) and restricted tensor estimator (RE). Fourth, we propose a class of shrinkage-type estimators in the context of tensor regression, and under a general loss function, we derive sufficient conditions for which the shrinkage estimators dominate the UE. In addition to these interesting contributions, we derive a kind of functional central limit theorem for mixingale vector-valued and we establish some identities which are useful in studying the risk dominance of shrinkage-type tensor estimators. Finally, to illustrate the application of the proposed methods, we corroborate the results by some simulation studies of binary, Normal and Poisson data and we analyze a multi-relational network and neuro-imaging datasets.

TensorTest2D: Fitting Generalized Linear Models with Matrix Covariates

The [TensorTest2D](https://CRAN.R-project.org/package=TensorTest2D) package provides the means to fit generalized linear models on second-order tensor type data. Functions within this package can be used for parameter estimation (e.g., estimating regression coefficients and their standard deviations) and hypothesis testing. We use two examples to illustrate the utility of our package in analyzing data from different disciplines. In the first example, a tensor regression model is used to study the effect of multi-omics predictors on a continuous outcome variable which is associated with drug sensitivity. In the second example, we draw a subset of the MNIST handwritten images and fit to them a logistic tensor regression model. A significance test characterizes the image pattern that tells the difference between two handwritten digits. We also provide a function to visualize the areas as effective classifiers based on a tensor regression model. The visualization tool can also be used together with other variable selection techniques, such as the LASSO, to inform the selection results.

Regularized high dimension low tubal-rank tensor regression

Tensor regression models are of emerging interest in diverse fields of social and behavioral sciences, including neuroimaging analysis, neural networks, image processing and so on. Recent theoretical advancements of tensor decomposition have facilitated significant development of various tensor regression models. The focus of most of the available literature has been on the Canonical Polyadic (CP) decomposition and its variants for the regression coefficient tensor. A CP decomposed coefficient tensor enables estimation with relatively small sample size, but it may not always capture the underlying complex structure in the data. In this work, we leverage the recently developed concept of tubal rank and develop a tensor regression model, wherein the coefficient tensor is decomposed into two components: a low tubal rank tensor and a structured sparse one. We first address the issue of identifiability of the two components comprising the coefficient tensor and subsequently develop a fast and scalable Alternating Minimization algorithm to solve the convex regularized program. Further, we provide finite sample error bounds under high dimensional scaling for the model parameters. The performance of the model is assessed on synthetic data and is also used in an application involving data from an intelligent tutoring platform.

Improved estimation in tensor regression with multiple change-points

In this paper, we consider an estimation problem about the tensor coefficient in a tensor regression model with multiple and unknown change-points. We generalize some recent findings in five ways. First, the problem studied is more general than the one in context of a matrix parameter with multiple change-points. Second, we develop asymptotic results of the tensor estimators in the context of a tensor regression with unknown change-points. Third, we construct a class of shrinkage tensor estimators that encompasses the unrestricted estimator (UE) and the restricted estimator (RE). Fourth, we generalize some identities which are crucial in deriving the asymptotic distributional risk (ADR) of the tensor estimators. Fifth, we show that the proposed shrinkage estimators perform better than the UE. The additional novelty of the established results consists in the fact that the dependence structure of the errors is as weak as that of an L2-mixingale. Finally, the theoretical results are corroborated by the simulation findings and our methods are applied to analyse MRI and fMRI datasets.

Fast Prediction of Thermal Data Stream for Direct Laser Deposition Processes Using Network-Based Tensor Regression

Abstract The objective of this research is to study an effective thermal data stream prediction method for additive manufacturing (AM) processes using thermal image streams in a layer-wise manner. Reliable physics-based models have been developed to delineate the underlying thermomechanical dynamics of AM processes. However, the computational cost is extremely high. We proposed a tensor-based surrogate modeling methodology to predict the layer-wise relationship in thermal data stream of the AM parts, which is time efficient compared to available physics-based prediction models. We constructed a network tensor structure for freeform shapes based on thermal image streams obtained in metal-based AM processes. Then, we simplified the network tensor structure by concatenating images to reach a layer-wise structure. Subsequent layers were predicted based on the antecedent layer using the tensor regression model. A generalized multilinear structure, called the higher order partial least squares (HOPLS), was used to estimate the tensor regression model parameters. Through the proposed method, high-dimensional thermal histories of AM components were predicted accurately in a computationally efficient manner. Prediction performance indices (i.e., Q2=0.999 and root-mean-square errors of prediction (RMSEP) = 31.212 °C) demonstrated a significantly more efficient layer-wise prediction of thermal data stream—a larger Q2 (0 ≤ Q2 ≤ 1) and a smaller RMSEP indicated a better prediction performance. The proposed thermal data stream prediction was validated on simulated thermal images from finite element (FE) simulations.

Multi-view feature selection via sparse tensor regression

In this paper, we propose a sparse tensor regression model for multi-view feature selection. Apart from the most of existing methods, our model adopts a tensor structure to represent multi-view data, which aims to explore their underlying high-order correlations. Based on this tensor structure, our model can effectively select the meaningful feature set for each view. We also develop an iterative optimization algorithm to solve our model, together with analysis about the convergence and computational complexity. Experimental results on several popular multi-view data sets confirm the effectiveness of our model.

Predicting multidimensional data via tensor learning

The analysis of multidimensional data is becoming a more and more relevant topic in statistical and machine learning research. Given their complexity, such data objects are usually reshaped into matrices or vectors and then analysed. However, this methodology presents several drawbacks. First of all, it destroys the intrinsic interconnections among datapoints in the multidimensional space and, secondly, the number of parameters to be estimated in a model increases exponentially. We develop a model that overcomes such drawbacks. In particular, in this paper, we propose a parsimonious tensor regression model that retains the intrinsic multidimensional structure of the dataset. Tucker structure is employed to achieve parsimony and a shrinkage penalization is introduced to deal with over-fitting and collinearity. To estimate the model parameters, an Alternating Least Squares algorithm is developed. In order to validate the model performance and robustness, a simulation exercise is produced. Moreover, we perform an empirical analysis that highlight the forecasting power of the model with respect to benchmark models. This is achieved by implementing an autoregressive specification on the Foursquares spatio-temporal dataset together with a macroeconomic panel dataset. Overall, the proposed model is able to outperform benchmark models present in the forecasting literature.

An improved tensor regression model via location smoothing

Many applications of regression study the predictors with complex forms such as tensors. Besides low dimensional assumption, the effects of predictors with a tensor structure typically show clustered or smooth phenomena in the sense that adjacent elements have a similar effect on the response. To simultaneously incorporate the low‐rank and smoothness in tensor regression, we generalize the CANDECOMP/PARAFAC (CP) decomposition to a smooth version and propose a novel regression model based on the smoothed decomposition. The asymptotic theory of the proposed method is studied, which shows a faster rate of convergence than the one without incorporating the smoothness. The experiments on both synthetic and real data confirm that the proposed method has advantages over the alternative especially when the sample size is small.

Low-rank tensor ring learning for multi-linear regression

The emergence of large-scale data demands new regression models with multi-dimensional coefficient arrays, known as tensor regression models. The recently proposed tensor ring decomposition has interesting properties of enhanced representation and compression capability, cyclic permutation invariance and balanced tensor ring rank, which may lead to efficient computation and fewer parameters in regression problems. In this paper, a generally multi-linear tensor-on-tensor regression model is proposed that the coefficient array has a low-rank tensor ring structure, which is termed tensor ring ridge regression (TRRR). Two optimization models are developed for the TRRR problem and solved by different algorithms: the tensor factorization based one is solved by alternating least squares algorithm, and accelerated by a fast network contraction, while the rank minimization based one is addressed by the alternating direction method of multipliers algorithm. Comparative experiments, including Spatio-temporal forecasting tasks and 3D reconstruction of human motion capture data from its temporally synchronized video sequences, demonstrate the enhanced performance of our algorithms over existing state-of-the-art ones, especially in terms of training time.

Sparse regularized low-rank tensor regression with applications in genomic data analysis

Many applications in biomedical informatics deal with data in the tensor form. Traditional regression methods which take vectors as covariates may encounter difficulties in handling tensors due to their ultrahigh dimensionality and complex structure. In this paper, we introduce a novel sparse regularized Tucker tensor regression model to exploit the structure of tensor covariates and perform feature selection on tensor data. Based on Tucker decomposition of the regression coefficient tensor, we reduce the ultrahigh dimensionality to a manageable level. To make our model identifiable, we impose the orthonormality constraint on the factor matrices. Unlike previous tensor regression models that impose sparse penalty on the factor matrices of the coefficient tensor, our model directly imposes sparse penalty on the coefficient tensor to select the relevant features on tensor data. An efficient optimization algorithm based on alternating direction method of multiplier (ADMM) algorithm is designed to solve our proposed model. The performance of our model is evaluated on both synthetic and real genomic data. Experiment results on synthetic data demonstrate that our model could identify the true related signals more accurately than other state-of-the-art regression models. The analysis on genomic data of melanoma demonstrates that our model can achieve better prediction performance and identify markers with important implications. Our model and the associated studies can provide useful insights to disease or pathogenesis mechanisms, and will benefit further studies in variable selection.

Detecting Structural Changes in Longitudinal Network Data

Dynamic modeling of longitudinal networks has been an increasingly important topic in applied research. While longitudinal network data commonly exhibit dramatic changes in its structures, existing methods have largely focused on modeling smooth topological changes over time. In this paper, we develop a hidden Markov network change-point model (HNC) that combines the multilinear tensor regression model (Hoff, 2011) with a hidden Markov model using Bayesian inference. We model changes in network structure as shifts in discrete states yielding particular sets of network generating parameters. Our simulation results demonstrate that the proposed method correctly detects the number, locations, and types of changes in latent node characteristics. We apply the proposed method to international military alliance networks to find structural changes in the coalition structure among nations.

张量回归模型及其应用研究综述

张量回归模型及其应用研究综述

Quantitative prediction of the aged state of Ni-base superalloys using PCA and tensor regression

The microstructure of Ni-base superalloy components evolves and degrades during the operation of gas turbines. Since the remaining life depends on the degradation, it is highly desirable to have a quantitative descriptor of the aged state of the microstructure that can be linked to the operating conditions. In this paper, data analytics algorithms are used to develop such relationships. High-throughput aging experiments were performed to generate a dataset comprising multiple aged microstructure images. The digital images of the γ/γ′ phase are used as an indicator of the aged state and statistically evaluated using 2-point spatial correlation functions. To reduce the high-dimensional structural information so that a quantitative linkage can be made between aging conditions and the aged state, two algorithms were considered. The first algorithm involves two steps, first using conventional principal component analysis (PCA) to provide a lower dimension descriptor of the microstructure and then regression analysis to generate the linkage. The second algorithm, called tensor regression (TR), is a novel algorithm that merges the dimensionality reduction and model construction step into a single step. The output of the TR model is directly the statistical descriptors of the microstructure rather than the PC scores, thereby reducing the amount of information loss. Even though PCA provides an effective tool for visualization and classification of data, the model built based on the TR algorithm is shown to have stronger prediction capability.

Tucker Tensor Regression and Neuroimaging Analysis.

Neuroimaging data often take the form of high dimensional arrays, also known as tensors. Addressing scientific questions arising from such data demands new regression models that take multidimensional arrays as covariates. Simply turning an image array into a vector would both cause extremely high dimensionality and destroy the inherent spatial structure of the array. In a recent work, Zhou et al. (2013) proposed a family of generalized linear tensor regression models based upon the CP (CANDECOMP/PARAFAC) decomposition of regression coefficient array. Low rank approximation brings the ultrahigh dimensionality to a manageable level and leads to efficient estimation. In this article, we propose a tensor regression model based on the more flexible Tucker decomposition. Compared to the CP model, Tucker regression model allows different number of factors along each mode. Such flexibility leads to several advantages that are particularly suited to neuroimaging analysis, including further reduction of the number of free parameters, accommodation of images with skewed dimensions, explicit modeling of interactions, and a principled way of image downsizing. We also compare the Tucker model with CP numerically on both simulated data and a real magnetic resonance imaging data, and demonstrate its effectiveness in finite sample performance.

MULTILINEAR TENSOR REGRESSION FOR LONGITUDINAL RELATIONAL DATA.

A fundamental aspect of relational data, such as from a social network, is the possibility of dependence among the relations. In particular, the relations between members of one pair of nodes may have an effect on the relations between members of another pair. This article develops a type of regression model to estimate such effects in the context of longitudinal and multivariate relational data, or other data that can be represented in the form of a tensor. The model is based on a general multilinear tensor regression model, a special case of which is a tensor autoregression model in which the tensor of relations at one time point are parsimoniously regressed on relations from previous time points. This is done via a separable, or Kronecker-structured, regression parameter along with a separable covariance model. In the context of an analysis of longitudinal multivariate relational data, it is shown how the multilinear tensor regression model can represent patterns that often appear in relational and network data, such as reciprocity and transitivity.