Tucker Decomposition Research Articles

Tensor Dictionary Self-Taught Learning Classification Method for Hyperspectral Image

Precise object classification based on Hyperspectral imagery with limited training data presents a challenging task. We propose a tensor-based dictionary self-taught learning (TDSL) classification method to provide some insight into these challenges. The idea of TDSL is to utilize a small amount of unlabeled data to improve the supervised classification. The TDSL trains tensor feature extractors from unlabeled data, extracts joint spectral-spatial tensor features and performs classification on the labeled data set. These two data sets can be gathered over different scenes even by different sensors. Therefore, TDSL can complete cross-scene and cross-sensor classification tasks. For training tensor feature extractors on unlabeled data, we propose a sparse tensor-based dictionary learning algorithm for three-dimensional samples. In the algorithm, we initialize dictionaries using Tucker decomposition and update these dictionaries based on the K higher-order singular value decomposition. These dictionaries are feature extractors, which are used to extract sparse joint spectral-spatial tensor features on the labeled data set. To provide classification results, the support vector machine as the classifier is applied to the tensor features. The TDSL with the majority vote (TDSLMV) can reduce the misclassified pixels in homogenous regions and at the edges of different homogenous regions, which further refines the classification. The proposed methods are evaluated on Indian Pines, Pavia University, and Houston2013 datasets. The classification results show that TDSLMV achieves as high as 99.13%, 99.28%, and 99.76% accuracies, respectively. Compared with several state-of-the-art methods, the classification accuracies of the proposed methods are improved by at least 2.5%.

Low Tucker rank tensor completion using a symmetric block coordinate descent method

AbstractLow Tucker rank tensor completion has wide applications in science and engineering. Many existing approaches dealt with the Tucker rank by unfolding matrix rank. However, unfolding a tensor to a matrix would destroy the data's original multi‐way structure, resulting in vital information loss and degraded performance. In this article, we establish a relationship between the Tucker ranks and the ranks of the factor matrices in Tucker decomposition. Then, we reformulate the low Tucker rank tensor completion problem as a multilinear low rank matrix completion problem. For the reformulated problem, a symmetric block coordinate descent method is customized. For each matrix rank minimization subproblem, the classical truncated nuclear norm minimization is adopted. Furthermore, temporal characteristics in image and video data are introduced to such a model, which benefits the performance of the method. Numerical simulations illustrate the efficiency of our proposed models and methods.

Three-dimensional wind field reconstruction using tucker decomposition with optimal sensor placement

Short-term wind prediction is of great significance for wind power trading strategy, equipment protection, and effective control. Physical models can provide accurate and detailed wind speed data but require a large amount of time for calculation. To solve this problem, based on Tucker decomposition, a sensor and CFD (Computational Fluid Dynamics) data fusion technique is proposed, which allows us to readily obtain an estimation of the ‘actual’ three-dimensional wind field from sensor observations. Moreover, a new greedy algorithm based on minimum condition is developed to stabilize the inverse process and derive optimal sensor placement. In the simulation, the ‘actual’ wind field can be reconstructed with reasonable accuracy for different wind shear characteristics, and the relative errors of absolute velocity is less than 0.2%. Additionally, the reconstruction time is much less than ultra-short-term forecasting and accounting for only 0.2%–3.2% of the CFD calculation time. The experimental results show that the optimal placement is much more effective than random placement in terms of minimizing the relative error (the relative error can be controlled within 6%). In general, the innovative approach combines the advantages of both statistical and physical approaches while compensating for their shortcomings, making it potentially valuable in short-term wind forecasting.

Compressing Deep Model With Pruning and Tucker Decomposition for Smart Embedded Systems

Deep learning has been proved to be one of the most effective method in feature encoding for different intelligent applications such as video-based human action recognition. However, its nonconvex optimization mechanism leads large memory consumption, which hinders its deployment on the smart embedded systems with limited computational resources. To overcome this challenge, we propose a novel deep model compression technique for smart embedded systems, which realizes both the memory size reduction and inference complexity decrease within a small drop of accuracy. First, we propose an improved naive Bayes inference-based channel parameter pruning to obtain a sparse model with higher accuracy. Then, to improve the inference efficiency, the improved Tucker decomposition method is proposed, where an improved genetic algorithm is used to optimize the Tucker ranks. Finally, to elevate the effectiveness of our proposed method, extensive experiments are conducted. The experimental results show that our method can achieve the state-of-the-art performance compared with existing methods in terms of accuracy, parameter compression, and floating-point operations reduction.

Adaptive Band Extraction Based on Low Rank Approximated Nonnegative Tucker Decomposition for Anti-Friction Bearing Faults Diagnosis Using Measured Vibration Data

Condition monitoring and fault diagnosis are topics of growing interest for improving the reliability of modern industrial systems. As critical structural components, anti-friction bearings often operate under harsh conditions and are contributing factors of system failures. Efforts have been cast on bearing diagnostics under the sensor fusion and machine learning framework, whilst challenges remain open on the identification of incipient faults. In this paper, exploiting multi-way representations and decompositions of measured vibration data, a novel band separation method based on the factorization of spectrogram tensors using the low rank approximated nonnegative Tucker decomposition (LRANTD) is proposed and applied to identify detailed fault signatures from the spectral, temporal, and spatial dimensions, flexible for extracting multi-sensor features and multi-dimensional correlations. With the proposed method, informative frequency bands of the latent vibrational components can be automatically extracted, in accordance with the inherent temporal patterns that can be conveniently fed for spectral analysis and fault discrimination. Furthermore, an improved cross-spectrum can be calculated from multi-channel vibrations via LRANTD with enhanced fault features. Based on the real-world vibration data of the accelerated bearing life tests, detailed experimental studies and thorough comparisons to the conventional benchmarks have verified the effectiveness of the reported diagnostic methodology. The proposed method significantly improves the presence of the bearing frequency peaks distinctly over the background noises in the spectrum and hence improves the bearing defect detection process.

A-Tucker: fast input-adaptive and matricization-free Tucker decomposition of higher-order tensors on GPUs

a-Tucker: fast input-adaptive and matricization-free Tucker decomposition of higher-order tensors on GPUs

Time-Constrained Adversarial Defense in IoT Edge Devices through Kernel Tensor Decomposition and Multi-DNN Scheduling.

The development of deep learning technology has resulted in great contributions in many artificial intelligence services, but adversarial attack techniques on deep learning models are also becoming more diverse and sophisticated. IoT edge devices take cloud-independent on-device DNN (deep neural network) processing technology to exhibit a fast response time. However, if the computational complexity of the denoizer for adversarial noises is high, or if a single embedded GPU is shared by multiple DNN models, adversarial defense at the on-device level is bound to represent a long latency. To solve this problem, eDenoizer is proposed in this paper. First, it applies Tucker decomposition to reduce the computational amount required for convolutional kernel tensors in the denoizer. Second, eDenoizer effectively orchestrates both the denoizer and the model defended by the denoizer simultaneously. In addition, the priority of the CPU side can be projected onto the GPU which is completely priority-agnostic, so that the delay can be minimized when the denoizer and the defense target model are assigned a high priority. As a result of confirming through extensive experiments, the reduction of classification accuracy was very marginal, up to 1.78%, and the inference speed accompanied by adversarial defense was improved up to 51.72%.

Impacts of the COVID-19 pandemic on the spatio-temporal characteristics of a bicycle-sharing system: A case study of Pun Pun, Bangkok, Thailand.

The COVID-19 pandemic is found to be one of the external stimuli that greatly affects mobility of people, leading to a shift of transportation modes towards private individual ones. To properly explain the change in people's transport behavior, especially in pre- and post- pandemic periods, a tensor-based framework is herein proposed and applied to Pun Pun-the only public bicycle-sharing system in Bangkok, Thailand-where multidimensional trip data of Pun Pun are decomposed into four different modes related to their spatial and temporal dimensions by a non-negative Tucker decomposition approach. According to our computational results, the first pandemic wave has a sizable influence not only on Pun Pun but also on other modes of transportation. Nonetheless, Pun Pun is relatively more resilient, as it recovers more quickly than other public transportation modes. In terms of trip patterns, we find that, prior to the pandemic, trips made during weekdays are dominated by business trips with two peak periods (morning and evening peaks), while those made during weekends are more related to leisure activities as they involve stations nearby a public park. However, after the first pandemic wave ends, the patterns of weekday trips have been drastically changed, as the number of business trips sharply drops, while that of educational trips connecting metro/subway stations with a major educational institute in the region significantly rises. These findings may be regarded as a reflection of the ever-changing transport behavior of people seeking a sustainable mode of private transport, with a more positive outlook on the use of bicycle-sharing system in Bangkok, Thailand.

Displacement Data Imputation in Urban Internet of Things System Based on Tucker Decomposition With L2 Regularization

Missing data are critical deficiency in the investigation of displacement measurement in urban Internet of Things system. In the insight of recovering missing displacement data, this article presents a data-driven and high-dimensional gap-imputation method, Tucker decomposition with L2 regularization. Results on the global navigation satellite system (GNSS) time series collected from an intelligent structural health monitoring system show that the recovery accuracy is improved compared with some popular benchmark methods. When the missing rate is 50%, compared with singular spectrum analysis, singular value decomposition, CP optimization algorithm, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> -nearest neighbors, and Tucker decomposition via alternating least squares, Tucker decomposition with L2 regularization can improve the average mean absolute error by about 4.74, 4.95, 5.82, 2.29, and 5.67 mm for all locations. It can be concluded that the consideration of multiple temporal correlations is necessary for missing data imputation. Compared with matrix decomposition, tensor decomposition can improve the ability for high-dimensional correlations in the GNSS time series.

Three-dimensional SAR imaging with sparse linear array using tensor completion in embedded space

Due to the huge data storage and transmission pressure, sparse data collection strategy has provided opportunities and challenges for 3D SAR imaging. However, sparse data brought by the sparse linear array will produce high-level side-lobes, as well as the aliasing and the false-alarm targets. Simultaneously, the vectorizing or matrixing of 3D data makes high computational complexity and huge memory usage, which is not practicable in real applications. To deal with these problems, tensor completion (TC), as a convex optimization problem, is used to solve the 3D sparse imaging problem efficiently. Unfortunately, the traditional TC methods are invalid to the incomplete tensor data with missing slices brought by sparse linear arrays. In this paper, a novel 3D imaging algorithm using TC in embedded space is proposed to produce 3D images with efficient side-lobes suppression. With the help of sparsity and low-rank property hidden in the 3D radar signal, the incomplete tensor data is taken as the input and converted into a higher order incomplete Hankel tensor by multiway delay embedding transform (MDT). Then, the tucker decomposition with incremental rank has been applied for completion. Subsequently, any traditional 3D imaging methods can be employed to obtain excellent imaging performance for the completed tensor. The proposed method achieves high resolution and low-level side-lobes compared with the traditional TC-based methods. It is verified by several numerical simulations and multiple comparative studies on real data. Results clearly demonstrate that the proposed method can generate 3D images with small reconstruction error even when the sparse sampling rate or signal to noise ratio is low, which confirms the validity and advantage of the proposed method.

A Hybrid Norm for Guaranteed Tensor Recovery

Benefiting from the superiority of tensor Singular Value Decomposition (t-SVD) in excavating low-rankness in the spectral domain over other tensor decompositions (like Tucker decomposition), t-SVD-based tensor learning has shown promising performance and become an emerging research topic in computer vision and machine learning very recently. However, focusing on modeling spectral low-rankness, the t-SVD-based models may be insufficient to exploit low-rankness in the original domain, leading to limited performance while learning from tensor data (like videos) that are low-rank in both original and spectral domains. To this point, we define a hybrid tensor norm dubbed the “Tubal + Tucker” Nuclear Norm (T2NN) as the sum of two tensor norms, respectively, induced by t-SVD and Tucker decomposition to simultaneously impose low-rankness in both spectral and original domains. We further utilize the new norm for tensor recovery from linear observations by formulating a penalized least squares estimator. The statistical performance of the proposed estimator is then analyzed by establishing upper bounds on the estimation error in both deterministic and non-asymptotic manners. We also develop an efficient algorithm within the framework of Alternating Direction Method of Multipliers (ADMM). Experimental results on both synthetic and real datasets show the effectiveness of the proposed model.

Covariate-Assisted Community Detection in Multi-Layer Networks

ABSTRACT Communities in multi-layer networks consist of nodes with similar connectivity patterns across all layers. This article proposes a tensor-based community detection method in multi-layer networks, which leverages available node-wise covariates to improve community detection accuracy. This is motivated by the network homophily principle, which suggests that nodes with similar covariates tend to reside in the same community. To take advantage of the node-wise covariates, the proposed method augments the multi-layer network with an additional layer constructed from the node similarity matrix with proper scaling, and conducts a Tucker decomposition of the augmented multi-layer network, yielding the spectral embedding vector of each node for community detection. Asymptotic consistencies of the proposed method in terms of community detection are established, which are also supported by numerical experiments on various synthetic networks and two real-life multi-layer networks.

A Hybrid Tucker-VQ Tensor Sketch decomposition model for coding and streaming real world light fields using stack of differently focused images

• Lightfield representation, coding and streaming schemes using a stack of focal images. • Layer pattern optimization from focal stack for efficient streaming on tensor display. • Hybrid Tucker Tensor Sketch Scheme with Vector Quantization for low-rank modeling. • A single integrated, adaptable and flexible system to realize a range of bitrates. Computational multi-view displays involving light fields are a fast emerging choice for 3D presentation of real-world scenes. Tensor autostereoscopic glasses-free displays use just few light attenuating layers in front of a backlight to output high quality light field. We propose three novel schemes, Focal Stack - Hybrid Tucker-TensorSketch Vector Quantization (FS-HTTSVQ), Focal Stack - Tucker-TensorSketch (FS-TTS), and Focal Stack - Tucker Alternating Least-Squares (FS-TALS), for efficient representation, streaming and coding of light fields using a stack of differently focused images. Working with a focal stack instead of the entire light field majorly reduces the data acquisition cost as well as the computation and processing cost. Extensive experiments with real world light field focal stacks demonstrate that proposed novel one-pass Tucker decomposition using TensorSketch with hybrid vector quantization in FS-HTTSVQ, compactly represents the approximated focal stack in codebook form for better transmission and streaming. Encoding with High Efficiency Video Coding (HEVC) eliminates all intrinsic redundancies present in the approximated focal stack. Resultant low-rank approximated and coded focal stack is then employed to analytically optimize layer patterns for the tensor display. The complete end-to-end light field processing pipelines flexibly work for multiple bitrates and are adaptable for a variety of multi-view autostereoscopic platforms. Our schemes exhibit note-worthy performances on focal stacks compared to direct encoding of an entire light field using a standard codec like HEVC.

Random Tensor Theory for Tensor Decomposition

We propose a new framework for tensor decomposition based on trace invariants, which are particular cases of tensor networks. In general, tensor networks are diagrams/graphs that specify a way to "multiply" a collection of tensors together to produce another tensor, matrix or scalar. The particularity of trace invariants is that the operation of multiplying copies of a certain input tensor that produces a scalar obeys specific symmetry constraints. In other words, the scalar resulting from this multiplication is invariant under some specific transformations of the involved tensor. We focus our study on the O(N)-invariant graphs, i.e. invariant under orthogonal transformations of the input tensor. The proposed approach is novel and versatile since it allows to address different theoretical and practical aspects of both CANDECOMP/PARAFAC (CP) and Tucker decomposition models. In particular we obtain several results: (i) we generalize the computational limit of Tensor PCA (a rank-one tensor decomposition) to the case of a tensor with axes of different dimensions (ii) we introduce new algorithms for both decomposition models (iii) we obtain theoretical guarantees for these algorithms and (iv) we show improvements with respect to state of the art on synthetic and real data which also highlights a promising potential for practical applications.

Robust fast PMU measurement recovery enhanced by randomized singular value and sequential Tucker decomposition

The development of cyber-physical power systems raises concerns about the data quality issue of phasor measurement units (PMUs). Low signal-to-noise ratios (SNRs) and data losses caused by malicious electromagnetic interference, false data injections, and equipment malfunctioning may jeopardize the data integrity and availability necessary for power system monitoring, protection, and control. To ensure grid resiliency, this paper proposes a robust fast PMU measurement recovery (RFMR) algorithm based on improved singular spectrum analysis (SSA) of Hankel structures. It utilizes single or multiple channels of PMU time-series to restore the problematic phasor measurements with low-SNR noises and data losses. Additionally, the traditional singular value decomposition (SVD) and Tucker decomposition (TD) in RFMR are replaced by randomized SVD (RSVD) and sequential TD (STD) to reduce the computational complexity in single-channel and multi-channel RFMR, respectively. Numerical case studies demonstrate that the proposed algorithm can recover the noise-contaminated measurements with higher accuracy than existing methods, such as matrix/tensor decomposition approaches and robust principal component analysis (RPCA), and effectively complement the missing data with the observed measurements corrupted by low SNRs. Moreover, the latency margins of various power system synchrophasor application scenarios can be satisfied with the reduced computational complexity.

Scalable tensor factorization for recovering multiday missing intramuscular electromyography data

To design a prosthetic hand which can classify movements based on the electromyography (EMG) signals, complete and good quality signals are essential. However, due to different reasons such as disconnection of electrodes or muscles fatigue the recorded EMG data can be incomplete, which degrades the classification of test movements. In this paper, we first acquire multiday intramuscular EMG (iEMG) signals (which are invasive) with higher Signal-to-Noise Ratio (SNR) compared to surface EMG (sEMG) signals; followed by application of matrix (non-negative matrix factorization – NMF) and tensor factorization methods (Canonical Polyadic Decomposition (CPD), Tucker decomposition (TD) &amp; Canonical Polyadic-Weighted Optimization (CP-WOPT)) for recovering structured missing data i.e., chunks of missing samples in channels. Furthermore, we tested the scalability of NMF, CPD, TD and CP-WOPT by employing them on the large multiday (seven days) iEMG data where the size of missing data is increased from day 1 to day 7, and for each day a fixed percentage of missing data is introduced from 10% to worst case of 50%. Results show that CP-WOPT outperformed NMF, CPD and TD to recover large percentage of missing data in terms of Relative Mean Error (RME) even when 7 days of data is considered. CP-WOPT showed robustness even for the worse case even when 50% iEMG data is removed from day 1 to day 7 where it’s RME degraded slightly from 0.08 to 0.1.

Coupled Heterogeneous Tucker Decomposition: A Feature Extraction Method for Multisource Fusion and Domain Adaptation Using Multisource Heterogeneous Remote Sensing Data

To excavate adequately the rich information contained in multisource remote sensing data, feature extraction as basic yet important research has two typical applications: one of which is to extract complementary information of multisource data to improve classification; and the other is to extract shared information across sources for domain adaptation. However, typical feature extraction methods require the input represented as vectors or homogeneous tensors and fail to process multisource data represented as heterogeneous tensors. Therefore, the coupled heterogeneous Tucker decomposition (C-HTD) containing two sub-methods, namely coupled factor matrix-based HTD (CFM-HTD) and coupled core tensor-based HTD (CCT-HTD), is proposed to establish a unified feature extraction framework for multisource fusion and domain adaptation. To handle multisource heterogeneous tensors, multiple Tucker models were constructed to extract features of different sources separately. To cope with the supervised and semi-supervised cases, the class-indicator factor matrix was built to enhance the separability of features using known labels and learned labels. To mine the complementarity of paired multisource samples, coupling constraint was imposed on multiple factor matrices to form CFM-HTD to extract multisource information jointly. To extract domain-adapted features, coupling constraint was imposed on multiple core tensors to form CCT-HTD to encourage data from different sources to have the same class centroid. In addition, to reduce the impact of interference samples on domain adaptation, an adaptive sample-weighting matrix was designed to autonomously remove outliers. Using multiresolution multiangle optical and MSTAR datasets, experimental results show that the C-HTD outperforms typical multisource fusion and domain adaptation methods.

Hyperspectral Image Denoising via Group Sparsity Regularized Hybrid Spatio-Spectral Total Variation

In this paper, we propose a new hyperspectral image (HSI) denoising model with the group sparsity regularized hybrid spatio-spectral total variation (GHSSTV) and low-rank tensor decomposition, which is based on the analysis of structural sparsity of HSIs. First, the global correlations among all modes are explored by the Tucker decomposition, which applies low-rank constraints to the clean HSIs. To avoid over-smoothing, we propose GHSSTV regularization to ensure the group sparsity not only in the first-order gradient domain but also in the second-order ones along the spatio-spectral dimensions. Then, the sparse noise in HSI can be detected by the ℓ1 norm. Furthermore, strong Gaussian noise is simulated by the Frobenius norm. The alternating direction multiplier method (ADMM) algorithm is employed to effectively solve the GHSSTV model. Finally, experimental results from a series of simulations and real-world data suggest a superior performance of the GHSSTV method in HSIs denoising.

Unsupervised and Self-Supervised Tensor Train for Change Detection in Multitemporal Hyperspectral Images

Remote sensing change detection (CD) using multitemporal hyperspectral images (HSIs) provides detailed information on spectral–spatial changes and is useful in a variety of applications such as environmental monitoring, urban planning, and disaster detection. However, the high dimensionality and low spatial resolution of HSIs do not only lead to expensive computation but also bring about inter-class homogeneity and inner-class heterogeneity. Meanwhile, labeled samples are difficult to obtain in reality as field investigation is expensive, which limits the application of supervised CD methods. In this paper, two algorithms for CD based on the tensor train (TT) decomposition are proposed and are called the unsupervised tensor train (UTT) and self-supervised tensor train (STT). TT uses a well-balanced matricization strategy to capture global correlations from tensors and can therefore effectively extract low-rank discriminative features, so the curse of the dimensionality and spectral variability of HSIs can be overcome. In addition, the two proposed methods are based on unsupervised and self-supervised learning, where no manual annotations are needed. Meanwhile, the ket-augmentation (KA) scheme is used to transform the low-order tensor into a high-order tensor while keeping the total number of entries the same. Therefore, high-order features with richer texture can be extracted without increasing computational complexity. Experimental results on four benchmark datasets show that the proposed methods outperformed their tensor counterpart, the tucker decomposition (TD), the higher-order singular value decomposition (HOSVD), and some other state-of-the-art approaches. For the Yancheng dataset, OA and KAPPA of UTT reached as high as 98.11% and 0.9536, respectively, while OA and KAPPA of STT were at 98.20% and 0.9561, respectively.

Complex Plant Process Trait Evaluation Through Decomposition of Higher-order Interaction: A Case Study in Acclimation Responses of Cold-climate Hybrid Grapevine Through Bilinear and Multiway Methods

Traditionally, the structure of higher-order data in genotype-by-environment interaction requires simplification to use bilinear reduction models. Flexible multiway reduction models have been claimed to be more informative, as they allow exploration of individual trends and account for the covariance among data modes. In complex latent traits, such as acclimation response of grapevine (Vitis sp.), these methods may offer increased insight into plant adaptive processes. In a growth chamber study, data from seven phenotypic traits at 11 photoperiodic times in the presence of two temperatures of 30 accessions were analyzed. The four-way interaction among these data modes was isolated and further examined through bilinear singular value decomposition (SVD) and multiway Tucker decomposition models. A similar set of three latent process traits were identified regardless of model used. The Tucker decomposition model led to more concise clustering of wild-type accessions, was more interpretable, as trends could be evaluated separately, and had less indication of overfitting; therefore, the multiway method was preferred over the standard SVD bilinear method in the investigation of high-order interaction in acclimation response. This methodology may offer insight into other complex traits, such as phenolic development, drought tolerance, and horizontal disease resistance to improve breeding efforts as other individual mechanisms used by the organism are separated, quantified, and compared rather than the culmination of events as an end-product.