Canonical Polyadic Decomposition Research Articles

Denoising multispectral images using non-local rank tensor decomposition and bilateral filtering based on sunflower optimization

ABSTRACT Image denoising is an important pre-processing process in the fields of computer vision and image processing. Traditional denoising techniques blur edges excessively and degrade image quality by removing noise components but failing to maintain clarity. To overcome these problems, this paper proposes a multispectral image denoising strategy combining non-local rank tensor decomposition (NLRTD) and bilateral filtering. To extract patches from noisy images, single-level discrete wavelet transform (DWT) is utilized. Then, similar patches from the extracted images are grouped using spectral clustering. After that, mixed noise is reduced by separating clean images from each clustered group using NLRTD. An optimized bilateral filter using Sunflower optimization (SFO) is used for denoising by preserving edge details and is reconstructed using its constituent parts. The effectiveness of the proposed denoising method is assessed using performance matrices, such as BER, PSNR, MSE, RMSE, SNR and SSIM were 0.8544%, 53.21%, 2.41%, 2.41%, 25.06% and 0.90%, respectively.

Robust Image Hashing via CP Decomposition and DCT for Copy Detection

Copy detection is a key task of image copyright protection. This article proposes a robust image hashing algorithm by CP decomposition and discrete cosine transform (DCT) for copy detection. The first contribution is the third-order tensor construction with low-frequency coefficients in the DCT domain. Since the low-frequency DCT coefficients contain most of the image energy, they can reflect the basic visual content of the image and are less disturbed by noise. Hence, the third-order tensor construction with the low-frequency DCT coefficients can ensure robustness of our algorithm. Another contribution is the application of the CP decomposition to the third-order tensor for learning a short binary hash. As the factor matrices learned from the CP decomposition can preserve the topology of the original tensor, the binary hash derived from the factor matrices can reach good discrimination. Lots of experiments and comparisons are done to validate effectiveness and advantage of our algorithm. The results demonstrate that our algorithm has superior classification and copy detection performances than several baseline algorithms. In addition, our algorithm is also better than some baseline algorithms with regard to hash length and computational time.

An inexactly accelerated algorithm for nonnegative tensor CP decomposition with the column unit constraints

An inexactly accelerated algorithm for nonnegative tensor CP decomposition with the column unit constraints

Fake News Classification Using Tensor Decomposition and Graph Convolutional Network

With the widespread of fake news on social media, its impact has become a major concern of the public, so accurate detection methods are urgently needed. As content-based features are the most natural clues for fake news detection, many methods that rely solely on text content have been proposed. However, these methods rarely investigate the sentence interaction patterns of different news articles, and most of them do not consider fine-grained fake news classification. To overcome these issues, this article constructs a graph representation for news articles and employs a graph neural network (GNN) to classify fake news. The proposed method uses the local word co-occurrence information of sentences to obtain the interaction relationship between sentences, which is abstracted by the weight matrix of the graph representation. Accordingly, a third-order co-occurrence tensor is built, and the weight matrix is calculated based on the canonical polyadic (CP) decomposition of this tensor. Since our method considers the sentence interaction patterns, the computed representations can capture more accurate contextual information of news articles. The results on two real-world datasets demonstrate that our method outperforms the competing methods in both binary and multiclass classification tasks. In particular, for multiclass classification on the selected dataset with <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$70$</tex-math> </inline-formula> % of the training set for training, the improvements of accuracy and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$F1$</tex-math> </inline-formula> -score are <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1.82$</tex-math> </inline-formula> %p– <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$16.98$</tex-math> </inline-formula> %p and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1.85$</tex-math> </inline-formula> %p– <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$16.65$</tex-math> </inline-formula> %p, respectively.

Frame Semantic Role Labeling Using Arbitrary-Order Conditional Random Fields

This paper presents an approach to frame semantic role labeling (FSRL), a task in natural language processing that identifies semantic roles within a text following the theory of frame semantics. Unlike previous approaches which do not adequately model correlations and interactions amongst arguments, we propose arbitrary-order conditional random fields (CRFs) that are capable of modeling full interaction amongst an arbitrary number of arguments of a given predicate. To achieve tractable representation and inference, we apply canonical polyadic decomposition to the arbitrary-order factor in our proposed CRF and utilize mean-field variational inference for approximate inference. We further unfold our iterative inference procedure into a recurrent neural network that is connected to our neural encoder and scorer, enabling end-to-end training and inference. Finally, we also improve our model with several techniques such as span-based scoring and decoding. Our experiments show that our approach achieves state-of-the-art performance in FSRL.

Underdetermined blind source separation based on third‐order cumulant and tensor compression

AbstractA method for Underdetermined Blind Source Separation is proposed using third‐order cumulants and tensor compression. To effectively suppress symmetrical distributed noise, the third‐order cumulant is considered. Additionally, the complexity of high‐dimensional tensors can be reduced through high order singular value decomposition (HOSVD) for compression purposes. The method begins by calculating the third‐order cumulant tensor for whitening signals at different time delays, and then stacks several cumulants into a fourth‐order tensor. The HOSVD decomposition is applied to the fourth‐order tensor, compressing the high‐dimensional tensor into a low‐dimensional core tensor. Next, the core tensor is further decomposed using the canonical polyadic decomposition, and the resulting factor matrices are fused to obtain an estimation of the mixed matrix. Finally, leveraging the signal independence, a matrix diagonalisation method is employed to recover the source signals. Theoretical analysis and simulation results demonstrate that the proposed method effectively suppresses the influence of Gaussian noise, reduces computational complexity, and saves computational time. Moreover, compared with five representative approaches, the proposed method achieves superior separation results. Specifically, for the 3 × 4 mixed model with a signal‐to‐noise ratio of 20 dB, the average relative error of speech signal and radio signal are −11.02 and −6.8 dB respectively.

Randomized two-sided subspace iteration for low-rank matrix and tensor decomposition

The low-rank approximation of big data matrices and tensors plays a pivotal role in many modern applications. Although, a truncated version of the singular value decomposition (SVD) furnishes the best approximation, its computation is challenging on modern, multicore architectures. Recently, the randomized subspace iteration has shown to be a powerful tool in approximating large-scale matrices. In this paper we present a two-sided variant of the randomized subspace iteration. Novel in our work is the exploitation of the unpivoted QR factorization, rather than the SVD, for factorizing the compressed matrix. Hence our algorithm is a randomized rank-revealing URV decomposition. We prove the rank-revealingness of our algorithm by establishing bounds for the singular values as well as the other blocks of the compressed matrix. We further provide bounds on the error of the low-rank approximations of the proposed algorithm, in both 2- and Frobenius norm. In addition, we employ the proposed algorithm to efficiently compute low rank tensor decompositions: we present two randomized algorithms, one for the truncated higher-order SVD, and the other for the tensor SVD. We conduct numerical tests on (i) various classes of matrices, and (ii) synthetic tensors and real datasets to demonstrate the efficacy of the proposed algorithms.

Spatiotemporal knowledge graph completion via diachronic and transregional word embedding

Knowledge Graph Completion (KGC) is an essential application in the field of knowledge graphs (KGs) that attempts to fill in the missing information in the process of KG modelling. With the popularity of temporal knowledge graphs (TKGs), a wide range of techniques based on temporal knowledge graph completion (TKGC) have appeared, solving the issue of real-world knowledge with temporal properties. However, there is little study on KGC with spatiotemporal attributes, some real-world data include both spatial and temporal attributes. Effectively handling the completion of missing entities or predicates in spatiotemporal knowledge graphs (STKGs) is an important challenge. Our study fills the gap in knowledge completion techniques in the field of STKG. We present a model for completion based on the well-known tensor factorization canonical polyadic (CP) decomposition. It introduces temporal and spatial attributes into the decomposition vector to achieve entity and link predictions. We name it diachronic and transregional word embedding (DT-WE), which includes two different modules: the embedding framework and the scoring module. Firstly, send the vectors to the embedding framework to get the new vector representation, then, we send it to the scoring module to compute, and finally, the resulting values are added together to compute the prediction probability. We conducted extensive experiments on three real-world STKGs: YAGO10K, Wikidata40K and Opensky. The results indicate that the newly introduced spatiotemporal attributes not only improve accuracy in predicting entities and predicates compared to temporal models but also achieve state-of-the-art performance with lower spatial complexity.

Stable Low-Rank CP Decomposition for Compression of Convolutional Neural Networks Based on Sensitivity

Modern convolutional neural networks (CNNs) play a crucial role in computer vision applications. The intricacy of the application scenarios and the growing dataset both significantly raise the complexity of CNNs. As a result, they are often overparameterized and have significant computational costs. One potential solution for optimizing and compressing the CNNs is to replace convolutional layers with low-rank tensor decomposition. The most suitable technique for this is Canonical Polyadic (CP) decomposition. However, there are two primary issues with CP decomposition that lead to a significant loss in accuracy. Firstly, the selection of tensor ranks for CP decomposition is an unsolved issue. Secondly, degeneracy and instability are common problems in the CP decomposition of contractional tensors, which makes fine-tuning the compressed model difficult. In this study, a novel approach was proposed for compressing CNNs by using CP decomposition. The first step involves using the sensitivity of convolutional layers to determine the tensor ranks for CP decomposition effectively. Subsequently, to address the degeneracy issue and enhance the stability of the CP decomposition, two novel techniques were incorporated: optimization with sensitivity constraints and iterative fine-tuning based on sensitivity order. Finally, the proposed method was examined on common CNN structures for image classification tasks and demonstrated that it provides stable performance and significantly fewer reductions in classification accuracy.

Compact sum-of-products form of the molecular electronic Hamiltonian based on canonical polyadic decomposition.

We propose an approach to represent the second-quantized electronic Hamiltonian in a compact sum-of-products (SOP) form. The approach is based on the canonical polyadic decomposition of the original Hamiltonian projected onto the sub-Fock spaces formed by groups of spin-orbitals. The algorithm for obtaining the canonical polyadic form starts from an exact sum-of-products, which is then optimally compactified using an alternating least squares procedure. We discuss the relation of this specific SOP with related forms, namely the Tucker format and the matrix product operator often used in conjunction with matrix product states. We benchmark the method on the electronic dynamics of an excited water molecule, trans-polyenes, and the charge migration in glycine upon inner-valence ionization. The quantum dynamics are performed with the multilayer multiconfiguration time-dependent Hartree method in second quantization representation. Other methods based on tree-tensor Ansätze may profit from this general approach.

Dynamic functional network connectivity analysis in schizophrenia based on a spatiotemporal CPD framework.

Objective.Dynamic functional network connectivity (dFNC), based on data-driven group independent component (IC) analysis, is an important avenue for investigating underlying patterns of certain brain diseases such as schizophrenia. Canonical polyadic decomposition (CPD) of a higher-way dynamic functional connectivity tensor, can offer an innovative spatiotemporal framework to accurately characterize potential dynamic spatial and temporal fluctuations. Since multi-subject dFNC data from sliding-window analysis are also naturally a higher-order tensor, we propose an innovative sparse and low-rank CPD (SLRCPD) for the three-way dFNC tensor to excavate significant dynamic spatiotemporal aberrant changes in schizophrenia.Approach.The proposed SLRCPD approach imposes two constraints. First, the L1regularization on spatial modules is applied to extract sparse but significant dynamic connectivity and avoid overfitting the model. Second, low-rank constraint is added on time-varying weights to enhance the temporal state clustering quality. Shared dynamic spatial modules, group-specific dynamic spatial modules and time-varying weights can be extracted by SLRCPD. The strength of connections within- and between-IC networks and connection contribution are proposed to inspect the spatial modules. K-means clustering and classification are further conducted to explore temporal group difference.Main results.82 subject resting-state functional magnetic resonance imaging (fMRI) dataset and opening Center for Biomedical Research Excellence (COBRE) schizophrenia dataset both containing schizophrenia patients (SZs) and healthy controls (HCs) were utilized in our work. Three typical dFNC patterns between different brain functional regions were obtained. Compared to the spatial modules of HCs, the aberrant connections among auditory network, somatomotor, visual, cognitive control and cerebellar networks in 82 subject dataset and COBRE dataset were detected. Four temporal states reveal significant differences between SZs and HCs for these two datasets. Additionally, the accuracy values for SZs and HCs classification based on time-varying weights are larger than 0.96.Significance.This study significantly excavates spatio-temporal patterns for schizophrenia disease.

Construction of a Mode-Combination Hamiltonian under the Grid-Based Representation for the Quantum Dynamics of OH + HO2 → O2 + H2O.

In this work, a systematic construction framework on a mode-combination Hamiltonian operator of a typical polyatomic reaction, OH + HO2 → O2 + H2O, is developed. First, a set of Jacobi coordinates are employed to construct the kinetic energy operator (KEO) through the polyspherical approach ( Phys. Rep. 2009, 484, 169). Second, due to the multiconfigurational electronic structure of this system, a non-adiabatic potential energy surface (PES) is constructed where the first singlet and triplet states are involved with spin-orbital coupling. To improve the training database, the training set of random energy data was optimized through a popular iterative optimization approach with extensive trajectories. Here, we propose an automatic trajectory method, instead of the classical trajectory on a crude PES, where the gradients are directly computed by the present ab initio calculations. Third, on the basis of the training set, the potential function is directly constructed in the canonical polyadic decomposition (CPD) form ( J. Chem. Theory Comput. 2021, 17, 2702-2713) which is helpful in propagating the nuclear wave function under the grid-based representation. To do this, the Gaussian process regression (GPR) approach for building the CPD form, called the CPD-GPR method ( J. Phys. Chem. Lett. 2022, 13, 11128-11135) is adopted where we further revise CPD-GPR by introducing the mode-combination (mc) scheme leading to the present CPD-mc-GPR approach. Constructing the full-dimension non-adiabatic Hamiltonian operator with mode combination, as test calculations, the nuclear wave function is propagated to preliminarily compute the reactive probability of OH + HO2 → O2 + H2O where the reactants are prepared in vibrational ground states and in the first triplet electronic state.

Operational modal analysis of under-determined system based on Bayesian CP decomposition (Translated)

Operational modal analysis of under-determined system based on Bayesian CP decomposition (Translated)

An Approach for Using a Tensor-Based Method for Mobility-User Pattern Determining

Modern mobile networks exhibit a complex heterogeneous structure. To enhance the Quality of Service (QoS) in these networks, intelligent control mechanisms should be implemented. These functions are based on the processing of large amounts of data and feature extraction. One such feature is information about user mobility. However, directly determining user mobility remains challenging. To address this issue, this study proposes an approach based on multi-linear data processing. The user mobility is proposed to determine, using the multi-linear data, about the changing of the Signal-to-Interference-plus-Noise-Ratio (SINR). SINR varies individually for each user over time, relative to the network’s base stations. It is natural to represent these data as a tensor. A tensor-based preprocessing step employing Canonical Polyadic Decomposition (CPD) is proposed to extract user mobility information and reduce the data volume. In the next step, using the DBSCAN algorithm, users are clustered according to their mobility patterns. Subsequently, users are clustered based on their mobility patterns using the DBSCAN algorithm. The proposed approach is evaluated utilizing data from Network Simulator 3 (NS-3), which simulates a portion of the mobile network. The results of processing these data using the proposed method demonstrate superior performance in determining user mobility.

Performance optimization and parameters estimation for MIMO-OFDM dual-functional communication-radar systems

Dual-function communication radar systems use common Radio Frequency (RF) signals are used for both communication and detection. For better compatibility with existing communication systems, we adopt Multiple-Input Multiple-Output (MIMO) Orthogonal Frequency Division Multiplexing (OFDM) signals as integrated signals and investigate the estimation performance of MIMO-OFDM signals. First, we analyze the Cramer-Rao Lower Bound (CRLB) of parameter estimation. Then, the transmit powers over different subcarriers are optimized to achieve the best tradeoff between the transmission rate and the estimation performance. Finally, we propose a more accurate estimation method that uses Canonical Polyadic Decomposition (CPD) of the third-order tensor to obtain the parameter matrices. Due to the characteristic of the column structure of the parameter matrices, we only need to use DFT / IDFT to recover the parameters of multiple targets. The simulation results show that tensor-based estimation method can achieve a performance close to CRLB, and the estimation performance can be improved by optimizing the transmit powers.

The tensor of the exact circle: reconstructing geometry

Developing a theory for quantum gravity is one of the big open questions in theoretical high-energy physics. Recently, a tensor model approach has been considered that treats tensors as the generators of commutative non-associative algebras, which might be an appropriate interpretation of the canonical tensor model. In this approach, the non-associative algebra is assumed to be a low-energy description of the so-called associative closure, which gives the full description of spacetime including the high-energy modes. In the previous work it has been shown how to (re)construct a topological space with a measure on it, and one of the prominent examples that was used to develop the framework was the exact circle. In this work we will further investigate this example, and show that it is possible to reconstruct the full Riemannian geometry by reconstructing the metric tensor. Furthermore, it is demonstrated how diffeomorphisms behave in this formalism, firstly by considering a specific class of diffeomorphisms of the circle, namely the ellipses, and subsequently by performing an explicit diffeomorphism to ‘smoothen’ sets of points generated by the tensor rank decomposition.

Quantum and classical molecular dynamics for H atom scattering from graphene.

This work presents systematic comparisons between classical molecular dynamics (cMD) and quantum dynamics (QD) simulations of 15-dimensional and 75-dimensional models in their description of H atom scattering from graphene. We use an experimentally validated full-dimensional neural network potential energy surface of a hydrogen atom interacting with a large cell of graphene containing 24 carbon atoms. For quantum dynamics simulations, we apply Monte Carlo canonical polyadic decomposition to transform the original potential energy surface (PES) into a sum of products form and use the multi-layer multi-configuration time-dependent Hartree method to simulate the quantum scattering of a hydrogen or deuterium atom with an initial kinetic energy of 1.96 or 0.96 eV and an incident angle of 0°, i.e., perpendicular to the graphene surface. The cMD and QD initial conditions have been carefully chosen in order to be as close as possible. Our results show little differences between cMD and QD simulations when the incident energy of the H atom is equal to 1.96 eV. However, a large difference in sticking probability is observed when the incident energy of the H atom is equal to 0.96 eV, indicating the predominance of quantum effects. To the best of our knowledge, our work provides the first benchmark of quantum against classical simulations for a system of this size with a realistic PES. Additionally, new projectors are implemented in the Heidelberg multi-configuration time-dependent Hartree package for the calculation of the atom scattering energy transfer distribution as a function of outgoing angles.

Dynamic Multimodal Fusion via Meta-Learning Towards Micro-Video Recommendation

Multimodal information (e.g., visual, acoustic, and textual) has been widely used to enhance representation learning for micro-video recommendation. For integrating multimodal information into a joint representation of micro-video, multimodal fusion plays a vital role in the existing micro-video recommendation approaches. However, the static multimodal fusion used in previous studies is insufficient to model the various relationships among multimodal information of different micro-videos. In this article, we develop a novel meta-learning-based multimodal fusion framework called Meta Multimodal Fusion (MetaMMF), which dynamically assigns parameters to the multimodal fusion function for each micro-video during its representation learning. Specifically, MetaMMF regards the multimodal fusion of each micro-video as an independent task. Based on the meta information extracted from the multimodal features of the input task, MetaMMF parameterizes a neural network as the item-specific fusion function via a meta learner. We perform extensive experiments on three benchmark datasets, demonstrating the significant improvements over several state-of-the-art multimodal recommendation models, like MMGCN, LATTICE, and InvRL. Furthermore, we lighten our model by adopting canonical polyadic decomposition to improve the training efficiency, and validate its effectiveness through experimental results. Codes are available at https://github.com/hanliu95/MetaMMF .

Alternating Mahalanobis Distance Minimization for Accurate and Well-Conditioned CP Decomposition

Alternating Mahalanobis Distance Minimization for Accurate and Well-Conditioned CP Decomposition

A Contemporary and Comprehensive Survey on Streaming Tensor Decomposition

Tensor decomposition has been demonstrated to be successful in a wide range of applications, from neuroscience and wireless communications to social networks. In an online setting, factorizing tensors derived from multidimensional data streams is however nontrivial due to several inherent problems of real-time stream processing. In recent years, many research efforts have been dedicated to developing online techniques for decomposing such tensors, resulting in significant advances in streaming tensor decomposition or tensor tracking. This topic is emerging and enriches the literature on tensor decomposition, particularly from the data stream analystics perspective. Thus, it is imperative to carry out an overview of tensor tracking to help researchers and practitioners understand its developments and achievements, summarise the current trends and advances, and identify challenging problems. In this article, we provide a contemporary and comprehensive survey on different types of tensor tracking techniques. We particularly categorize the state-of-the-art methods into three main groups: streaming CP decompositions, streaming Tucker decompositions, and streaming decompositions under other tensor formats (i.e., tensor-train, t-SVD, and BTD). In each group, we further divide the existing algorithms into sub-categories based on their main optimization framework and model architectures. Finally, we present several applications, research challenges, open problems, and potential directions of tensor tracking in the future.