Nonnegative Tensor Factorization Research Articles

HyperNTF: A hypergraph regularized nonnegative tensor factorization for dimensionality reduction

Tensor decomposition is an effective tool for learning multi-way structures and heterogeneous features from high-dimensional data, such as the multi-view images and multichannel electroencephalography (EEG) signals, are often represented by tensors. However, most of tensor decomposition methods are the linear feature extraction techniques, which are unable to reveal the nonlinear structure within high-dimensional data. To address such problem, a lot of algorithms have been proposed for simultaneously performs linear and non-linear feature extraction. A representative algorithm is the Graph Regularized Nonnegative Matrix Factorization (GNMF) for image clustering. However, the normal 2-order graph can only model the pairwise similarity of objects, which cannot sufficiently exploit the complex structures of samples. Thus, we propose a novel method, named Hypergraph Regularized Nonnegative Tensor Factorization (HyperNTF), which utilizes hypergraph to model the complex connections among samples and employs the factor matrix corresponding with last mode of Canonical Polyadic (CP) decomposition as low-dimensional representation of original data. Extensive experiments on synthetic manifolds, real-world image datasets, and EEG signals, demonstrating that HyperNTF outperforms the state-of-the-art methods in terms of dimensionality reduction, clustering, and classification.

A sequential two-step priming scheme reproduces diversity in synaptic strength and short-term plasticity

Glutamatergic synapses display variable strength and diverse short-term plasticity (STP), even for a given type of connection. Using nonnegative tensor factorization and conventional state modeling, we demonstrate that a kinetic scheme consisting of two sequential and reversible steps of release-machinery assembly and a final step of synaptic vesicle (SV) fusion reproduces STP and its diversity among synapses. Analyzing transmission at the calyx of Held synapses reveals that differences in synaptic strength and STP are not primarily caused by variable fusion probability (pfusion) but are determined by the fraction of docked synaptic vesicles equipped with a mature release machinery. Our simulations show that traditional quantal analysis methods do not necessarily report pfusion of SVs with a mature release machinery but reflect both pfusion and the distribution between mature and immature priming states at rest. Thus, the approach holds promise for a better mechanistic dissection of the roles of presynaptic proteins in the sequence of SV docking, two-step priming, and fusion. It suggests a mechanism for activity-induced redistribution of synaptic efficacy.

Fast multiplicative algorithms for symmetric nonnegative tensor factorization

Symmetric nonnegative tensor factorization (SNTF) is an important tool for clustering analysis. To date, most of algorithms for SNTF are based on multiplicative update rules, which have many attractive properties, e.g., they are often simple to implement and can enforce nonnegativity without extra projection steps. However, the existing multiplicative algorithms often converge slowly due to the conservative multiplicative learning steps or the use of low-level BLAS (basic linear algebra subprograms) in their implementation. In this paper, three new multiplicative algorithms are proposed for SNTF to overcome the drawback of slow convergence. First, a parallel multiplicative algorithm, which can be implemented with high-level BLAS, is derived by auxiliary optimization. To further accelerate the convergence, two new parallel multiplicative algorithms, which enable larger learning steps for improving efficiency, are developed based on weighted geometric mean and weighted arithmetic mean, respectively. Finally, we apply the proposed algorithms to multiway probabilistic clustering, where a new hyper-stochastic normalization scheme based on Euclidean distance is developed for better data preprocessing. The experiment results on both synthetic and real-world data show that the proposed SNTF algorithms converge faster than the state-of-the-art algorithms.

Tensor electrical impedance myography identifies bulbar disease progression in amyotrophic lateral sclerosis

ObjectiveElectrical impedance myography (EIM) is a promising biomarker for amyotrophic lateral sclerosis (ALS). A key issue is how best to utilise the complex high dimensional, multi-frequency data output by EIM to fully characterise the progression of disease. MethodsMuscle volume conduction properties were obtained from EIM recordings of the tongue across three electrode configurations and 14 input frequencies (76 Hz–625 kHz). Analyses of individual frequencies, averaged EIM spectra and non-negative tensor factorisation were undertaken. Longitudinal data were collected from 28 patients and 17 healthy volunteers at 3-monthly intervals for a maximum of 9 months. EIM was evaluated against the Amyotrophic Lateral Sclerosis Functional Rating Scale-Revised (ALSFRS-R) bulbar sub-score, tongue strength and an overall bulbar disease burden score. ResultsLongitudinal changes to individual patient EIM spectra demonstrated complex shifts in the spectral shape. At a group level, a clear pattern emerged over time, characterised by an increase in centre frequency and general shift to the right of the spectral shape. Tensor factorisation reduced the spectral data from a total of 168 data points per participant per recording to a single value which captured the complexity of the longitudinal data and which we call tensor EIM (T-EIM). The absolute change in tensor EIM significantly increased within 3 months and continued to do so over the 9-month study duration. In a hypothetical clinical trial scenario tensor EIM required fewer participants (n = 64 at 50% treatment effect), than single frequency measures (n range 87–802) or ALSFRS-R bulbar subscore (n = 298). ConclusionsChanges to tongue EIM spectra over time in ALS are complex. Tensor EIM captured and quantified disease progression and was more sensitive to changes than single frequency EIM measures and other biomarkers of bulbar disease. SignificanceObjective biomarkers for the assessment of bulbar disease in ALS are lacking. Tensor EIM enhances the biomarker potential of EIM data and can improve bulbar symptom monitoring in clinical trials.

Explainability of the COVID-19 epidemiological model with nonnegative tensor factorization

The world is witnessing the devastating effects of the COVID-19 pandemic. Each country responded to contain the spread of the virus in the early stages through diverse response measures. Interpreting these responses and their patterns globally is essential to inform future responses to COVID-19 variants and future pandemics. A stochastic epidemiological model (SEM) is a well-established mathematical tool that helps to analyse the spread of infectious diseases through communities and the effects of various response measures. However, interpreting the outcome of these models is complex and often requires manual effort. In this paper, we propose a novel method to provide the explainability of an epidemiological model. We represent the output of SEM as a tensor model. We then apply nonnegative tensor factorization (NTF) to identify patterns of global response behaviours of countries and cluster the countries based on these patterns. We interpret the patterns and clusters to understand the global response behaviour of countries in the early stages of the pandemic. Our experimental results demonstrate the advantage of clustering using NTF and provide useful insights into the characteristics of country clusters.

Nonnegative canonical tensor decomposition with linear constraints: nnCANDELINC

AbstractThere is an emerging interest for tensor factorization applications in big‐data analytics and machine learning. To speed up the factorization of extra‐large datasets, organized in multidimensional arrays (also known as tensors), easy to compute compression‐based tensor representations, such as, Tucker and tensor train formats, are used to approximate the initial large‐tensor. Further, tensor factorization is used to extract latent features that can facilitate discoveries of new mechanisms and signatures hidden in the data, where the explainability of the latent features is of principal importance. Nonnegative tensor factorization extracts latent features that are naturally sparse and parts of the data, which makes them easily interpretable. However, to take into account available domain knowledge and subject matter expertise, often additional constraints need to be imposed, which lead us to canonical decomposition with linear constraints (CANDELINC), a canonical polyadic decomposition with rank deficient factors. In CANDELINC, Tucker compression is used as a preprocessing step, which lead to a larger residual error but to more explainable latent features. Here, we propose a nonnegative CANDELINC (nnCANDELINC) accomplished via a specific nonnegative Tucker decomposition; we refer to as minimal or canonical nonnegative Tucker. We derive several results required to understand the specificity of nnCANDELINC, focusing on the difficulties of preserving the nonnegative rank of a tensor to its Tucker core and comparing the real valued to nonnegative case. Finally, we demonstrate nnCANDELINC performance on synthetic and real‐world examples.

Sparse Nonnegative Tensor Factorization and Completion With Noisy Observations

In this paper, we study the sparse nonnegative tensor factorization and completion problem from partial and noisy observations for third-order tensors. Because of sparsity and nonnegativity, the underlying tensor is decomposed into the tensor-tensor product of one sparse nonnegative tensor and one nonnegative tensor. We propose to minimize the sum of the maximum likelihood estimation for the observations with nonnegativity constraints and the tensor <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\ell _{0}$ </tex-math></inline-formula> norm for the sparse factor. We show that the error bounds of the estimator of the proposed model can be established under general noise observations. The detailed error bounds under specific noise distributions including additive Gaussian noise, additive Laplace noise, and Poisson observations can be derived. Moreover, the minimax lower bounds are shown to be matched with the established upper bounds up to a logarithmic factor of the sizes of the underlying tensor. These theoretical results for tensors are better than those obtained for matrices, and this illustrates the advantage of the use of nonnegative sparse tensor models for completion and denoising. Numerical experiments are provided to validate the superiority of the proposed tensor-based method compared with the matrix-based approach.

Fast Nonnegative Tensor Factorizations with Tensor Train Model

Fast Nonnegative Tensor Factorizations with Tensor Train Model

Weighted Group Sparsity-Constrained Tensor Factorization for Hyperspectral Unmixing

Recently, unmixing methods based on nonnegative tensor factorization have played an important role in the decomposition of hyperspectral mixed pixels. According to the spatial prior knowledge, there are many regularizations designed to improve the performance of unmixing algorithms, such as the total variation (TV) regularization. However, these methods mostly ignore the similar characteristics among different spectral bands. To solve this problem, this paper proposes a group sparse regularization that uses the weighted constraint of the L2,1 norm, which can not only explore the similar characteristics of the hyperspectral image in the spectral dimension, but also keep the data smooth characteristics in the spatial dimension. In summary, a non-negative tensor factorization framework based on weighted group sparsity constraint is proposed for hyperspectral images. In addition, an effective alternating direction method of multipliers (ADMM) algorithm is used to solve the algorithm proposed in this paper. Compared with the existing popular methods, experiments conducted on three real datasets fully demonstrate the effectiveness and advancement of the proposed method.

Hyperspectral Unmixing Using Nonlocal Similarity-Regularized Low-Rank Tensor Factorization

Recently, methods based on nonnegative tensor factorization (NTF), which benefits from the tensor representation of hyperspectral imagery (HSI) without any information loss, have attracted increasing attention. However, most existing methods fail to explore the internal spatial structure of data, resulting in low unmixing performance. Moreover, when the algorithm is optimized, the solution is unstable. In this article, a regularizer based on nonlocal tensor similarity is proposed, which can not only fully preserve the global information of HSI but also mine the internal information of data in the spatial domain. HSI is regarded as a 3-D tensor and is directly subjected to endmember extraction and abundance estimation. To fully explore the structural characteristics of data, we simultaneously use the local smoothing and low tensor rank prior of the data to constrain the unmixing model. First, several 4-D tensor groups can be obtained after the nonlocal similarity structure of HSI is learned. Subsequently, a low tensor rank prior is applied to each 4-D tensor, which can fully simulate the nonlocal similarity in the image. In addition, total variation (TV) is also used to explore the local spatial relationship of data, which can generate a smooth abundance map through edge preservation. The optimization is solved by the ADMM algorithm. Experiments on synthetic and real data illustrate the superiority of the proposed method.

Detecting Time-Evolving Phenotypic Components of Adverse Reactions Against BNT162b2 mRNA SARS-CoV-2 Vaccine Via Non-Negative Tensor Factorization

Detecting Time-Evolving Phenotypic Components of Adverse Reactions Against BNT162b2 mRNA SARS-CoV-2 Vaccine Via Non-Negative Tensor Factorization

Efficient Weighted-Adaptive Sparse Constrained Nonnegative Tensor Factorization for Hyperspectral Unmixing

Hyperspectral unmixing aims to separate pure materials and their corresponding proportions that constitute the mixed pixels of hyperspectral imagery (HSI). Recently, the matrix-vector nonnegative tensor factorization (MV-NTF) has attracted wide attention in this field due to its natural third-tensor representation of HSI. However, the NTF-based unmixing approaches are limited to the nonunique solution and long computational time. To solve these problems, we consider the low-rank and sparse priors of each abundance map simultaneously, and propose a new unmixing model adopting a weighted nuclear norm and an <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$L_{1/2}$</tex-math></inline-formula> norm under the MV-NTF framework. Instead of using low-rank matrix decomposition of MV-NTF, this model imposes the low-rank property on the whole abundance map, which avoids determining the rank of the abundance map in advance. Observe that each abundance map is different, we build an adaptive update mechanism to treat each low-rank and sparse constraint differently in the model. Furthermore, a new multiplicative iterative algorithm is designed to solve the proposed model. Specially, the algorithm designed for this tensor model is simplified by using the equivalence relation with nonnegative matrix factorization (NMF). Experiments demonstrate that the proposed method is effective in improving both the unmixing effect and the solving speed.

Structure-Preserving Joint Non-negative Tensor Factorization to Identify Reaction Pathways Using Bayesian Networks.

Extracting meaningful information from spectroscopic data is key to species identification as a first step to monitoring chemical reactions in unknown complex mixtures. Spectroscopic data obtained over multiple process modes (temperature, residence time) from different sensors [Fourier transform infrared (FTIR), proton nuclear magnetic resonance (1H NMR)] comprise hidden complementary information of the underlying chemical system. This work proposes an approach to jointly capture these hidden patterns in a structure-preserving and interpretable manner using coupled non-negative tensor factorization to achieve uniqueness in decomposition. Projections onto the modes of spectral channels, specific to each sensor, are interpreted as pseudo-component spectra, while projections onto the shared process modes are interpreted as the corresponding pseudo-component concentrations across temperature and residence times. Causal structure inference among these pseudo-component spectra (using Bayesian networks) is then used to identify plausible reaction pathways among the identified species representing each pseudo-component. Tensor decomposition of the FTIR data enables the development of reaction sequences based on the identified functional groups, while that of 1H NMR by itself is lacking in mechanism development as it solely reveals the proton environments in a pseudo-component. However, jointly parsing spectra from both the sensors is seen to capture complementary information, wherein insights into the proton environment from 1H NMR disambiguate pseudo-components that have similar FTIR peaks. A scalable method of parallelizing tensor decomposition to handle high-dimensional modes in process data by using grid tensor factorization, while being robust to process data artifacts like outliers, noise, and missing data, has been developed.

A2Sign: Agnostic Algorithms for Signatures-a universal method for identifying molecular signatures from transcriptomic datasets prior to cell-type deconvolution.

Molecular signatures are critical for inferring the proportions of cell types from bulk transcriptomics data. However, the identification of these signatures is based on a methodology that relies on prior biological knowledge of the cell types being studied. When working with less known biological material, a data-driven approach is required to uncover the underlying classes and generate ad hoc signatures from healthy or pathogenic tissue. We present a new approach, A2Sign: Agnostic Algorithms for Signatures, based on a non-negative tensor factorization (NTF) strategy that allows us to identify cell-type-specific molecular signatures, greatly reduce collinearities and also account for inter-individual variability. We propose a global framework that can be applied to uncover molecular signatures for cell-type deconvolution in arbitrary tissues using bulk transcriptome data. We also present two new molecular signatures for deconvolution of up to 16 immune cell types using microarray or RNA-seq data. All steps of our analysis were implemented in annotated Python notebooks (https://github.com/paulfogel/A2SIGN). To perform NTF, we used the NMTF package, which can be downloaded using Python pip install. Supplementary data are available at Bioinformatics online.

Spatial feature extraction non-negative tensor factorization for hyperspectral unmixing

Estimating endmembers and corresponding abundances from mixed pixels are essential steps for hyperspectral unmixing. In hyperspectral unmixing, obtaining accurate unmixing results is difficult since less prior knowledge is available. Besides, the unmixing results are influenced by noise and highly correlated endmembers, so that the obtained abundance maps exist small values which are not present in the image. In this paper, we separate each abundance map into a feature layer and a sparse layer to protect the obtained abundance maps from the above-mentioned factors. The feature layer represents the main information of the abundance map. And the sparse layer contains outliers dominated by the above-mentioned factors. In particular, we design a feature extraction regularization to describe the feature layer and use a weighted ℓ1 norm to describe the sparse layer. Then, under the framework of non-negative tensor factorization (NTF), we propose a novel unmixing algorithm named spatial feature extraction NTF (SFE-NTF) for hyperspectral unmixing. The proposed SFE-NTF is based on an augmented multiplicative algorithm. Experimental results on both synthetic and real hyperspectral data demonstrate that the proposed algorithm outperforms other state-of-the-art algorithms.

Comparative Analysis of Spatial–Temporal Distribution between Traditional Taxi Service and Emerging Ride-Hailing

The paradigms of taxis and ride-hailing, the two major players in the personal mobility market, are compared systematically and empirically in a unified spatial–temporal context. Supported by real field data from Xiamen, China, this research proposes a three-fold analytical framework to compare their mobilities, including (1) the spatial distributions of departures and arrivals by rank–size and odds ratio analysis, (2) the statistical characteristics of trip distances by spatial statistics and considering distance-decay effect, and (3) the meta-patterns inherent in the mobility processes by nonnegative tensor factorization. Our findings suggest that taxis and ride-hailing services share similar spatial patterns in terms of travel demand, but taxi demand heterogenizes more quickly with changes in population density. Additionally, the relative balance between the taxi industry and ride-hailing services shows opposite trends inside and outside Xiamen Island. Although the trip distances have similar statistical properties, the spatial distribution of the median trip distances reflects different urban structures. The meta-patterns detected from the origin–destination-time system via tensor factorization suggest that taxi mobilities feature exclusive nighttime intensities, whereas ride-hailing exhibits more prominent morning peaks on weekdays. Although ride-hailing contributes significantly to cross–strait interactions during daytime, there is a lack of efficient services to maintain such interactions at night.

Tensor electrical impedance myography identifies clinically relevant features in amyotrophic lateral sclerosis

Objective. Electrical impedance myography (EIM) shows promise as an effective biomarker in amyotrophic lateral sclerosis (ALS). EIM applies multiple input frequencies to characterise muscle properties, often via multiple electrode configurations. Herein, we assess if non-negative tensor factorisation (NTF) can provide a framework for identifying clinically relevant features within a high dimensional EIM dataset. Approach. EIM data were recorded from the tongue of healthy and ALS diseased individuals. Resistivity and reactivity measurements were made for 14 frequencies, in three electrode configurations. This gives 84 (2 × 14 × 3) distinct data points per participant. NTF was applied to the dataset for dimensionality reduction, termed tensor EIM. Significance tests, symptom correlation and classification approaches were explored to compare NTF to using all raw data and feature selection. Main Results. Tensor EIM provides highly significant differentiation between healthy and ALS patients (p < 0.001, AUROC = 0.78). Similarly tensor EIM differentiates between mild and severe disease states (p < 0.001, AUROC = 0.75) and significantly correlates with symptoms (ρ = 0.7, p < 0.001). A trend of centre frequency shifting to the right was identified in diseased spectra, which is in line with the electrical changes expected following muscle atrophy. Significance. Tensor EIM provides clinically relevant metrics for identifying ALS-related muscle disease. This procedure has the advantage of using the whole spectral dataset, with reduced risk of overfitting. The process identifies spectral shapes specific to disease allowing for a deeper clinical interpretation.

Localizing hot spots in Poisson radiation data matrices: nonnegative tensor factorization and phase congruency

Detecting and delineating hot spots in data from radiation sensors is required in applications ranging from monitoring large geospatial areas to imaging small objects in close proximity. This paper describes a computational method for localizing potential hot spots in matrices of independent Poisson data where, in numerical terms, a hot spot is a cluster of locally higher sample mean values (higher Poisson intensity) embedded in lower sample mean values (lower background intensity). Two numerical algorithms are computed sequentially for a 3D array of 2D matrices of gross Poisson counts: (1) nonnegative tensor factorization of the 3D array to maximize a Poisson likelihood and (2) phase congruency in pertinent matrices. The indicators of potential hot spots are closed contours in phase congruency in these matrices. The method is illustrated for simulated Poisson radiation datasets, including visualization of the phase congruency contours. The method may be useful in other applications in which there are matrices of nonnegative counts, provided that a Poisson distribution fits the dataset.

Deep non-negative tensor factorization with multi-way EMG data

Deep non-negative tensor factorization with multi-way EMG data

Identifying Covid-19 misinformation tweets and learning their spatio-temporal topic dynamics using Nonnegative Coupled Matrix Tensor Factorization.

Social media platforms like Twitter have become an easy portal for billions of people to connect and exchange their thoughts. Unfortunately, people commonly use these platforms to share misinformation which can influence other people adversely. The spread of misinformation is unavoidable in an extraordinary situation like Covid-19, and the consequences can be dreadful. This paper proposes a two-step ranking-based misinformation detection (RMiD) technique. Firstly, a novel ranking-based approach leveraging the scalable information retrieval infrastructure is applied to detect misinformation from a huge collection of unlabelled tweets based on a related but very small labelled misinformation data set. Secondly, the identified misinformation tweets are represented as a coupled matrix tensor model and Nonnegative Coupled Matrix Tensor Factorization is applied to learn their spatio-temporal topic dynamics. The experimental analysis shows that RMiD is capable of detecting misinformation with better coverage and less noise in comparison with existing techniques. Moreover, the coupled matrix tensor representation has improved the quality of topics discovered from unlabelled data up to 4% by leveraging the semantic similarity of terms in labelled data.Supplementary InformationThe online version supplementary material available at 10.1007/s13278-021-00767-7.