Nonnegative Tensor Factorization Research Articles

CS2DIPs: Unsupervised HSI Super-Resolution Using Coupled Spatial and Spectral DIPs.

In recent years, fusing high spatial resolution multispectral images (HR-MSIs) and low spatial resolution hyperspectral images (LR-HSIs) has become a widely used approach for hyperspectral image super-resolution (HSI-SR). Various unsupervised HSI-SR methods based on deep image prior (DIP) have gained wide popularity thanks to no pre-training requirement. However, DIP-based methods often demonstrate mediocre performance in extracting latent information from the data. To resolve this performance deficiency, we propose a coupled spatial and spectral deep image priors (CS2DIPs) method for the fusion of an HR-MSI and an LR-HSI into an HR-HSI. Specifically, we integrate the nonnegative matrix-vector tensor factorization (NMVTF) into the DIP framework to jointly learn the abundance tensor and spectral feature matrix. The two coupled DIPs are designed to capture essential spatial and spectral features in parallel from the observed HR-MSI and LR-HSI, respectively, which are then used to guide the generation of the abundance tensor and spectral signature matrix for the fusion of the HSI-SR by mode-3 tensor product, meanwhile taking some inherent physical constraints into account. Free from any training data, the proposed CS2DIPs can effectively capture rich spatial and spectral information. As a result, it exhibits much superior performance and convergence speed over most existing DIP-based methods. Extensive experiments are provided to demonstrate its state-of-the-art overall performance including comparison with benchmark peer methods.

On Ambisonic Source Separation with Spatially Informed Non-negative Tensor Factorization

On Ambisonic Source Separation with Spatially Informed Non-negative Tensor Factorization

New Penalized Criteria for Smooth Non-Negative Tensor Factorization With Missing Entries

New Penalized Criteria for Smooth Non-Negative Tensor Factorization With Missing Entries

Multi-faceted analysis and prediction for the outbreak of pediatric respiratory syncytial virus.

Respiratory syncytial virus (RSV) is a significant cause of pediatric hospitalizations. This article aims to utilize multisource data and leverage the tensor methods to uncover distinct RSV geographic clusters and develop an accurate RSV prediction model for future seasons. This study utilizes 5-year RSV data from sources, including medical claims, CDC surveillance data, and Google search trends. We conduct spatiotemporal tensor analysis and prediction for pediatric RSV in the United States by designing (i) a nonnegative tensor factorization model for pediatric RSV diseases and location clustering; (ii) and a recurrent neural network tensor regression model for county-level trend prediction using the disease and location features. We identify a clustering hierarchy of pediatric diseases: Three common geographic clusters of RSV outbreaks were identified from independent sources, showing an annual RSV trend shifting across different US regions, from the South and Southeast regions to the Central and Northeast regions and then to the West and Northwest regions, while precipitation and temperature were found as correlative factors with the coefficient of determination R2≈0.5, respectively. Our regression model accurately predicted the 2022-2023 RSV season at the county level, achieving R2≈0.3 mean absolute error MAE < 0.4 and a Pearson correlation greater than 0.75, which significantly outperforms the baselines with P-values <.05. Our proposed framework provides a thorough analysis of RSV disease in the United States, which enables healthcare providers to better prepare for potential outbreaks, anticipate increased demand for services and supplies, and save more lives with timely interventions.

Extraction and Prediction of User Communication Behaviors From DNS Query Logs Based on Nonnegative Tensor Factorization

Owing to the critical role of the domain name system (DNS), its query log data are utilized for various network monitoring purposes. With the diversification of network services, these data have become increasingly complex, making mining useful information challenging. DNS query log data can be considered as the superposition of two types of communication patterns: groups of domains accessed simultaneously (e.g., ad servers and content delivery network (CDN) servers) and time-series access patterns based on user behavior characteristics (e.g., access trends during the night). However, previous studies have not focused on extracting both access patterns hidden in the data. This study proposes a method that extracts both patterns of accessed domains and temporal access patterns as user communication behaviors from DNS query log data and predicts future accesses based on these patterns. The proposed method first aggregates similar fully qualified domain names (FQDNs) associated with the same service. We then present temporal regularized nonnegative tensor factorization (TR-NTF) that extracts both access patterns from a third-order tensor expressing DNS query log data and enables prediction. We evaluate the proposed method using synthetic and actual data and demonstrate that it successfully extracts hidden communication patterns and achieves sufficient prediction accuracy.

Study of performance of low-rank nonnegative tensor factorization methods

Abstract In the present paper we compare two different iterative approaches to constructing nonnegative tensor train and Tucker decompositions. The first approach is based on idea of alternating projections and randomized sketching for factorization of tensors with nonnegative elements. This approach can be useful for both TT and Tucker formats. The second approach consists of two stages. At the first stage we find the unconstrained tensor train decomposition for the target array. At the second stage we use this initial approximation in order to fix it within moderate number of operations and obtain the factorization with nonnegative factors either in tensor train or Tucker model. We study the performance of these methods for both synthetic data and hyper-spectral image and demonstrate the clear advantage of the latter technique in terms of computational time and wider range of possible applications.

Matrix Factorization for Globally Consistent Periodic Flow Prediction in Taxi Systems

Predicting flow in taxi systems can improve taxi operations and reduce passengers’ waiting time. With the increasing availability of taxi mobility data sets, more studies have tried to analyze and model taxi demand. However, most existing works merely attempt to predict the number of pick-ups and drop-offs by using local observations. Nevertheless, predicting the passengers’ pair-wise flow specifically through simultaneous consideration of local variations and network communities in the taxi systems has been neglected. In this paper, we introduce a globally consistent periodic flow prediction for taxi systems that integrates communities in the matrix factorization model. We apply non-negative tensor factorization to capture the periodic variations in the passenger flow in different areas and predict the flow (and pick-ups and drop-offs) in the next periods by integrating the most recent observation with detected patterns from historical data. The results show an improvement in the prediction accuracy from some baselines and a state-of-the-art method. We also propose a method for visualizing flow that potentially improves taxi operations by assisting drivers with selecting passengers.

Non-negative tensor factorization for vibration-based local damage detection

In this study, a novel non-negative tensor factorization (NTF)-based method for vibration-based local damage detection in rolling element bearings is proposed. As the diagnostic signal registered from a faulty machine is non-stationary, the time–frequency method is frequently used as a primary decomposition technique. It is proposed here to extract multi-linear NTF-based components from a 3D array of time–frequency representations of an observed signal partitioned into blocks. As a result, frequency and temporal informative components can be efficiently separated from non-informative ones. The experiments performed on synthetic and real signals demonstrate the high efficiency of the proposed method with respect to the already known non-negative matrix factorization approach.

Endmembers compression based nonnegative matrix factorization for hyperspectral unmixing

ABSTRACT Hyperspectral unmixing (HU) has grown more important in hyperspectral images (HSIs) research in recent years. Many models based on nonnegative matrix factorization (NMF) and nonnegative tensor factorization (NTF) have been extensively used to tackle the HU problem. Most of these models allow a large and extensive exploration of abundance and provide promising unmixing results. However, the exploration of each endmember information is insufficient. In this article, we propose an unmixing model called endmembers compressionbased NMF (EC-NMF) for HU. Specifically, by an orthogonal projection, the EC-NMF enables distinct endmembers to be differentiated from each other. In this space, each compressed endmember represents the distinct characteristics of the corresponding endmember, which makes it easier for the endmembers separated. Furthermore, the EC-NMF model incorporates the reweighted nuclear norm on the abundance map to fully account for spatial information and to constrain its low-rank properties. The EC-NMF model is solved by an augmented multiplicative algorithm. In particular, we achieve the proposed model based on the matrix representation, which reduces the computational complexity compared with NTF-based methods. Extensive experiments on synthetic and real data give concrete evidence that the proposed method outperforms other known methods in both the unmixing performance and running time.

General-purpose Unsupervised Cyber Anomaly Detection via Non-negative Tensor Factorization

Distinguishing malicious anomalous activities from unusual but benign activities is a fundamental challenge for cyber defenders. Prior studies have shown that statistical user behavior analysis yields accurate detections by learning behavior profiles from observed user activity. These unsupervised models are able to generalize to unseen types of attacks by detecting deviations from normal behavior without knowledge of specific attack signatures. However, approaches proposed to date based on probabilistic matrix factorization are limited by the information conveyed in a two-dimensional space. Non-negative tensor factorization, however, is a powerful unsupervised machine learning method that naturally models multi-dimensional data, capturing complex and multi-faceted details of behavior profiles. Our new unsupervised statistical anomaly detection methodology matches or surpasses state-of-the-art supervised learning baselines across several challenging and diverse cyber application areas, including detection of compromised user credentials, botnets, spam e-mails, and fraudulent credit card transactions.

Hyperspectral unmixing with LiDAR-Based total variation regularized non-negative tensor factorization

Hyperspectral unmixing with LiDAR-Based total variation regularized non-negative tensor factorization

Hyperspectral unmixing with LiDAR-Based total variation regularized non-negative tensor factorization

Hyperspectral unmixing with LiDAR-Based total variation regularized non-negative tensor factorization

Detecting time-evolving phenotypic components of adverse reactions against BNT162b2 SARS-CoV-2 vaccine via non-negative tensor factorization

SummarySymptoms of adverse reactions to vaccines evolve over time, but traditional studies have focused only on the frequency and intensity of symptoms. Here, we attempt to extract the dynamic changes in vaccine adverse reaction symptoms as a small number of interpretable components by using non-negative tensor factorization. We recruited healthcare workers who received two doses of the BNT162b2 mRNA COVID-19 vaccine at Chiba University Hospital and collected information on adverse reactions using a smartphone/web-based platform. We analyzed the adverse-reaction data after each dose obtained for 1,516 participants who received two doses of vaccine. The non-negative tensor factorization revealed four time-evolving components that represent typical temporal patterns of adverse reactions for both doses. These components were differently associated with background factors and post-vaccine antibody titers. These results demonstrate that complex adverse reactions against vaccines can be explained by a limited number of time-evolving components identified by tensor factorization.

Predicting multiple types of miRNA-disease associations using adaptive weighted nonnegative tensor factorization with self-paced learning and hypergraph regularization.

More and more evidence indicates that the dysregulations of microRNAs (miRNAs) lead to diseases through various kinds of underlying mechanisms. Identifying the multiple types of disease-related miRNAs plays an important role in studying the molecular mechanism of miRNAs in diseases. Moreover, compared with traditional biological experiments, computational models are time-saving and cost-minimized. However, most tensor-based computational models still face three main challenges: (i) easy to fall into bad local minima; (ii) preservation of high-order relations; (iii) false-negative samples. To this end, we propose a novel tensor completion framework integrating self-paced learning, hypergraph regularization and adaptive weight tensor into nonnegative tensor factorization, called SPLDHyperAWNTF, for the discovery of potential multiple types of miRNA-disease associations. We first combine self-paced learning with nonnegative tensor factorization to effectively alleviate the model from falling into bad local minima. Then, hypergraphs for miRNAs and diseases are constructed, and hypergraph regularization is used to preserve the high-order complex relations of these hypergraphs. Finally, we innovatively introduce adaptive weight tensor, which can effectively alleviate the impact of false-negative samples on the prediction performance. The average results of 5-fold and 10-fold cross-validation on four datasets show that SPLDHyperAWNTF can achieve better prediction performance than baseline models in terms of Top-1 precision, Top-1 recall and Top-1 F1. Furthermore, we implement case studies to further evaluate the accuracy of SPLDHyperAWNTF. As a result, 98 (MDAv2.0) and 98 (MDAv2.0-2) of top-100 are confirmed by HMDDv3.2 dataset. Moreover, the results of enrichment analysis illustrate that unconfirmed potential associations have biological significance.

HyperNTF: A hypergraph regularized nonnegative tensor factorization for dimensionality reduction

Tensor decomposition is an effective tool for learning multi-way structures and heterogeneous features from high-dimensional data, such as the multi-view images and multichannel electroencephalography (EEG) signals, are often represented by tensors. However, most of tensor decomposition methods are the linear feature extraction techniques, which are unable to reveal the nonlinear structure within high-dimensional data. To address such problem, a lot of algorithms have been proposed for simultaneously performs linear and non-linear feature extraction. A representative algorithm is the Graph Regularized Nonnegative Matrix Factorization (GNMF) for image clustering. However, the normal 2-order graph can only model the pairwise similarity of objects, which cannot sufficiently exploit the complex structures of samples. Thus, we propose a novel method, named Hypergraph Regularized Nonnegative Tensor Factorization (HyperNTF), which utilizes hypergraph to model the complex connections among samples and employs the factor matrix corresponding with last mode of Canonical Polyadic (CP) decomposition as low-dimensional representation of original data. Extensive experiments on synthetic manifolds, real-world image datasets, and EEG signals, demonstrating that HyperNTF outperforms the state-of-the-art methods in terms of dimensionality reduction, clustering, and classification.

A sequential two-step priming scheme reproduces diversity in synaptic strength and short-term plasticity

Glutamatergic synapses display variable strength and diverse short-term plasticity (STP), even for a given type of connection. Using nonnegative tensor factorization and conventional state modeling, we demonstrate that a kinetic scheme consisting of two sequential and reversible steps of release-machinery assembly and a final step of synaptic vesicle (SV) fusion reproduces STP and its diversity among synapses. Analyzing transmission at the calyx of Held synapses reveals that differences in synaptic strength and STP are not primarily caused by variable fusion probability (pfusion) but are determined by the fraction of docked synaptic vesicles equipped with a mature release machinery. Our simulations show that traditional quantal analysis methods do not necessarily report pfusion of SVs with a mature release machinery but reflect both pfusion and the distribution between mature and immature priming states at rest. Thus, the approach holds promise for a better mechanistic dissection of the roles of presynaptic proteins in the sequence of SV docking, two-step priming, and fusion. It suggests a mechanism for activity-induced redistribution of synaptic efficacy.

Fast multiplicative algorithms for symmetric nonnegative tensor factorization

Symmetric nonnegative tensor factorization (SNTF) is an important tool for clustering analysis. To date, most of algorithms for SNTF are based on multiplicative update rules, which have many attractive properties, e.g., they are often simple to implement and can enforce nonnegativity without extra projection steps. However, the existing multiplicative algorithms often converge slowly due to the conservative multiplicative learning steps or the use of low-level BLAS (basic linear algebra subprograms) in their implementation. In this paper, three new multiplicative algorithms are proposed for SNTF to overcome the drawback of slow convergence. First, a parallel multiplicative algorithm, which can be implemented with high-level BLAS, is derived by auxiliary optimization. To further accelerate the convergence, two new parallel multiplicative algorithms, which enable larger learning steps for improving efficiency, are developed based on weighted geometric mean and weighted arithmetic mean, respectively. Finally, we apply the proposed algorithms to multiway probabilistic clustering, where a new hyper-stochastic normalization scheme based on Euclidean distance is developed for better data preprocessing. The experiment results on both synthetic and real-world data show that the proposed SNTF algorithms converge faster than the state-of-the-art algorithms.

Tensor electrical impedance myography identifies bulbar disease progression in amyotrophic lateral sclerosis

ObjectiveElectrical impedance myography (EIM) is a promising biomarker for amyotrophic lateral sclerosis (ALS). A key issue is how best to utilise the complex high dimensional, multi-frequency data output by EIM to fully characterise the progression of disease. MethodsMuscle volume conduction properties were obtained from EIM recordings of the tongue across three electrode configurations and 14 input frequencies (76 Hz–625 kHz). Analyses of individual frequencies, averaged EIM spectra and non-negative tensor factorisation were undertaken. Longitudinal data were collected from 28 patients and 17 healthy volunteers at 3-monthly intervals for a maximum of 9 months. EIM was evaluated against the Amyotrophic Lateral Sclerosis Functional Rating Scale-Revised (ALSFRS-R) bulbar sub-score, tongue strength and an overall bulbar disease burden score. ResultsLongitudinal changes to individual patient EIM spectra demonstrated complex shifts in the spectral shape. At a group level, a clear pattern emerged over time, characterised by an increase in centre frequency and general shift to the right of the spectral shape. Tensor factorisation reduced the spectral data from a total of 168 data points per participant per recording to a single value which captured the complexity of the longitudinal data and which we call tensor EIM (T-EIM). The absolute change in tensor EIM significantly increased within 3 months and continued to do so over the 9-month study duration. In a hypothetical clinical trial scenario tensor EIM required fewer participants (n = 64 at 50% treatment effect), than single frequency measures (n range 87–802) or ALSFRS-R bulbar subscore (n = 298). ConclusionsChanges to tongue EIM spectra over time in ALS are complex. Tensor EIM captured and quantified disease progression and was more sensitive to changes than single frequency EIM measures and other biomarkers of bulbar disease. SignificanceObjective biomarkers for the assessment of bulbar disease in ALS are lacking. Tensor EIM enhances the biomarker potential of EIM data and can improve bulbar symptom monitoring in clinical trials.

Explainability of the COVID-19 epidemiological model with nonnegative tensor factorization

The world is witnessing the devastating effects of the COVID-19 pandemic. Each country responded to contain the spread of the virus in the early stages through diverse response measures. Interpreting these responses and their patterns globally is essential to inform future responses to COVID-19 variants and future pandemics. A stochastic epidemiological model (SEM) is a well-established mathematical tool that helps to analyse the spread of infectious diseases through communities and the effects of various response measures. However, interpreting the outcome of these models is complex and often requires manual effort. In this paper, we propose a novel method to provide the explainability of an epidemiological model. We represent the output of SEM as a tensor model. We then apply nonnegative tensor factorization (NTF) to identify patterns of global response behaviours of countries and cluster the countries based on these patterns. We interpret the patterns and clusters to understand the global response behaviour of countries in the early stages of the pandemic. Our experimental results demonstrate the advantage of clustering using NTF and provide useful insights into the characteristics of country clusters.

Nonnegative canonical tensor decomposition with linear constraints: nnCANDELINC

AbstractThere is an emerging interest for tensor factorization applications in big‐data analytics and machine learning. To speed up the factorization of extra‐large datasets, organized in multidimensional arrays (also known as tensors), easy to compute compression‐based tensor representations, such as, Tucker and tensor train formats, are used to approximate the initial large‐tensor. Further, tensor factorization is used to extract latent features that can facilitate discoveries of new mechanisms and signatures hidden in the data, where the explainability of the latent features is of principal importance. Nonnegative tensor factorization extracts latent features that are naturally sparse and parts of the data, which makes them easily interpretable. However, to take into account available domain knowledge and subject matter expertise, often additional constraints need to be imposed, which lead us to canonical decomposition with linear constraints (CANDELINC), a canonical polyadic decomposition with rank deficient factors. In CANDELINC, Tucker compression is used as a preprocessing step, which lead to a larger residual error but to more explainable latent features. Here, we propose a nonnegative CANDELINC (nnCANDELINC) accomplished via a specific nonnegative Tucker decomposition; we refer to as minimal or canonical nonnegative Tucker. We derive several results required to understand the specificity of nnCANDELINC, focusing on the difficulties of preserving the nonnegative rank of a tensor to its Tucker core and comparing the real valued to nonnegative case. Finally, we demonstrate nnCANDELINC performance on synthetic and real‐world examples.