Nonnegative Tensor Factorization Research Articles

Image representation using Laplacian regularized nonnegative tensor factorization

Tensor provides a better representation for image space by avoiding information loss in vectorization. Nonnegative tensor factorization (NTF), whose objective is to express an n-way tensor as a sum of k rank-1 tensors under nonnegative constraints, has recently attracted a lot of attentions for its efficient and meaningful representation. However, NTF only sees Euclidean structures in data space and is not optimized for image representation as image space is believed to be a sub-manifold embedded in high-dimensional ambient space. To avoid the limitation of NTF, we propose a novel Laplacian regularized nonnegative tensor factorization (LRNTF) method for image representation and clustering in this paper. In LRNTF, the image space is represented as a 3-way tensor and we explicitly consider the manifold structure of the image space in factorization. That is, two data points that are close to each other in the intrinsic geometry of image space shall also be close to each other under the factorized basis. To evaluate the performance of LRNTF in image representation and clustering, we compare our algorithm with NMF, NTF, NCut and GNMF methods on three standard image databases. Experimental results demonstrate that LRNTF achieves better image clustering performance, while being more insensitive to noise.

Application of Non-Negative Tensor Factorization in Intelligent Fault Diagnosis of Gearboxes

In this paper, a novel approach is proposed to extract features from the bispectra of mechanical fault signals using non-negative tensor factorization (NTF) for the purpose of establishing an intrinsic relationship between mechanical faults and extracted characteristics. First, the bispectra of mechanical signals are obtained and stacked to form a three-dimensional (3D) tensor. Then, an NTF method is applied to extract features, which are represented by a series of “basis images,” from this tensor. Finally, coefficients indicating these basis images’ weights in constituting original bispectral images are calculated for fault classification. Experiments on fault datasets of gearboxes showed that the results of the proposed method not only reveal some nonlinear features of the system, but also have low data dimensions and intuitive meanings with regard to fault characteristic frequencies, which bring great convenience to an explanation of the relationship between mechanical faults and the corresponding bispectra.

Non-Negative Tensor Factorization Applied to Music Genre Classification

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> Music genre classification techniques are typically applied to the data matrix whose columns are the feature vectors extracted from music recordings. In this paper, a feature vector is extracted using a texture window of one sec, which enables the representation of any 30 sec long music recording as a time sequence of feature vectors, thus yielding a feature matrix. Consequently, by stacking the feature matrices associated to any dataset recordings, a tensor is created, a fact which necessitates studying music genre classification using tensors. First, a novel algorithm for non-negative tensor factorization (NTF) is derived that extends the non-negative matrix factorization. Several variants of the NTF algorithm emerge by employing different cost functions from the class of Bregman divergences. Second, a novel supervised NTF classifier is proposed, which trains a basis for each class separately and employs basis orthogonalization. A variety of spectral, temporal, perceptual, energy, and pitch descriptors is extracted from 1000 recordings of the GTZAN dataset, which are distributed across 10 genre classes. The NTF classifier performance is compared against that of the multilayer perceptron and the support vector machines by applying a stratified 10-fold cross validation. A genre classification accuracy of 78.9% is reported for the NTF classifier demonstrating the superiority of the aforementioned multilinear classifier over several data matrix-based state-of-the-art classifiers. </para>

A non-negative tensor factorization model for selectional preference induction

AbstractThe distributional similarity methods have proven to be a valuable tool for the induction of semantic similarity. Until now, most algorithms use two-way co-occurrence data to compute the meaning of words. Co-occurrence frequencies, however, need not be pairwise. One can easily imagine situations where it is desirable to investigate co-occurrence frequencies of three modes and beyond. This paper will investigate tensor factorization methods to build a model of three-way co-occurrences. The approach is applied to the problem of selectional preference induction, and automatically evaluated in a pseudo-disambiguation task. The results show that tensor factorization, and non-negative tensor factorization in particular, is a promising tool for Natural Language Processing (nlp).

Nonnegative tensor factorization as an alternative Csiszar–Tusnady procedure: algorithms, convergence, probabilistic interpretations and novel probabilistic tensor latent variable analysis algorithms

In this paper we study Nonnegative Tensor Factorization (NTF) based on the Kullback---Leibler (KL) divergence as an alternative Csiszar---Tusnady procedure. We propose new update rules for the aforementioned divergence that are based on multiplicative update rules. The proposed algorithms are built on solid theoretical foundations that guarantee that the limit point of the iterative algorithm corresponds to a stationary solution of the optimization procedure. Moreover, we study the convergence properties of the optimization procedure and we present generalized pythagorean rules. Furthermore, we provide clear probabilistic interpretations of these algorithms. Finally, we discuss the connections between generalized Probabilistic Tensor Latent Variable Models (PTLVM) and NTF, proposing in that way algorithms for PTLVM for arbitrary multivariate probabilistic mass functions.

Brain CT image database building for computer-aided diagnosis using content-based image retrieval

Content-based image retrieval for medical images is a primary technique for computer-aided diagnosis. While it is a premise for computer-aided diagnosis system to build an efficient medical image database which is paid less attention than that it deserves. In this paper, we provide an efficient approach to develop the archives of large brain CT medical data. Medical images are securely acquired along with relevant diagnosis reports and then cleansed, validated and enhanced. Then some sophisticated image processing algorithms including image normalization and registration are applied to make sure that only corresponding anatomy regions could be compared in image matching. A vector of features is extracted by non-negative tensor factorization and associated with each image, which is essential for the content-based image retrieval. Our experiments prove the efficiency and promising prospect of this database building method for computer-aided diagnosis system. The brain CT image database we built could provide radiologists with a convenient access to retrieve pre-diagnosed, validated and highly relevant examples based on image content and obtain computer-aided diagnosis.

Robust feature extraction for speaker recognition based on constrained nonnegative tensor factorization

How to extract robust feature is an important research topic in machine learning community. In this paper, we investigate robust feature extraction for speech signal based on tensor structure and develop a new method called constrained Nonnegative Tensor Factorization (cNTF). A novel feature extraction framework based on the cortical representation in primary auditory cortex (AI) is proposed for robust speaker recognition. Motivated by the neural firing rates model in AI, the speech signal first is represented as a general higher order tensor. cNTF is used to learn the basis functions from multiple interrelated feature subspaces and find a robust sparse representation for speech signal. Computer simulations are given to evaluate the performance of our method and comparisons with existing speaker recognition methods are also provided. The experimental results demonstrate that the proposed method achieves higher recognition accuracy in noisy environment.

Robust Feature Extraction for Speaker Recognition Based on Constrained Nonnegative Tensor Factorization

Robust Feature Extraction for Speaker Recognition Based on Constrained Nonnegative Tensor Factorization

On the equivalence between nonnegative tensor factorization and tensorial probabilistic latent semantic analysis

Non-negative Matrix Factorization (NMF) and Probabilistic Latent Semantic Analysis (PLSA) are two widely used methods for non-negative data decomposition of two-way data (e.g., document-term matric...

Non-Negative Multilinear Principal Component Analysis of Auditory Temporal Modulations for Music Genre Classification

Motivated by psychophysiological investigations on the human auditory system, a bio-inspired two-dimensional auditory representation of music signals is exploited, that captures the slow temporal modulations. Although each recording is represented by a second-order tensor (i.e., a matrix), a third-order tensor is needed to represent a music corpus. Non-negative multilinear principal component analysis (NMPCA) is proposed for the unsupervised dimensionality reduction of the third-order tensors. The NMPCA maximizes the total tensor scatter while preserving the non-negativity of auditory representations. An algorithm for NMPCA is derived by exploiting the structure of the Grassmann manifold. The NMPCA is compared against three multilinear subspace analysis techniques, namely the non-negative tensor factorization, the high-order singular value decomposition, and the multilinear principal component analysis as well as their linear counterparts, i.e., the non-negative matrix factorization, the singular value decomposition, and the principal components analysis in extracting features that are subsequently classified by either support vector machine or nearest neighbor classifiers. Three different sets of experiments conducted on the GTZAN and the ISMIR2004 Genre datasets demonstrate the superiority of NMPCA against the aforementioned subspace analysis techniques in extracting more discriminating features, especially when the training set has small cardinality. The best classification accuracies reported in the paper exceed those obtained by the state-of-the-art music genre classification algorithms applied to both datasets.

Michromatic scope for enhancement of color difference

AbstractA michromatic (microscope plus chromatic) scope is a device that enhances the color discrimination between two spectral color datasets. Three spectral filters are required, instead of the conventional red, green, and blue filters, for the implementation of a michromatic camera. In this study, we describe two approaches to the design of these filters: in the first case, the design is based on the direct optimization of the filter characteristics (transmittance), whereas in the second case, the design is based on the nonnegative tensor factorization (NTF) of the spectral datasets. A michromatic camera can be implemented using these filters along with compatible postprocessing in‐camera firmware. Here, we performed experiments with two color datasets: one comprising skin and vein colors, and one comprising skin and cosmetics colors. These were further divided into a training set and a test set. The filters were defined using the training set, and the operation of the filters was tested and magnified using the test set. Our experiments demonstrated that the proposed approaches are suitable for color discrimination. For the first color dataset, the enhancement produced using the optimized filters was up to 252% of the original value, and the average color difference ΔE was increased from 2.82 to 9.93. NTF and preprocessing further enhanced the ΔE up to 21.84. For the second color dataset, NTF and postprocessing enhanced the ΔE from 4.33 to 29.19. The proposed discrimination enhancement could be physically implemented in a designated digital charge‐coupled device camera with proper filter installation and compatible postprocessing. © 2010 Wiley Periodicals, Inc. Col Res Appl, 2010

Further Results for Perron–Frobenius Theorem for Nonnegative Tensors

For a nonnegative irreducible tensor, we give distribution properties of its eigenvalues. In particular, the spectral radius of a nonnegative irreducible tensor with positive trace is proved to be the unique eigenvalue on the spectral circle. Unlike the matrix setting, we give an example to present that this type of tensor is not always primitive. Thus, for a nonnegative irreducible tensor, the primitivity is a sufficient condition only for the spectral radius to be the unique eigenvalue on the spectral circle. Also, the stochastic tensor is defined, and we show that every nonnegative irreducible tensor with unit spectral radius is diagonally similar to a certain irreducible stochastic tensor. Based on this result, the minimax theorem for tensors is proved by using an alternative approach. Further, with the help of the minimax theorem, we illustrate that the problem of finding the spectral radius (largest singular value) of a nonnegative irreducible square (rectangular) tensor can be converted into a convex optimization problem. Additionally, we give an equivalent condition of irreducible nonnegative tensors. By this condition, one can easily determine whether or not a nonnegative tensor is irreducible.

Blind multispectral image decomposition by 3D nonnegative tensor factorization

Alpha-divergence-based nonnegative tensor factorization (NTF) is applied to blind multispectral image (MSI) decomposition. The matrix of spectral profiles and the matrix of spatial distributions of the materials resident in the image are identified from the factors in Tucker3 and PARAFAC models. NTF preserves local structure in the MSI that is lost as a result of vectorization of the image when nonnegative matrix factorization (NMF)- or independent component analysis (ICA)-based decompositions are used. Moreover, NTF based on the PARAFAC model is unique up to permutation and scale under mild conditions. To achieve this, NMF- and ICA-based factorizations, respectively, require enforcement of sparseness (orthogonality) and statistical independence constraints on the spatial distributions of the materials resident in the MSI, and these conditions do not hold. We demonstrate efficiency of the NTF-based factorization in relation to NMF- and ICA-based factorizations on blind decomposition of the experimental MSI with the known ground truth.

Discriminant Nonnegative Tensor Factorization Algorithms

Nonnegative matrix factorization (NMF) has proven to be very successful for image analysis, especially for object representation and recognition. NMF requires the object tensor (with valence more than one) to be vectorized. This procedure may result in information loss since the local object structure is lost due to vectorization. Recently, in order to remedy this disadvantage of NMF methods, nonnegative tensor factorizations (NTF) algorithms that can be applied directly to the tensor representation of object collections have been introduced. In this paper, we propose a series of unsupervised and supervised NTF methods. That is, we extend several NMF methods using arbitrary valence tensors. Moreover, by incorporating discriminant constraints inside the NTF decompositions, we present a series of discriminant NTF methods. The proposed approaches are tested for face verification and facial expression recognition, where it is shown that they outperform other popular subspace approaches.

Fast Local Algorithms for Large Scale Nonnegative Matrix and Tensor Factorizations

Nonnegative matrix factorization (NMF) and its extensions such as Nonnegative Tensor Factorization (NTF) have become prominent techniques for blind sources separation (BSS), analysis of image databases, data mining and other information retrieval and clustering applications. In this paper we propose a family of efficient algorithms for NMF/NTF, as well as sparse nonnegative coding and representation, that has many potential applications in computational neuroscience, multi-sensory processing, compressed sensing and multidimensional data analysis. We have developed a class of optimized local algorithms which are referred to as Hierarchical Alternating Least Squares (HALS) algorithms. For these purposes, we have performed sequential constrained minimization on a set of squared Euclidean distances. We then extend this approach to robust cost functions using the alpha and beta divergences and derive flexible update rules. Our algorithms are locally stable and work well for NMF-based blind source separation (BSS) not only for the over-determined case but also for an under-determined (over-complete) case (i.e., for a system which has less sensors than sources) if data are sufficiently sparse. The NMF learning rules are extended and generalized for N-th order nonnegative tensor factorization (NTF). Moreover, these algorithms can be tuned to different noise statistics by adjusting a single parameter. Extensive experimental results confirm the accuracy and computational performance of the developed algorithms, especially, with usage of multi-layer hierarchical NMF approach [3].

Computing non-negative tensor factorizations

Non-negative tensor factorization (NTF) is a technique for computing a parts-based representation of high-dimensional data. NTF excels at exposing latent structures in datasets, and at finding good low-rank approximations to the data. We describe an approach for computing the NTF of a dataset that relies only on iterative linear-algebra techniques and that is comparable in cost to the non-negative matrix factorization (NMF). (The better-known NMF is a special case of NTF and is also handled by our implementation.) Some important features of our implementation include mechanisms for encouraging sparse factors and for ensuring that they are equilibrated in norm. The complete MATLAB software package is available under the GPL license.

Advances in Nonnegative Matrix and Tensor Factorization

Advances in Nonnegative Matrix and Tensor Factorization

Single-Trial Decoding of Bistable Perception Based on Sparse Nonnegative Tensor Decomposition

The study of the neuronal correlates of the spontaneous alternation in perception elicited by bistable visual stimuli is promising for understanding the mechanism of neural information processing and the neural basis of visual perception and perceptual decision-making. In this paper, we develop a sparse nonnegative tensor factorization-(NTF)-based method to extract features from the local field potential (LFP), collected from the middle temporal (MT) visual cortex in a macaque monkey, for decoding its bistable structure-from-motion (SFM) perception. We apply the feature extraction approach to the multichannel time-frequency representation of the intracortical LFP data. The advantages of the sparse NTF-based feature extraction approach lies in its capability to yield components common across the space, time, and frequency domains yet discriminative across different conditions without prior knowledge of the discriminating frequency bands and temporal windows for a specific subject. We employ the support vector machines (SVMs) classifier based on the features of the NTF components for single-trial decoding the reported perception. Our results suggest that although other bands also have certain discriminability, the gamma band feature carries the most discriminative information for bistable perception, and that imposing the sparseness constraints on the nonnegative tensor factorization improves extraction of this feature.

Nonnegative Matrix and Tensor Factorization [Lecture Notes

In these lecture notes, the authors have outlined several approaches to solve a NMF/NTF problem. The following main conclusions can be drawn: 1) Multiplicative algorithms are not necessary the best approaches for NMF, especially if data representations are not very redundant or sparse. 2) Much better performance can be achieved using the FP-ALS (especially for large-scale problems), IPC, and QN methods. 3) To achieve high performance it is quite important to use the multilayer structure with multistart initialization conditions. 4) To estimate physically meaningful nonnegative components it is often necessary to use some a priori knowledge and impose additional constraints or regularization terms (to control sparsity, boundness, continuity or smoothness of the estimated nonnegative components).

NONNEGATIVE TENSOR FACTORIZATION FOR CONTINUOUS EEG CLASSIFICATION

In this paper we present a method for continuous EEG classification, where we employ nonnegative tensor factorization (NTF) to determine discriminative spectral features and use the Viterbi algorithm to continuously classify multiple mental tasks. This is an extension of our previous work on the use of nonnegative matrix factorization (NMF) for EEG classification. Numerical experiments with two data sets in BCI competition, confirm the useful behavior of the method for continuous EEG classification.