Part-based Representation Research Articles

Graph-based discriminative nonnegative matrix factorization with label information

Nonnegative matrix factorization (NMF) is a very effective technique for image representation, which has been widely applied in computer vision and pattern recognition. This is because it can capture the underlying intrinsic structure of data by using its parts-based representations in the low dimensional space. However, NMF is an unsupervised learning method without utilizing both the available label information and the manifold geometrical structure of data space. In this paper, we propose a new semi-supervised graph-based discriminative nonnegative matrix factorization (GDNMF) method, which incorporates the local manifold regularization and the label information of the data into the NMF. GDNMF not only encodes the local geometrical structure of the data space by constructing k-nearest graph, but also takes into account the available label information. Thus, the discriminative abilities of clustering representations are greatly enhanced. Clustering experiments on five popular databases verify the effectiveness of our proposed GDNMF compared to the other state-of-the-art methods.

Face synthesis based on parts-based sparse component analysis face representation

Face synthesis has become a highly challenging task due to illumination, face expression variation and occlusion. The point of such technique is to efficiently represent face. Recently, sparse component analysis and parts-based representation are two widely used paradigms for face representation. In this paper, we propose a probabilistic generative model for face representation towards face synthesis, which simultaneously takes advantage of the robustness of sparse component analysis and the flexibility of parts-based representation. For a given image, we project the image on the trained model and obtain the projection coefficients. Finally, a new face is reconstructed according to the learned model and projection coefficients. This model is driven by data and is a function over hidden variable and model parameters in essence. As a result, it is specifically good at representing face images. The learned face parts prior is reasonable, continuous and flexible. To validate the eff ; ;ectiveness of the proposed method on face synthesis, we perform experiments in two applications: face restoration and learning to smile. The experimental results show its advantages.

Part-based 3D object retrieval via multi-label optimization

This paper deals with the problem of 3D object retrieval using a part-based representation. The premise in this context is that similar objects will consist of similar parts. A part-based representation is proposed, where each object is segmented, and represented as a labeled graph, with nodes corresponding to parts, and edges connecting adjacent parts. The calculation of the distance between two segmented objects is formulated as a multi-label optimization problem, taking into account the aforementioned graphs. The proposed method achieves comparable performance to the state-of-the-art on several datasets, while it has a clear advantage in the case of articulated objects.

Large Sparse Cone Non-negative Matrix Factorization for Image Annotation

Image annotation assigns relevant tags to query images based on their semantic contents. Since Non-negative Matrix Factorization (NMF) has the strong ability to learn parts-based representations, recently, a number of algorithms based on NMF have been proposed for image annotation and have achieved good performance. However, most of the efforts have focused on the representations of images and annotations. The properties of the semantic parts have not been well studied. In this article, we revisit the sparseness-constrained NMF (sNMF) proposed by Hoyer [2004]. By endowing the sparseness constraint with a geometric interpretation and sNMF with theoretical analyses of the generalization ability, we show that NMF with such a sparseness constraint has three advantages for image annotation tasks: (i) The sparseness constraint is more ℓ 0 -norm oriented than the ℓ 1 -norm-based sparseness, which significantly enhances the ability of NMF to robustly learn semantic parts. (ii) The sparseness constraint has a large cone interpretation and thus allows the reconstruction error of NMF to be smaller, which means that the learned semantic parts are more powerful to represent images for tagging. (iii) The learned semantic parts are less correlated, which increases the discriminative ability for annotating images. Moreover, we present a new efficient large sparse cone NMF (LsCNMF) algorithm to optimize the sNMF problem by employing the Nesterov’s optimal gradient method. We conducted experiments on the PASCAL VOC07 dataset and demonstrated the effectiveness of LsCNMF for image annotation.

Robust Structured Nonnegative Matrix Factorization for Image Representation

Dimensionality reduction has attracted increasing attention, because high-dimensional data have arisen naturally in numerous domains in recent years. As one popular dimensionality reduction method, nonnegative matrix factorization (NMF), whose goal is to learn parts-based representations, has been widely studied and applied to various applications. In contrast to the previous approaches, this paper proposes a novel semisupervised NMF learning framework, called robust structured NMF, that learns a robust discriminative representation by leveraging the block-diagonal structure and the -norm (especially when ) loss function. Specifically, the problems of noise and outliers are well addressed by the -norm ( ) loss function, while the discriminative representations of both the labeled and unlabeled data are simultaneously learned by explicitly exploring the block-diagonal structure. The proposed problem is formulated as an optimization problem with a well-defined objective function solved by the proposed iterative algorithm. The convergence of the proposed optimization algorithm is analyzed both theoretically and empirically. In addition, we also discuss the relationships between the proposed method and some previous methods. Extensive experiments on both the synthetic and real-world data sets are conducted, and the experimental results demonstrate the effectiveness of the proposed method in comparison to the state-of-the-art methods.

Learning part-based spatial models for laser-vision-based room categorization

Room categorization, that is, recognizing the functionality of a never before seen room, is a crucial capability for a household mobile robot. We present a new approach for room categorization that is based on two-dimensional laser range data. The method is based on a novel spatial model consisting of mid-level parts that are built on top of a low-level part-based representation. The approach is then fused with a vision-based method for room categorization, which is also based on a spatial model consisting of mid-level visual parts. In addition, we propose a new discriminative dictionary learning technique that is applied for part-dictionary selection in both laser-based and vision-based modalities. Finally, we present a comparative analysis between laser-based, vision-based, and laser-vision-fusion-based approaches in a uniform part-based framework, which is evaluated on a large dataset with several categories of rooms from domestic environments.

Parts-based stereoscopic image assessment by learning binocular manifold color visual properties

Existing stereoscopic image quality assessment (SIQA) methods are mostly based on the luminance information, in which color information is not sufficiently considered. Actually, color is part of the important factors that affect human visual perception, and nonnegative matrix factorization (NMF) and manifold learning are in line with human visual perception. We propose an SIQA method based on learning binocular manifold color visual properties. To be more specific, in the training phase, a feature detector is created based on NMF with manifold regularization by considering color information, which not only allows parts-based manifold representation of an image, but also manifests localized color visual properties. In the quality estimation phase, visually important regions are selected by considering different human visual attention, and feature vectors are extracted by using the feature detector. Then the feature similarity index is calculated and the parts-based manifold color feature energy (PMCFE) for each view is defined based on the color feature vectors. The final quality score is obtained by considering a binocular combination based on PMCFE. The experimental results on LIVE I and LIVE Π 3-D IQA databases demonstrate that the proposed method can achieve much higher consistency with subjective evaluations than the state-of-the-art SIQA methods.

Accelerated parallel and distributed algorithm using limited internal memory for nonnegative matrix factorization

Nonnegative matrix factorization (NMF) is a powerful technique for dimension reduction, extracting latent factors and learning part-based representation. For large datasets, NMF performance depends on some major issues such as fast algorithms, fully parallel distributed feasibility and limited internal memory. This research designs a fast fully parallel and distributed algorithm using limited internal memory to reach high NMF performance for large datasets. Specially, we propose a flexible accelerated algorithm for NMF with all its $$L_1$$L1$$L_2$$L2 regularized variants based on full decomposition, which is a combination of exact line search, greedy coordinate descent, and accelerated search. The proposed algorithm takes advantages of these algorithms to converges linearly at an over-bounded rate $$(1-\frac{\mu }{L})(1 - \frac{\mu }{rL})^{2r}$$(1-μL)(1-μrL)2r in optimizing each factor matrix when fixing the other factor one in the sub-space of passive variables, where r is the number of latent components, and $$\mu $$μ and L are bounded as $$\frac{1}{2} \le \mu \le L \le r$$12≤μ≤L≤r. In addition, the algorithm can exploit the data sparseness to run on large datasets with limited internal memory of machines, which is is advanced compared to fast block coordinate descent methods and accelerated methods. Our experimental results are highly competitive with seven state-of-the-art methods about three significant aspects of convergence, optimality and average of the iteration numbers.

Orthogonal Nonnegative Matrix Factorization Combining Multiple Features for Spectral–Spatial Dimensionality Reduction of Hyperspectral Imagery

Nonnegative matrix factorization (NMF), which can lead to nonsubtractive parts-based representation, has been demonstrated to be effective for dimensionality reduction of hyperspectral imagery (HSI). However, existing NMF methods applied to HSI use only a single spectral feature and do not take into consideration spatial information, such as texture or morphological features, while it has been widely acknowledged that exploiting multiple features can improve performance. Consequently, a variant of orthogonal NMF, which can not only achieve a nonnegative factorization but also exploit the complementary information that arises among heterogeneous features, is proposed for hyperspectral dimensionality reduction. The proposed method, which couples orthogonal NMF with a previous multiple-features-combining algorithm, yields a discriminative low-dimensional feature representation that matches the intuition that parts should sum to produce a whole. An efficient multiplicative updating procedure is derived, and its local convergence is guaranteed theoretically. Experimental results on two hyperspectral data sets demonstrate the effectiveness of the proposed method.

Multiple clusters parts-based sparse representation for single example face identification

While great progress has been made on face identification, it is still highly challenging under real-world conditions, e.g., illumination variation and occlusion with insufficient labeled examples. In this paper, we proposed a probabilistic generative model and a face identification approach based on the model to attack the above challenge. The proposed model learns multiple clusters parts-based representation built upon three concepts: part, component and cluster. Face parts, represented by the linear combination of a set of components, refer to regions like eye and mouth. Cluster refers to a group of faces with similar appearance. A face is the summation of all parts. Based upon the learned face representation, we derived a similarity measure over faces for single example face identification. The proposed approach is able to fully exploit unlabeled data and face representation can provide informative features. We validate the effectiveness of the proposed approach through a set of experiments.

Geometric affine transformation estimation via correlation filter for visual tracking

Correlation filter achieves promising performance with high speed in visual tracking. However, conventional correlation filter based trackers cannot tackle affine transformation issues such as scale variation, rotation and skew. To address this problem, in this paper, we propose a part-based representation tracker via kernelized correlation filter (KCF) for visual tracking. A Spatial–Temporal Angle Matrix (STAM), severed as confidence metric, is proposed to select reliable patches from parts via multiple correlation filters. These stable patches are used to estimate a 2D affine transformation matrix of the target in a geometric method. Specially, the whole combination scheme for these stable patches is proposed to exploit sampling space in order to obtain numerous affine matrices and their corresponding candidates. The diversiform candidates would help to seek for the optimal candidate to represent the object's accurate affine transformation in a higher probability. Both qualitative and quantitative evaluations on VOT2014 challenge and Object Tracking Benchmark (OTB) show that the proposed tracking method achieves favorable performance compared with other state-of-the-art methods.

Semi-supervised convex nonnegative matrix factorizations with graph regularized for image representation

Non-negative matrix factorization (NMF) is a very effective method for high dimensional data analysis, which has been widely used in computer vision. It can capture the underlying structure of image in the low dimensional space using its parts-based representations. However, nonnegative entries are usually required for the data matrix in NMF, which limits its application. Besides, it is actually an unsupervised method without making use of prior information of data. In this paper, we propose a novel method called Pairwise constrained Graph Regularized Convex Nonnegative Matrix Factorization (PGCNMF), which not only allows the processing of mixed-sign data matrix but also incorporates pairwise constraints generated among all labeled data into Convex NMF framework. We expect that images which have the same class label will have very similar representations in the low dimensional space as much as possible, while images with different class labels will have dissimilar representations as much as possible. Clustering experiments on nonnegative and mixed-sign real-world image datasets are conducted to demonstrate the effectiveness of the proposed method.

Using semi-nonnegative matrix underapproximation for statistical process monitoring

In this paper, we explore the application of nonnegative matrix factorization (NMF) in statistical process monitoring. NMF has become a famous dimension reduction and feature extraction method due to its ability to produce highly localized, sparse and parts-based representation. However, NMF and all kinds of its variants have to suffer their failure to obtain some kind of statistical property of data. In this paper, semi-NMF, a variant of NMF, is first improved to reduce the correlation of the obtained low-dimensional data using an underapproximation technique called semi-nonnegative matrix underapproximation (semi-NMU). As a result, semi-NMU can extract latent variables with lower correlation and additionally obtain base matrix with higher sparsity and orthogonality. Then semi-NMU is used for fault detection. The monitoring performance of semi-NMU is evaluated by three case studies.

Multisensory Part-based Representations of Objects in Human Lateral Occipital Cortex.

The format of high-level object representations in temporal-occipital cortex is a fundamental and as yet unresolved issue. Here we use fMRI to show that human lateral occipital cortex (LOC) encodes novel 3-D objects in a multisensory and part-based format. We show that visual and haptic exploration of objects leads to similar patterns of neural activity in human LOC and that the shared variance between visually and haptically induced patterns of BOLD contrast in LOC reflects the part structure of the objects. We also show that linear classifiers trained on neural data from LOC on a subset of the objects successfully predict a novel object based on its component part structure. These data demonstrate a multisensory code for object representations in LOC that specifies the part structure of objects.

NMF-Based Image Quality Assessment Using Extreme Learning Machine.

Numerous state-of-the-art perceptual image quality assessment (IQA) algorithms share a common two-stage process: distortion description followed by distortion effects pooling. As for the first stage, the distortion descriptors or measurements are expected to be effective representatives of human visual variations, while the second stage should well express the relationship among quality descriptors and the perceptual visual quality. However, most of the existing quality descriptors (e.g., luminance, contrast, and gradient) do not seem to be consistent with human perception, and the effects pooling is often done in ad-hoc ways. In this paper, we propose a novel full-reference IQA metric. It applies non-negative matrix factorization (NMF) to measure image degradations by making use of the parts-based representation of NMF. On the other hand, a new machine learning technique [extreme learning machine (ELM)] is employed to address the limitations of the existing pooling techniques. Compared with neural networks and support vector regression, ELM can achieve higher learning accuracy with faster learning speed. Extensive experimental results demonstrate that the proposed metric has better performance and lower computational complexity in comparison with the relevant state-of-the-art approaches.

Deep Learning of Part-Based Representation of Data Using Sparse Autoencoders With Nonnegativity Constraints.

We demonstrate a new deep learning autoencoder network, trained by a nonnegativity constraint algorithm (nonnegativity-constrained autoencoder), that learns features that show part-based representation of data. The learning algorithm is based on constraining negative weights. The performance of the algorithm is assessed based on decomposing data into parts and its prediction performance is tested on three standard image data sets and one text data set. The results indicate that the nonnegativity constraint forces the autoencoder to learn features that amount to a part-based representation of data, while improving sparsity and reconstruction quality in comparison with the traditional sparse autoencoder and nonnegative matrix factorization. It is also shown that this newly acquired representation improves the prediction performance of a deep neural network.

A-Optimal Non-negative Projection with Hessian regularization

With the demand of high dimensional data analysis, data representation (or feature learning) has attracted more and more attention and becomes a central problem in pattern recognition and data mining. Non-negative Matrix Factorization (NMF) which is a useful data representation method makes great contribution in finding the latent structure of the data and leading to a parts-based representation by decomposing the data matrix into a few bases and encodings with the non-negative constraint. Considering the learned encodings from a statistical view by modeling the data points via ridge regression and minimizing the variance of the parameter, A-Optimal Non-negative Projection (ANP) improves the performance of NMF. However, it neglects the intrinsic geometric structure of the data. We introduce Hessian regularization and propose a novel method called A-Optimal Non-negative Projection with Hessian regularization (AHNP) to address this problem. Therefore, AHNP not only leads to parts-based and precise representations but preserves the intrinsic geometrical structure of the obtained subspace. We demonstrate the effectiveness of this novel algorithm through a set of evaluations on real world applications.

Dual Graph Regularized Latent Low-Rank Representation for Subspace Clustering.

Low-rank representation (LRR) has received considerable attention in subspace segmentation due to its effectiveness in exploring low-dimensional subspace structures embedded in data. To preserve the intrinsic geometrical structure of data, a graph regularizer has been introduced into LRR framework for learning the locality and similarity information within data. However, it is often the case that not only the high-dimensional data reside on a non-linear low-dimensional manifold in the ambient space, but also their features lie on a manifold in feature space. In this paper, we propose a dual graph regularized LRR model (DGLRR) by enforcing preservation of geometric information in both the ambient space and the feature space. The proposed method aims for simultaneously considering the geometric structures of the data manifold and the feature manifold. Furthermore, we extend the DGLRR model to include non-negative constraint, leading to a parts-based representation of data. Experiments are conducted on several image data sets to demonstrate that the proposed method outperforms the state-of-the-art approaches in image clustering.

A sparse neighborhood preserving non-negative tensor factorization algorithm for facial expression recognition

In this paper, a novel sparse neighborhood preserving non-negative tensor factorization (SNPNTF) algorithm is proposed for facial expression recognition. It is derived from non-negative tensor factorization (NTF), and it works in the rank-one tensor space. A sparse constraint is adopted into the objective function, which takes the optimization step in the direction of the negative gradient, and then projects onto the sparse constrained space. To consider the spatial neighborhood structure and the class-based discriminant information, a neighborhood preserving constraint is adopted based on the manifold learning and graph preserving theory. The Laplacian graph which encodes the spatial information in the face samples and the penalty graph which considers the pre-defined class information are considered in this constraint. By using it, the obtained parts-based representations of SNPNTF vary smoothly along the geodesics of the data manifold and they are more discriminant for recognition. SNPNTF is a quadratic convex function in the tensor space, and it could converge to the optimal solution. The gradient descent method is used for the optimization of SNPNTF to ensure the convergence property. Experiments are conducted on the JAFFE database, the Cohn---Kanade database and the AR database. The results demonstrate that SNPNTF provides effective facial representations and achieves better recognition performance, compared with non-negative matrix factorization, NTF and some variant algorithms. Also, the convergence property of SNPNTF is well guaranteed.

비음수 텐서 분해를 이용한 차량 인식

The active control of a vehicle based on vehicle recognition is one of key technologies for the intelligent vehicle, and the part-based image representation is necessary to recognize vehicles with only partial shapes of vehicles especially in urban scene where occlusions frequently occur. In this paper, we implemented a part-based image representation scheme using non-negative tensor factorization(NTF) and realized a robust vehicle recognition system using the NTF feature. The result shows that the proposed method gives more intuitive part-based representation and more robust recognition in urban scene.