Feature Transformation Matrix Research Articles

Robust feature selection via central point link information and sparse latent representation

Before conducting unsupervised feature selection, it is usually assumed that these data are independent of each other. On the contrary, real data will influence each other. Therefore, traditional feature selection methods may lose information related to each other between data. This can lead to inaccurately generated pseudo-label information and may result in poor feature selection results. To find solutions to this issue, this paper proposes robust feature selection via central point link information and sparse latent representation (CPSLR). Firstly, structure a link graph by calculating the center matrix to store the distance information from the sample to the center point. If two samples have similar distances to the center point, it can be determined that they belong to the same class. Therefore, the similarity between samples is preserved, and more accurate pseudo-label information is obtained. Secondly, CPSLR uses data graph and link graph to form a dual graph structure. It can not only retain the link information between samples but also retain the manifold structures of the samples. Then, CPSLR saves the interconnection contents between samples by sparse latent representation. That is, the constraint l2,1-norm is exerted on the expression of latent representation, and sparse non-redundant interconnection information is preserved. And by combining central point link information with sparse latent representation makes the interconnections between data reserved more comprehensive. That is to say, the pseudo-labels obtained are more like the real labels of the classes. Finally, CPSLR constrains the feature transformation matrix by l2,1/2-norm constraint so as to select robust and sparse features. CPSLR uses l2,1/2-norm constraint to assure that the feature transformation matrix is sparse, selecting more discriminative features, thereby obtaining the feature selection that can improve its efficiency. The experiments demonstrate that the clustering result of CPSLR outperform six classical or latest compared algorithms on eight datasets.

Non-convex feature selection based on feature correlation representation and dual manifold optimization

The existing feature selection algorithms often utilize local structure of data, but do not fully mine internal structure and ignore potential correlation information between samples. To address the problems and fully utilize manifold information of data, this paper proposes non-convex feature selection based on feature correlation representation and dual manifold optimization (FDNFS). Firstly, FDNFS constructs feature graph of original data, which can obtain feature correlation representation to represent the interconnection information. Based on the obtained interconnection information, FDNFS unifies feature correlation representation learning and feature selection through feature transformation matrix, so that the interconnection information between data guides feature selection. Secondly, to make multivariate frameworks guide feature selection, FDNFS introduces self-representation on the improved sparse regression model. Using self-representation can make basis matrix and reconstruction coefficient matrix reconstruct original data more accurately. Next, in order to preserve local structure information abundantly, FDNFS has two-part manifold regularization on the pseudo-label matrix in sparse regression model and the reconstructed coefficient matrix in self-representation framework. This can fully use the manifold information of data. In addition, FDNFS imposes the non-convex constraints. It can ensure the sparsity of feature selection matrix. In turn, this can select features with lower redundancy, and then select a better feature subset. Finally, this paper adopts an iterative optimization method. FDNFS is compared with nine algorithms on seven datasets. The clustering results reflect better performance of FDNFS.

Partial Multilabel Learning Using Fuzzy Neighborhood-Based Ball Clustering and Kernel Extreme Learning Machine

Partial multilabel learning (PML) has attracted considerable interest from scholars. Most PML models construct objective functions and optimize target parameters, which add noise to the training process and results in a poor classification effect. In addition, feature correlation is limited to linear transformations while ignoring the complex relationships among features. In this study, we develop a novel PML model with fuzzy neighbourhood-based ball clustering and kernel extreme learning machine (KELM). To reduce the interference from noise, ball k-means clustering is introduced to preprocess partial multilabel data and initialize ball clustering. A new ball clustering model based on fuzzy neighbourhood is first designed to address partial multilabel systems. The particle-ball fusion strategy is developed to merge the particle balls reasonably, and the fuzzy membership function and label enhancement are studied for the subsequent training process. Then novel KELM with feature transformation matrix of training data is produced to analyse the nonlinear relationships among features. Finally, a nonsmooth convex objective function with the regression model is constructed to analyse the complex nonlinear relationships among features, and the optimal solutions of three objective parameters are solved by accelerated proximal gradient optimization. Experiments on 14 datasets reveal the effectiveness of the developed algorithm.

Uncorrelated feature selection via sparse latent representation and extended OLSDA

Modern unsupervised feature selection methods predominantly obtain the cluster structure and pseudo-labels information through spectral clustering. However, the pseudo-labels obtained by spectral clustering are usually mixed between positive and negative. Moreover, the Laplacian matrix in spectral clustering typically affects feature selection. Additionally, spectral clustering does not consider the interconnection information between data. To address these problems, this paper proposes uncorrelated feature selection via sparse latent representation and extended orthogonal least square discriminant analysis (OLSDA), which we term SLREO). Firstly, SLREO retains the interconnection between data by latent representation learning, and preserves the internal information between the data. In order to remove redundant interconnection information, an l2,1-norm constraint is applied to the residual matrix of potential representation learning. Secondly, SLREO obtains non-negative pseudo-labels through orthogonal least square discriminant analysis (OLSDA) of embedded non-negative manifold structure. It not only avoids the appearance of negative pseudo-labels, but also eliminates the effect of the Laplacian matrix on feature selection. The manifold information of the data is also preserved. Furthermore, the matrix of the learned latent representation and OLSDA is used as pseudo-labels information. It not only ensures that the generated pseudo-labels are non-negative, but also makes the pseudo-labels closer to the true class labels. Finally, in order to avoid trivial solutions, an uncorrelated constraint and l2,1-norm constraint are imposed on the feature transformation matrix. These constraints ensure row sparsity of the feature transformation matrix, select low-redundant and discriminative features, and improve the effect of feature selection. Experimental results show that the Clustering Accuracy (ACC) and Normalized Mutual Information (NMI) of SLREO are significantly improved, as compared with six other published algorithms, tested on 11 benchmark datasets.

Feature selection via Non-convex constraint and latent representation learning with Laplacian embedding

In unsupervised feature selection, the relationship between pseudo-labels is often ignored, and the interconnection information between the data is not fully utilized. In order to solve these problems, this paper proposes a feature selection method via non-convex constraint and latent representation learning with Laplacian embedding (NLRL-LE). NLRL-LE keeps the correlation between the pseudo-labels to make the pseudo-label closer to the true label. And it combines with the interconnection information between data, learns the latent representation matrix to guide feature selection. Specifically, first, NLRL-LE regards each pseudo-label as a latent feature of the sample, constructs a latent feature graph, and retains the inherent attributes of the pseudo-labels. Second, latent representation learning is performed in the space which is made up of the latent feature space and data space. Since the latent feature graph retains the correlation between pseudo-labels, latent representation learning considers the interconnection information between data, and the information contained in the latent representation space is more complete. In addition, in order to make full use of pseudo-labels, the learned latent representation matrix is used as pseudo-label information to provide cluster labels in the latent representation space to guide feature selection. Finally, non-negative and l2,1-2-norm non-convex constraint are applied to the feature transformation matrix. The combination of non-negative constraint and non-convex constraint, compared with convex constraint, can ensure the row sparsity of the feature transformation matrix, select low-redundant features, and improve the feature selection effect. The experimental results show that the ACC and NMI of the NLRL-LE are better than the other seven compared algorithms on twelve datasets.

Unsupervised domain adaptation for in-field cotton boll status identification

In-field identification of cotton boll status is an important indicator of maturity grading and precise field management. However, the growth status of cotton boll is highly affected by environmental factors. Differentiation and correlation exhibit in data distribution of different domains, caused by distinct districts, time, weather, and farming operations. Therefore, distribution mismatch is a common phenomenon in agricultural image acquisition such that traditional manual observation measures or standard classification models that are independent and identically distributed (i.i.d.) often cannot obtain satisfactory results. One feasible solution to address this problem is to use domain adaptation that adapts knowledge from the original training data, a.k.a. the source domain, to the new testing data, a.k.a. the target domain. In this paper, we propose a novel NCA-based unsupervised domain adaptation method termed NCADA, which includes three procedures: feature extraction using a deep CNN, feature transformation matrix generation, and target label inference. We validate the NCADA method on our constructed in-field cotton boll dataset with 1,600 images. Extensive experiments show that NCADA method achieves accurate identification performances of 86.4%,85.3% and 81.2% on ‘Internet → Field’ and two different ‘Field → Field’ settings, demonstrating that NCADA can be a useful tool to replace manual observation and standard classification methods.

A Deep Multiscale Fusion Method via Low-Rank Sparse Decomposition for Object Saliency Detection Based on Urban Data in Optical Remote Sensing Images

The urban data provides a wealth of information that can support the life and work for people. In this work, we research the object saliency detection in optical remote sensing images, which is conducive to the interpretation of urban scenes. Saliency detection selects the regions with important information in the remote sensing images, which severely imitates the human visual system. It plays a powerful role in other image processing. It has successfully made great achievements in change detection, object tracking, temperature reversal, and other tasks. The traditional method has some disadvantages such as poor robustness and high computational complexity. Therefore, this paper proposes a deep multiscale fusion method via low-rank sparse decomposition for object saliency detection in optical remote sensing images. First, we execute multiscale segmentation for remote sensing images. Then, we calculate the saliency value, and the proposal region is generated. The superpixel blocks of the remaining proposal regions of the segmentation map are input into the convolutional neural network. By extracting the depth feature, the saliency value is calculated and the proposal regions are updated. The feature transformation matrix is obtained based on the gradient descent method, and the high-level semantic prior knowledge is obtained by using the convolutional neural network. The process is iterated continuously to obtain the saliency map at each scale. The low-rank sparse decomposition of the transformed matrix is carried out by robust principal component analysis. Finally, the weight cellular automata method is utilized to fuse the multiscale saliency graphs and the saliency map calculated according to the sparse noise obtained by decomposition. Meanwhile, the object priors knowledge can filter most of the background information, reduce unnecessary depth feature extraction, and meaningfully improve the saliency detection rate. The experiment results show that the proposed method can effectively improve the detection effect compared to other deep learning methods.

Supervised heterogeneous feature transfer via random forests

Transfer learning across heterogeneous feature spaces can, in general, be a very difficult problem in practice due to the heterogeneity of features and lack of correspondence between data points of different domains. In this paper, we present a novel supervised domain adaptation algorithm (SHDA-RF) that transfers knowledge from a data-rich source domain to a target domain with only few training instances. The proposed method makes use of random forests to identify pivot features that bridge the two domains. The key idea of the proposed feature transfer approach is that every path in a decision tree leading to a partition of the data is associated with a certain label distribution and the label distributions that appear both in the source and target random forest models can be used as pivots for bridging the two domains. This information is used to generate a sparse feature transformation matrix, which maps patterns from the source feature space to the target feature space. The target model is then retrained along with the projected source data. We conduct extensive experiments on diverse datasets of varying dimensions and sparsity to verify the superiority of the proposed approach over other baseline and state of the art transfer approaches.

Online Feature Transformation Learning for Cross-Domain Object Category Recognition.

In this paper, we introduce a new research problem termed online feature transformation learning in the context of multiclass object category recognition. The learning of a feature transformation is viewed as learning a global similarity metric function in an online manner. We first consider the problem of online learning a feature transformation matrix expressed in the original feature space and propose an online passive aggressive feature transformation algorithm. Then these original features are mapped to kernel space and an online single kernel feature transformation (OSKFT) algorithm is developed to learn a nonlinear feature transformation. Based on the OSKFT and the existing Hedge algorithm, a novel online multiple kernel feature transformation algorithm is also proposed, which can further improve the performance of online feature transformation learning in large-scale application. The classifier is trained with k nearest neighbor algorithm together with the learned similarity metric function. Finally, we experimentally examined the effect of setting different parameter values in the proposed algorithms and evaluate the model performance on several multiclass object recognition data sets. The experimental results demonstrate the validity and good performance of our methods on cross-domain and multiclass object recognition application.

An improved feature transformation method using mutual information

The feature transformation is a very important step in pattern recognition systems. A feature transformation matrix can be obtained using different criteria such as discrimination between classes or feature independence or mutual information between features and classes. The obtained matrix can also be used for feature reduction. In this paper, we propose a new method for finding a feature transformation-based on Mutual Information (MI). For this purpose, we suppose that the Probability Density Function (PDF) of features in classes is Gaussian, and then we use the gradient ascent to maximize the mutual information between features and classes. Experimental results show that the proposed MI projection consistently outperforms other methods for a variety of cases. In the UCI Glass database we improve the classification accuracy up to 7.95 %. Besides, the improvement of phoneme recognition rate is 3.55 % on TIMIT.

Speaker Adaptation Using ICA-Based Feature Transformation

Speaker adaptation techniques are generally used to reduce speaker differences in speech recognition. In this work, we focus on the features fitted to a linear regression-based speaker adaptation. These are obtained by feature transformation based on independent component analysis (ICA), and the feature transformation matrices are estimated from the training data and adaptation data. Since the adaptation data is not sufficient to reliably estimate the ICA-based feature transformation matrix, it is necessary to adjust the ICA-based feature transformation matrix estimated from a new speaker utterance. To cope with this problem, we propose a smoothing method through a linear interpolation between the speaker-independent (SI) feature transformation matrix and the speaker-dependent (SD) feature transformation matrix. From our experiments, we observed that the proposed method is more effective in the mismatched case. In the mismatched case, the adaptation performance is improved because the smoothed feature transformation matrix makes speaker adaptation using noisy speech more robust.

Discriminative metric design for robust pattern recognition

Motivated by the development of discriminative feature extraction (DFE), many researchers have come to realize the importance of designing a front-end feature extraction unit with an appropriate link to backend classification. This paper proposes an advanced formalization of DFE, which we call the discriminative metric design (DMD), and elaborates on its exemplar implementation by using a simple, linear feature transformation matrix. The resulting DMD implementation is shown to have a close relationship to various discriminative pattern recognizers, including artificial neural networks. The utility of the proposed method is clearly demonstrated in speech pattern recognition experiments.