Feature Weight Learning Research Articles

On the strong consistency of feature‐weighted k‐means clustering in a nearmetric space

Weighted k‐means (WK‐means) is a well‐known method for automated feature weight learning in a conventional k‐means clustering framework. In this paper, we analytically explore the strong consistency of the WK‐means algorithm under independent and identically distributed sampling of the data points. The choice of dissimilarity measure plays a key role in data partitioning and detecting the inherent groups existing in a dataset. We propose a proof of strong consistency of the WK‐means algorithm when the dissimilarity measure used is assumed to be a nearmetric. The proof can be further extended to those dissimilarity measures which are an increasing function of a nearmetric. Through detailed experiments, we demonstrate that WK‐means‐type algorithms, equipped with a nearmetric, can be pretty effective especially when some of the features are unimportant in revealing the cluster structure of the dataset.

Simultaneous variable weighting and determining the number of clusters—A weighted Gaussian means algorithm

We propose a simple variable (feature) weight learning strategy for the Gaussian means algorithm which can automatically determine the number of clusters in a dataset as well. We investigate some important theoretical properties and convergence behavior of the proposed algorithm.

MGA for feature weight learning in SVM —a novel optimization method in pedestrian detection

Pedestrian detection is a challenging task in computer vision, which is often treated as classification problem of pattern recognition. However, dealing with the high dimensional features extracted from images, it turns out to be difficult to choose and combine the informative features for classification. In this paper, a novel optimization method—Metropolis based Genetic Algorithm (MGA) is proposed to solve this problem, and a novel pedestrian detector MGA-SVM is presented and implemented. In MGA, the metropolis criterion is adopted into GA for dynamical parents’ selection, which makes the algorithm get a stronger ability to jump out of local minimum as well as achieve convergence. To test the effectiveness of the proposed MGA, we implement it for feature weight learning in SVM pedestrian detector, which is named as MGA-SVM. The experimental results demonstrate the MGA has a better optimization capacity than original GA, which leads to a more accurate pedestrian detection result by using MGA-SVM.

Improving EEG signal peak detection using feature weight learning of a neural network with random weights for eye event-related applications

The optimization of peak detection algorithms for electroencephalogram (EEG) signal analysis is an ongoing project; previously existing algorithms have been used with different models to detect EEG peaks in various applications. However, none of the existing techniques perform adequately in eye event-related applications. Therefore, we aimed to develop a general procedure for eye event-related applications based on feature weight learning (FWL), through the use of a neural network with random weights (NNRW) as the classifier. The FWL is performed using a particle swarm optimization algorithm, applied to the well-studied Dumpala, Acir, Liu and Dingle peak detection models, where the associated features are considered as inputs to the NNRW with and without FWL. The combination of all the associated features from the four models is also considered, as a comprehensive model for validation purposes. Real EEG data recorded from two channels of 20 healthy volunteers were used to perform the model simulations. The data set consisted of 40 peaks arising in the frontal eye field in association with a change of horizontal eye gaze direction. It was found that the NNRW in conjunction with FWL has better performance than NNRW alone for all four peak detection models, of which the Dingle model gave the highest performance, with 74% accuracy.

Exploiting score distribution for heterogenous feature fusion in image classification

Over the past decades, features generated by different models have been designed to describe various aspects of object. To connect the complementary information and represent the data properly, effective heterogeneous feature fusion methods are required. Multiple kernel learning (MKL) methods are widely adopted to learn the feature weights and to fuse features on score-level. In this paper, we exploit score distribution to address the feature fusion problem and propose a novel method named score-distribution MKL (SD-MKL) for image classification. Different from existing MKL methods, SD-MKL uses weights which are learned from score curves as a constraint on the weights of kernels. It contains two stages in off-line part: (1) independent data is used to construct reference curves according to classes and feature type; (2) samples and corresponding score-distribution weights are put into multi-kernel support vector machine (MKSVM) to learn feature weights. Our experimental results demonstrate the effect of exploiting score-distribution information on two datasets, which significantly benefits the performance of image classification.

Building a multi-domain comparable corpus using a learning to rank method

AbstractComparable corpora are key translation resources for both languages and domains with limited linguistic resources. The existing approaches for building comparable corpora are mostly based on ranking candidate documents in the target language for each source document using a cross-lingual retrieval model. These approaches also exploit other evidence of document similarity, such as proper names and publication dates, to build more reliable alignments. However, the importance of each evidence in the scores of candidate target documents is determined heuristically. In this paper, we employ a learning to rank method for ranking candidate target documents with respect to each source document. The ranking model is constructed by defining each evidence for similarity of bilingual documents as a feature whose weight is learned automatically. Learning feature weights can significantly improve the quality of alignments, because the reliability of features depends on the characteristics of both source and target languages of a comparable corpus. We also propose a method to generate appropriate training data for the task of building comparable corpora. We employed the proposed learning-based approach to build a multi-domain English–Persian comparable corpus which covers twelve different domains obtained from Open Directory Project. Experimental results show that the created alignments have high degrees of comparability. Comparison with existing approaches for building comparable corpora shows that our learning-based approach improves both quality and coverage of alignments.

Robust Joint Feature Weights Learning Framework

Feature selection, selecting the most informative subset of features, is an important research direction in dimension reduction. The combinatorial search in feature selection is essentially a binary optimization problem, known as NP hard, which can be alleviated by learning feature weights. Traditional feature weights algorithms rely on heuristic search path. These approaches neglect the interaction and dependency between different features, and thus provide no guarantee for optimality. In this paper, we propose a novel joint feature weights learning framework, which imposes both nonnegative and $\ell _{2,1}$ -norm constraints on the feature weights matrix. The nonnegative property ensures the physical significance of learned feature weights. Meanwhile, $\ell _{2,1}$ -norm minimization achieves joint selection of the most relevant features by exploiting the whole feature space. More importantly, an efficient iterative algorithm with proved convergence is designed to optimize a convex objective function. Using this framework as a platform, we propose new supervised and unsupervised joint feature selection methods. Particularly, in the proposed unsupervised method, nonnegative graph embedding is developed to exploit intrinsic structure in the weighted space. Comparative experiments on seven real-world data sets indicate that our framework is both effective and efficient.

Deep feature weighting for naive Bayes and its application to text classification

Naive Bayes (NB) continues to be one of the top 10 data mining algorithms due to its simplicity, efficiency and efficacy. Of numerous proposals to improve the accuracy of naive Bayes by weakening its feature independence assumption, the feature weighting approach has received less attention from researchers. Moreover, to our knowledge, all of the existing feature weighting approaches only incorporate the learned feature weights into the classification of formula of naive Bayes and do not incorporate the learned feature weights into its conditional probability estimates at all. In this paper, we propose a simple, efficient, and effective feature weighting approach, called deep feature weighting (DFW), which estimates the conditional probabilities of naive Bayes by deeply computing feature weighted frequencies from training data. Empirical studies on a collection of 36 benchmark datasets from the UCI repository show that naive Bayes with deep feature weighting rarely degrades the quality of the model compared to standard naive Bayes and, in many cases, improves it dramatically. Besides, we apply the proposed deep feature weighting to some state-of-the-art naive Bayes text classifiers and have achieved remarkable improvements.

Sparse preserving feature weights learning

In this paper, we propose a novel unsupervised feature selection algorithm, named sparse preserving feature weights learning (SPFW), which is based on the recent local data representation theory, sparse representation. SPFW differs from traditional feature selection algorithms in two aspects: (1) SPFW is designed on the locality measurement criterion with sparse reconstruction residual minimization. It adaptively determines the locality based on sparse representation, instead of fixing the k-nearest neighbors in the original feature space. (2) SPFW selects the most discriminative feature subset from the whole feature set in batch mode, instead of selecting features individually. To optimize the proposed formulation, we propose an efficient iterative algorithm, where each iteration reduces to a subproblem which can be solved with some off-the-shelf toolboxes. We conduct experiments on two face datasets to evaluate the performance of feature selection in terms of classification and clustering, which demonstrate the effectiveness of the proposed algorithm.

Sparse discriminative feature weights learning

Sparse representation, a locality-based data representation method, leads to promising results in many scientific and engineering fields. Meanwhile in the study of feature selection, locality preserving is widely recognized as an effective measurement criterion. In this paper, we introduce l1-norm driven sparse representation into feature selection, and propose a novel joint feature weights learning algorithm, named sparse discriminative feature weights (SDFW). SDFW assigns the highest score to the feature that has the smallest difference between within-class reconstruction residual and between-class reconstruction residual in the space of selected features. It possesses the following advantages: (1) compared with feature selection methods based on k nearest neighbors, SDFW automatically (vs. manually) determines neighborhood for individual sample; (2) compared with conventional heuristic feature search which selects features individually, SDFW selects feature subset in batch mode. Extensive experiments on different data types demonstrate the effectiveness of SDFW.

Detection of social events in streams of social multimedia

Combining items from social media streams, such as Flickr photos and Twitter tweets, into meaningful groups can help users contextualise and consume more effectively the torrents of information continuously being made available on the social web. This task is made challenging due to the scale of the streams and the inherently multimodal nature of the information being contextualised. The problem of grouping social media items into meaningful groups can be seen as an ill-posed and application specific unsupervised clustering problem. A fundamental question in multimodal contexts is determining which features best signify that two items should belong to the same grouping. This paper presents a methodology which approaches social event detection as a streaming multi-modal clustering task. The methodology takes advantage of the temporal nature of social events and as a side benefit, allows for scaling to real-world datasets. Specific challenges of the social event detection task are addressed: the engineering and selection of the features used to compare items to one another; a feature fusion strategy that incorporates relative importance of features; the construction of a single sparse affinity matrix; and clustering techniques which produce meaningful item groups whilst scaling to cluster very large numbers of items. The state-of-the-art approach presented here is evaluated using the ReSEED dataset with standardised evaluation measures. With automatically learned feature weights, we achieve an $${F}_1$$ score of 0.94, showing that a good compromise between precision and recall of clusters can be achieved. In a comparison with other state-of-the-art algorithms our approach is shown to give the best results.

Locality preserving score for joint feature weights learning

Locality preserving measurement criterion is frequently used for assessing the quality of features. However, locality preserving criterion based unsupervised feature selection algorithms have two widely acknowledged weaknesses: (1) The performance of feature selection heavily depends on the effectiveness of the similarity matrix, which is defined in the original space, and thus it is probably inconsistent with the one in the weighted space. (2) Greedy searching strategy neglects the correlation and redundancy among features. To alleviate these deficiencies, we propose a novel unsupervised feature selection algorithm by jointly learning adaptive nearest neighbors in the weighed space. An effective iterative algorithm is developed to solve the proposed formulation, where each iteration reduces to a convex subproblem which can be efficiently solved with some off-the-shelf toolboxes. The results of experiments on the UCI and face data sets demonstrate the effectiveness of the proposed algorithm, for outperforming many state-of-the-art unsupervised and supervised feature selection methods in terms of classification accuracy.

An Improved Clustering Algorithm Based on Feature-weight Learning

An Improved Clustering Algorithm Based on Feature-weight Learning

Categorical fuzzy k-modes clustering with automated feature weight learning

This article presents and investigates a new variant of the fuzzy k-Modes clustering algorithm for categorical data with automated feature weight learning. The modification strengthens the classical fuzzy k-Modes algorithm by associating higher weights to features which are instrumental in recognizing the clustering pattern of the data. A statistical comparison between the performances of the proposed algorithm and the conventional fuzzy k-Modes algorithm on synthetic and real world datasets, have been carried out with respect to mean values, best performance count, and medians. We take a novel approach towards the comparison of the fuzziness of the obtained clusters. To the best of our knowledge, such comparison has been reported here for the first time for the case of categorical data. The results obtained, shows that the proposed algorithm enjoys an edge over the conventional fuzzy k-Modes algorithm both in terms of Rand Index and fuzziness measures.

An application of case-based reasoning with machine learning for forensic autopsy

Case-based reasoning (CBR) is one of the matured paradigms of artificial intelligence for problem solving. CBR has been applied in many areas in the commercial sector to assist daily operations. However, CBR is relatively new in the field of forensic science. Even though forensic personnel have consciously used past experiences in solving new cases, the idea of applying machine intelligence to support decision-making in forensics is still in its infancy and poses a great challenge. This paper highlights the limitation of the methods used in forensics compared with a CBR method in the analysis of forensic evidences. The design and development of an Intelligent Forensic Autopsy Report System (I-AuReSys) basing on a CBR method along with the experimental results are presented. Our system is able to extract features by using an information extraction (IE) technique from the existing autopsy reports; then the system analyzes the case similarities by coupling the CBR technique with a Naïve Bayes learner for feature-weights learning; and finally it produces an outcome recommendation. Our experimental results reveal that the CBR method with the implementation of a learner is indeed a viable alternative method to the forensic methods with practical advantages.

Joint Laplacian feature weights learning

Some filter methods stemming from statistics or geometry theory select features individually. Hence they neglect the combination of features and lead to suboptimal subset of features. To address this problem, a joint feature weights learning framework, which automatically determines the optimal size of the feature subset and selects the best features corresponding to a given adjacency graph, is proposed in this paper. In particular, our framework imposes nonnegative and l22-norm constraints on feature weights and iteratively learns feature weights jointly and simultaneously. A new minimization algorithm with proved convergence is also developed to optimize the non-convex objective function. Utilizing this framework as a tool, we propose a new unsupervised feature selection algorithm called Joint Laplacian Feature Weights Learning. Experimental results on five real-world datasets demonstrate the effectiveness of our algorithm.

Reinforced Similarity Integration in Image-Rich Information Networks

Social multimedia sharing and hosting websites, such as Flickr and Facebook, contain billions of user-submitted images. Popular Internet commerce websites such as Amazon.com are also furnished with tremendous amounts of product-related images. In addition, images in such social networks are also accompanied by annotations, comments, and other information, thus forming heterogeneous image-rich information networks. In this paper, we introduce the concept of (heterogeneous) image-rich information network and the problem of how to perform information retrieval and recommendation in such networks. We propose a fast algorithm heterogeneous minimum order k-SimRank (HMok-SimRank) to compute link-based similarity in weighted heterogeneous information networks. Then, we propose an algorithm Integrated Weighted Similarity Learning (IWSL) to account for both link-based and content-based similarities by considering the network structure and mutually reinforcing link similarity and feature weight learning. Both local and global feature learning methods are designed. Experimental results on Flickr and Amazon data sets show that our approach is significantly better than traditional methods in terms of both relevance and speed. A new product search and recommendation system for e-commerce has been implemented based on our algorithm.

Image Similarity Learning through Rankboost Mechanism Based On Keywords and Content Queries

Different types of search engines are used to search image and text contents. Two types of image search methods are available in the Internet. They are query keyword based model and content based image retrieval models. Text query strings are used in the textual image retrieval model. Content based image retrieval (CBIR) model uses the visual information of the images. Image search methods use the text annotation and image visual features. Google image search and Bing image search engines are used to fetch images from the web. Image query string is used to search image on Internet. One click query image selection method is used to submit user intention for image retrieval. Content based image re-ranking is performed with visual and textual similarity metrics. Adaptive Weight Schema is used for the similarity analysis. Feature weight learning algorithm is applied to estimate feature weights for the images and its category. Query is expanded with keyword and visual information. Rank boost framework algorithm is enhanced to rank images with photographic quality. Content similarity and visual quality factors are used for the re-ranking process. In this paper, we propose an image indexing and retrieval using speech annotations based on a predefined structured syntax. To improve the retrieval effectively, N-best lists for index generation is used .so, a query expansion technique is explored to enhance the query terms. All this process is automatic, without extra effort from the user. This is critically important in web-based image search engine for any commercial, where the user interface has to be extremely simple.

Machine learning with naturally labeled data for identifying abbreviation definitions

BackgroundThe rapid growth of biomedical literature requires accurate text analysis and text processing tools. Detecting abbreviations and identifying their definitions is an important component of such tools. Most existing approaches for the abbreviation definition identification task employ rule-based methods. While achieving high precision, rule-based methods are limited to the rules defined and fail to capture many uncommon definition patterns. Supervised learning techniques, which offer more flexibility in detecting abbreviation definitions, have also been applied to the problem. However, they require manually labeled training data.MethodsIn this work, we develop a machine learning algorithm for abbreviation definition identification in text which makes use of what we term naturally labeled data. Positive training examples are naturally occurring potential abbreviation-definition pairs in text. Negative training examples are generated by randomly mixing potential abbreviations with unrelated potential definitions. The machine learner is trained to distinguish between these two sets of examples. Then, the learned feature weights are used to identify the abbreviation full form. This approach does not require manually labeled training data.ResultsWe evaluate the performance of our algorithm on the Ab3P, BIOADI and Medstract corpora. Our system demonstrated results that compare favourably to the existing Ab3P and BIOADI systems. We achieve an F-measure of 91.36% on Ab3P corpus, and an F-measure of 87.13% on BIOADI corpus which are superior to the results reported by Ab3P and BIOADI systems. Moreover, we outperform these systems in terms of recall, which is one of our goals.

Improving Definite Anaphora Resolution by Effective Weight Learning and Web-Based Knowledge Acquisition

In this paper, effective Chinese definite anaphora resolution is addressed by using feature weight learning and Web-based knowledge acquisition. The presented salience measurement is based on entropy-based weighting on selecting antecedent candidates. The knowledge acquisition model is aimed to extract more semantic features, such as gender, number, and semantic compatibility by employing multiple resources and Web mining. The resolution is justified with a real corpus and compared with a classification-based model. Experimental results show that our approach yields 72.5% success rate on 426 anaphoric instances. In comparison with a general classification-based approach, the performance is improved by 4.7%.