Semi-supervised Feature Selection Research Articles

Adaptive Semi-Supervised Feature Selection for Cross-Modal Retrieval

In order to exploit the abundant potential information of the unlabeled data and contribute to analyzing the correlation among heterogeneous data, we propose the semi-supervised model named adaptive semi-supervised feature selection for cross-modal retrieval. First, we utilize the semantic regression to strengthen the neighboring relationship between the data with the same semantic. And the correlation between heterogeneous data can be optimized via keeping the pairwise closeness when learning the common latent space. Second, we adopt the graph-based constraint to predict accurate labels for unlabeled data, and it can also keep the geometric structure consistency between the label space and the feature space of heterogeneous data in the common latent space. Finally, an efficient joint optimization algorithm is proposed to update the mapping matrices and the label matrix for unlabeled data simultaneously and iteratively. It makes samples from different classes to be far apart, while the samples from same class lie as close as possible. Meanwhile, the ${l_{2,1}}$ -norm constraint is used for feature selection and outlier reduction when the mapping matrices are learned. In addition, we propose learning different mapping matrices corresponding to different sub-tasks to emphasize the semantic and structural information of query data. Experiment results on three datasets demonstrate that our method performs better than the state-of-the-art methods.

Rough set based semi-supervised feature selection via ensemble selector

Similar to feature selection over completely labeled data, the aim of feature selection over partially labeled data (semi-supervised feature selection) is also to find a feature subset which satisfies the intended constraint. Nevertheless, two difficulties may emerge in the semi-supervised feature selection: (1) labels are incomplete since labeled and unlabeled samples coexist in data; (2) the explanation of the selected feature subset is not clear. Therefore, such two problems will be mainly addressed in our research. Firstly, the unlabeled samples can be predicted through various semi-supervised learning methods. Secondly, the Local Neighborhood Decision Error Rate is proposed to construct multiple fitness functions for evaluating the significance of the candidate feature. Such mechanism not only realizes the ensemble selector in the process of feature selection, but also the qualified feature subset will bring us lower decision errors. Immediately, a heuristic algorithm is re-designed to execute feature selection. Finally, through testing nine different ratios (10%, 20%, …, 90%) of labeled samples in data, the experimental results demonstrate that our approach is superior to previous researches, mainly because: (1) the qualified feature subset derived by our approach can provide better classification performance; (2) the lower time consumption is required in our process of feature selection.

Semi-supervised feature selection analysis with structured multi-view sparse regularization

Facing abundant and various multi-view data, how to effectively combine the multi-view data information has become an important research topic in feature selection analysis. However, existing feature selection methods usually consider each view features as a whole without fully considering the individual feature in each view. In this paper, we construct a structured multi-view sparse regularization and then propose a novel semi-supervise feature selection framework, namely Structured Multi-view Hessian sparse Feature Selection (SMHFS)11SMHFS: Structured Multi-view Hessian sparse Feature Selection.. With the structured multi-view sparse regularization, SMHFS can simultaneously learn the importance of each view features and the importance of individual feature in each view. In addition, SMHFS utilizes multi-view Hessian regularization to enhance the semi-supervised learning performance. An iterative algorithm is introduced and its convergence is proven. Finally, SMHFS is applied into image annotation task and extensive experiments are conducted. The experimental results show SMHFS can effectively combine the multi-view data information to achieve better feature selection performance compared to other methods.

Semi-Supervised Feature Selection via Insensitive Sparse Regression with Application to Video Semantic Recognition

Feature selection plays a significant role in dealing with high-dimensional data to avoid the curse of dimensionality. In many real applications, like video semantic recognition, handling few labeled and large unlabeled data samples from the same population is a recently addressed challenge in feature selection. To solve this problem, we propose a novel semi-supervised feature selection method via insensitive sparse regression (ISR). Specifically, we compute the soft label matrix by the special label propagation, which can predict the labels of the unlabeled data. To guarantee the robustness of ISR to the false labeled instances or outliers, we propose Insensitive Regression Model (IRM) by capped $l_2$ - $l_p$ -norm loss. The soft label is imposed as the weights of IRM to fully utilize the label information. Meanwhile, to perform feature selection, we incorporate $l_{2,q}$ -norm regularizer with IRM as the structural sparsity constraint when $0 . Moreover, we put forward an effective approach for solving the formulated non-convex optimization problem. We analyze the performance of convergence rigorously and discuss the parameter determination problem. Extensive experimental results on several public data sets verify the effectiveness of our proposed algorithm in comparison with the state-of-art feature selection methods. Finally, we apply our method to video semantic recognition successfully.

Semi-supervised sparse feature selection via graph Laplacian based scatter matrix for regression problems

Semi-supervised feature selection, which exploits both labeled and unlabeled data to select the relevant features, has an important role in many real world applications. Most semi-supervised feature selection methods have been exclusively designed for classification problems. In this paper, a semi-supervised framework based on graph Laplacian and mixed convex and non-convex l2,p-norm (0 < p ≤ 1) regularization is proposed for regression problems. In the proposed framework, a semi-supervised graph Laplacain based scatter matrix constructed for regression problems is used to encode the label information of labeled data and the local structure of both labeled and unlabeled data. To solve the mixed convex and non-convex regularized l2,p-norm framework, a unified iterative algorithm is proposed. The convergence of the proposed unified algorithm is theoretically and experimentally proved. To evaluate the performance of the proposed framework for regression problems, we perform extensive experiments on different computational drug design regression datasets. The results demonstrate the superiority of the proposed framework in comparison with other feature selection methods in selecting the relevant features.

Discriminative Semi-Supervised Feature Selection via Rescaled Least Squares Regression-Supplement

In this paper, we propose a Discriminative Semi-Supervised Feature Selection (DSSFS) method. In this method, a ε-dragging technique is introduced to the Rescaled Linear Square Regression in order to enlarge the distances between different classes. An iterative method is proposed to simultaneously learn the regression coefficients, ε-draggings matrix and predicting the unknown class labels. Experimental results show the superiority of DSSFS.

Feature selection in machine learning: A new perspective

High-dimensional data analysis is a challenge for researchers and engineers in the fields of machine learning and data mining. Feature selection provides an effective way to solve this problem by removing irrelevant and redundant data, which can reduce computation time, improve learning accuracy, and facilitate a better understanding for the learning model or data. In this study, we discuss several frequently-used evaluation measures for feature selection, and then survey supervised, unsupervised, and semi-supervised feature selection methods, which are widely applied in machine learning problems, such as classification and clustering. Lastly, future challenges about feature selection are discussed.

Retail business analytics: Customer visit segmentation using market basket data

Basket analytics is a powerful tool in the retail context for acquiring knowledge about consumer shopping habits and preferences. In this paper, we propose a business analytics approach that mines customer visit segments from basket sales data. We characterize a customer visit by the purchased product categories in the basket and identify the shopping intention or mission behind the visit e.g. a ‘breakfast’ visit to purchase cereal, milk, bread, cheese etc. We also suggest a semi-supervised feature selection approach that uses the product taxonomy as input and suggests customized categories as output. This approach is utilized to balance the product taxonomy tree that has a significant effect on the data mining results. We demonstrate the utility of our approach by applying it to a real case of a major European fast-moving consumer goods (FMCG) retailer. Apart from its theoretical contribution, the proposed approach extracts knowledge that may support several decisions ranging from marketing campaigns per customer segment, redesign of a store's layout to product recommendations.

GMDH-Based Semi-Supervised Feature Selection for Electricity Load Classification Forecasting

With the development of smart power grids, communication network technology and sensor technology, there has been an exponential growth in complex electricity load data. Irregular electricity load fluctuations caused by the weather and holiday factors disrupt the daily operation of the power companies. To deal with these challenges, this paper investigates a day-ahead electricity peak load interval forecasting problem. It transforms the conventional continuous forecasting problem into a novel interval forecasting problem, and then further converts the interval forecasting problem into the classification forecasting problem. In addition, an indicator system influencing the electricity load is established from three dimensions, namely the load series, calendar data, and weather data. A semi-supervised feature selection algorithm is proposed to address an electricity load classification forecasting issue based on the group method of data handling (GMDH) technology. The proposed algorithm consists of three main stages: (1) training the basic classifier; (2) selectively marking the most suitable samples from the unclassified label data, and adding them to an initial training set; and (3) training the classification models on the final training set and classifying the test samples. An empirical analysis of electricity load dataset from four Chinese cities is conducted. Results show that the proposed model can address the electricity load classification forecasting problem more efficiently and effectively than the FW-Semi FS (forward semi-supervised feature selection) and GMDH-U (GMDH-based semi-supervised feature selection for customer classification) models.

A combined Fisher and Laplacian score for feature selection in QSAR based drug design using compounds with known and unknown activities.

Quantitative structure-activity relationship (QSAR) is an effective computational technique for drug design that relates the chemical structures of compounds to their biological activities. Feature selection is an important step in QSAR based drug design to select the most relevant descriptors. One of the most popular feature selection methods for classification problems is Fisher score which aim is to minimize the within-class distance and maximize the between-class distance. In this study, the properties of Fisher criterion were extended for QSAR models to define the new distance metrics based on the continuous activity values of compounds with known activities. Then, a semi-supervised feature selection method was proposed based on the combination of Fisher and Laplacian criteria which exploits both compounds with known and unknown activities to select the relevant descriptors. To demonstrate the efficiency of the proposed semi-supervised feature selection method in selecting the relevant descriptors, we applied the method and other feature selection methods on three QSAR data sets such as serine/threonine-protein kinase PLK3 inhibitors, ROCK inhibitors and phenol compounds. The results demonstrated that the QSAR models built on the selected descriptors by the proposed semi-supervised method have better performance than other models. This indicates the efficiency of the proposed method in selecting the relevant descriptors using the compounds with known and unknown activities. The results of this study showed that the compounds with known and unknown activities can be helpful to improve the performance of the combined Fisher and Laplacian based feature selection methods.

Semi-supervised adaptive feature analysis and its application for multimedia understanding

Multimedia understanding for high dimensional data is still a challenging work, due to redundant features, noises and insufficient label information it contains. Graph-based semi-supervised feature learning is an effective approach to address this problem. Nevertheless, Existing graph-based semi-supervised methods usually depend on the pre-constructed Laplacian matrix but rarely modify it in the subsequent classification tasks. In this paper, an adaptive local manifold learning based semi-supervised feature selection is proposed. Compared to the state-of-the-art, the proposed algorithm has two advantages: 1) Adaptive local manifold learning and feature selection are integrated jointly into a single framework, where both the labeled and unlabeled data are utilized. Besides, the correlations between different components are also considered. 2) A group sparsity constraint, i.e. l2?,?1-norm, is imposed to select the most relevant features. We also apply the proposed algorithm to serval kinds of multimedia understanding applications. Experimental results demonstrate the effectiveness of the proposed algorithm.

Simple strategies for semi-supervised feature selection

What is the simplest thing you can do to solve a problem? In the context of semi-supervised feature selection, we tackle exactly this—how much we can gain from two simple classifier-independent strategies. If we have some binary labelled data and some unlabelled, we could assume the unlabelled data are all positives, or assume them all negatives. These minimalist, seemingly naive, approaches have not previously been studied in depth. However, with theoretical and empirical studies, we show they provide powerful results for feature selection, via hypothesis testing and feature ranking. Combining them with some “soft” prior knowledge of the domain, we derive two novel algorithms (Semi-JMI, Semi-IAMB) that outperform significantly more complex competing methods, showing particularly good performance when the labels are missing-not-at-random. We conclude that simple approaches to this problem can work surprisingly well, and in many situations we can provably recover the exact feature selection dynamics, as if we had labelled the entire dataset.

GMDH-based semi-supervised feature selection for customer classification

Data dimension reduction is an important step for customer classification modeling, and feature selection has been a research focus of the data dimension reduction field. This study introduces the group method of data handling (GMDH), puts forward a GMDH-based semi-supervised feature selection (GMDH-SSFS) algorithm, and applies it to customer feature selection. The algorithm can utilize a few samples with class labels L, and a large number of samples without class labels U simultaneously. What is more, it considers the relationship between features and class labels, and that between features during feature selection. The GMDH-SSFS model mainly consists of three stages: 1) Train N basic classification models based on the dataset L with class labels; 2) Label samples selectively in the dataset U without class labels, and add them to L; 3) Train the GMDH neural network based on the new training set L, and select the optimal feature subset FS. Based on an empirical analysis of four customer classification datasets, results suggest that the features selected by the GMDH-SSFS model have a good explainability. Meanwhile, the customer classification performance of the classification model trained by the selected feature subset is superior to that of the models trained by the commonly used Laplacian score (an unsupervised feature selection algorithm), Fisher score (a supervised feature selection algorithm), and the FW-SemiFS and S3VM-FS (two semi-supervised feature selection algorithms).

Constraint score for semi-supervised feature selection in ligand-and receptor-based QSAR on serine/threonine-protein kinase PLK3 inhibitors

Constraint score is a feature selection method based on pairwise constraints which is used in classification problems. To the best of our knowledge, no work has used pairwise constraints for feature selection in QSAR models which have continuous values for activity of inhibitors. This study presented a semi-supervised constraint score for feature selection in ligand-and receptor-based QSAR. The semi-supervise constraint score used the supervision information of a small number of inhibitors with known inhibitory activity in the form of pairwise constraints beside the local structure of inhibitors with unknown inhibitory activity to evaluate descriptor importance. This method was used for selection of molecular structural descriptors and docking derived descriptors to predict the inhibitory activity of serine/threonine-protein kinase PLK3 inhibitors. The selected descriptors preserved the pairwise constraints defined on the inhibitors with known inhibitory activity and the local structure of inhibitors with unknown inhibitory activity. Performance comparison of the semi-supervised constraint score with other feature selection methods using RMSE, R2, and Q2ext values indicated its superiority in selecting relevant descriptors when a small number of inhibitors with known inhibitory activity are available. The results of this study showed that using the inhibitor with unknown inhibitory activity and a small number of supervision information in the form of pairwise constraints can achieve comparable or better performance than supervised and unsupervised methods.

Multimedia annotation via semi-supervised shared-subspace feature selection

With the rapid development of social network and computer technologies, we always confront with high-dimensional multimedia data. It is time-consuming and unrealistic to organize such a large amount of data. Most existing methods are not appropriate for large-scale data due to their dependence of Laplacian matrix on training data. Normally, a given multimedia sample is usually associated with multiple labels, which are inherently correlated to each other. Although traditional methods could solve this problem by translating it into several single-label problems, they ignore the correlation among different labels. In this paper, we propose a novel semi-supervised feature selection method and apply it to the multimedia annotation. Both labeled and unlabeled samples are sufficiently utilized without the need of graph construction, and the shared information between multiple labels is simultaneously uncovered. We apply the proposed algorithm to both web page and image annotation. Experimental results demonstrate the effectiveness of our method.

Semi-supervised feature selection with sparse representation for hyperspectral image classification

Dimensionality reduction is one of the most important steps for remotely sensed hyperspectral image classification. Feature selection as a kind of dimensionality reduction has attracted great attentions in the recent decades. In this paper, we proposed a novel feature selection method for hyperspectral image classification based on semi-supervised learning and sparsity score (or briefly called semi-supervised sparsity score (semi-SS)). In semi-SS, the pairwise constraints instead of class labels are used as the supervision information. Furthermore, the features chosen by Semi-SS have the ability to reconstruct the original data and sparsity preserving. Experiments conducted on two famous hyperspectral data sets illustrate that the proposed algorithm is remarkably effective in comparison to the existing feature selection methods.

Semi-supervised feature selection with sparse representation for hyperspectral image classification

Dimensionality reduction is one of the most important steps for remotely sensed hyperspectral image classification. Feature selection as a kind of dimensionality reduction has attracted great attentions in the recent decades. In this paper, we proposed a novel feature selection method for hyperspectral image classification based on semi-supervised learning and sparsity score (or briefly called semi-supervised sparsity score (semi-SS)). In semi-SS, the pairwise constraints instead of class labels are used as the supervision information. Furthermore, the features chosen by Semi-SS have the ability to reconstruct the original data and sparsity preserving. Experiments conducted on two famous hyperspectral data sets illustrate that the proposed algorithm is remarkably effective in comparison to the existing feature selection methods.

A Systematic Semi-Supervised Self-adaptable Fault Diagnostics approach in an evolving environment

Fault diagnostic methods are challenged by their applications to industrial components operating in evolving environments of their working conditions. To overcome this problem, we propose a Systematic Semi-Supervised Self-adaptable Fault Diagnostics approach (4SFD), which allows dynamically selecting the features to be used for performing the diagnosis, detecting the necessity of updating the diagnostic model and automatically updating it. Within the proposed approach, the main novelty is the semi-supervised feature selection method developed to dynamically select the set of features in response to the evolving environment. An artificial Gaussian and a real world bearing dataset are considered for the verification of the proposed approach.

A Survey on semi-supervised feature selection methods

Feature selection is a significant task in data mining and machine learning applications which eliminates irrelevant and redundant features and improves learning performance. In many real-world applications, collecting labeled data is difficult, while abundant unlabeled data are easily accessible. This motivates researchers to develop semi-supervised feature selection methods which use both labeled and unlabeled data to evaluate feature relevance. However, till-to-date, there is no comprehensive survey covering the semi-supervised feature selection methods. In this paper, semi-supervised feature selection methods are fully investigated and two taxonomies of these methods are presented based on two different perspectives which represent the hierarchical structure of semi-supervised feature selection methods. The first perspective is based on the basic taxonomy of feature selection methods and the second one is based on the taxonomy of semi-supervised learning methods. This survey can be helpful for a researcher to obtain a deep background in semi-supervised feature selection methods and choose a proper semi-supervised feature selection method based on the hierarchical structure of them.

Semi-supervised feature selection with exploiting shared information among multiple tasks

Given several related tasks, multi-task feature selection determines the importance of features by mining the correlations between them. There have already many efforts been made on the supervised multi-task feature selection. However, in real-world applications, it’s noticeably time-consuming and unpractical to collect sufficient labeled training data for each task. In this paper, we propose a novel feature selection algorithm, which integrates the semi-supervised learning and multi-task learning into a joint framework. Both the labeled and unlabeled samples are sufficiently utilized for each task, and the shared information between different tasks is simultaneously explored to facilitate decision making. Since the proposed objective function is non-smooth and difficult to be solved, we also design an efficient iterative algorithm to optimize it. Experimental results on different applications demonstrate the effectiveness of our algorithm.