Semi-supervised Feature Selection Algorithm Research Articles

Automatic focal EEG identification based on deep reinforcement learning

Electroencephalogram (EEG) signals convey information about the electrical activity of neurons and are commonly used in clinical practice to evaluate the epileptic activity of patients. There is a part of the human brain that is closely linked to epileptic activity, namely the epileptogenic zone. Successful resection of the epileptogenic zone requires high precision classification of focal (F) and non-focal (NF) EEG signals. In this paper, we improve the Deep Q-Network (DQN) by adding Additional Functional Modules (AFM) to propose a novel focal EEG recognition method which is named AFM-DQN. Compared to the traditional Reinforcement Learning (RL), this model incorporates a deep convolutional neural network (CNN), and unifies the decision making capability of Q-learning and the perceptual capability of CNN, which greatly improves the learning of this network. The AFM, including pre-training, the high performance classifier (HPC), reward control mechanism and three RL related techniques, imparts this model a stronger generalization ability. To reduce computational burden of the network while preserving the correlation and interdependence of EEG signals, we introduce different kinds of EEG features and a semi-supervised feature selection algorithm based on joint mutual information (Semi-JMI algorithm). Experiments on the two publicly available EEG databases have achieved classification accuracy of 95.87% and 97.5%, respectively, which demonstrates that the proposed method has the potential for clinical application of accurate localization of epileptogenic zone. Compared with other published methods, this method has better performance and generalization ability.

Robust fuzzy rough approximations with kNN granules for semi-supervised feature selection

Fuzzy rough set theory has attracted much attention because of its successful application in uncertainty measurement. To improve the efficiency and robustness of uncertainty measure using the theory, robust fuzzy rough approximations have become popular research. In this work, it first designs a novel uncertainty measure based on kNN granules. By combining the relative distance, the method is effectively applicable to the datasets with large density variation. Using the measure a robust FRS model is studied and simply denoted KNN-FRS model. The new model takes a compromise between the maximum lower approximation degree and the minimum classification loss in computing fuzzy rough approximations, which reduces the sensitivity of the classical FRS model to noise. Besides, some properties of KNN-FRS model are summarized. Next, KNN-FRS model is used to design a feature selection algorithm which is tested to validate the feasibility of the KNN-FRS model. And then the KNN-FRS model is applied to design a semi-supervised feature selection algorithm which selects features while labeling samples and the labeling method is designed based on the proposed uncertainty measure. Some experiments are conducted to validate the KNN-FRS model and the designed algorithms. The results illustrate the proposed KNN-FRS model is feasible, effective and robust. And the designed semi-supervised feature selection algorithm performs excellent performance.

Semi-supervised feature selection for partially labeled mixed-type data based on multi-criteria measure approach

In many real applications, the data are always collected from different types and they are subjected to obtain partial labeling information of objects. Such data are referred to as partially labeled mixed-type data. There is currently few work on feature selection approaches for these data. Motivated by this issue, this paper aims at selecting the informative feature subset from partially labeled mixed-type data. At first, to improve the classification performance, an improved label propagation algorithm based on K-nearest neighbor is proposed, which marks the decision labels of unlabeled objects by making use of the information between unlabeled objects and labeled objects. On this basis, a feature multi-criteria measure based on the dependency, information entropy and information granulation is proposed for selecting candidate features. Finally, the corresponding semi-supervised feature selection algorithm is developed to select the feature subset for the partially labeled mixed-type data. Experimental results on UCI data sets demonstrate the effectiveness of the proposed feature selection algorithm and the superiority in terms of the classification accuracy compared with other algorithms.

Mechanism of rainfall induced landslides in Yunnan Province using multi-scale spatiotemporal analysis and remote sensing interpretation

In order to explore the disaster mechanism of rainfall induced landslides in Yunnan Province, a new method is developed in the field of landslide research, which provides a strong basis for the study of rainfall induced landslide disaster mechanism in different regions. First, the mechanism of rainfall induced landslide initiation and the mechanism of rainfall impacting slope are introduced, and the temporal and spatial characteristics of landslide disasters in Yunnan Province are analyzed comprehensively. The influence of rainfall on landslide density is analyzed in 8 regions randomly, and a semi supervised feature selection algorithm based on manifold is proposed to optimize the remote sensing image of Zhaojiapo rainfall type landslide area in Yunnan Province. The results show that in the province the monthly rainfall days account for a large proportion from June to September, and the months with the largest number of landslides in the whole year are from June to August. The seasonal change of rainfall belt and the frequent occurrence time of landslide disasters and the spatial characteristics of landslide distribution and rainfall distribution basically belong to corresponding relationship; the greater the rainfall of 8 regions, the greater the probability of landslide occurrence, and the greater the landslide density, which belongs to the proportional relationship; the landform characteristics in the optimized remote sensing image can be clearly displayed, and the classification accuracy of the data is enhanced. The results show that based on multi-scale spatiotemporal analysis combined with optimized remote sensing technology, the cause of rainfall induced landslides in Yunnan Province is the change of landslide density caused by heavy rainfall, which eventually leads to disasters. It is conducive to scientifically understanding the inducement of rainfall induced landslides in Yunnan Province, provides a scientific basis for further disaster control and prevention, and provides a new possible method for landslide real-time prediction and alarm system.

Multi-view adaptive semi-supervised feature selection with the self-paced learning

In recent decades, semi-supervised feature selection (SSFS) has gained extensive research in the field of machine learning and computer vision. Most SSFS algorithms are based on graph-based semi-supervised learning (GSSL), and their performance depend heavily on the quality of the Laplacian weight graph. However, the Laplacian weight graph can't be changed once it is constructed, which greatly restricts the performance of SSFS. To address this defect, in this paper we propose a novel Multi-view Adaptive Semi-supervised Feature Selection (MASFS) algorithm, which introduces the self-paced learning (SPL) into SSFS to make the Laplacian weight graph adaptively change according to the current predicted information. Meanwhile, the MASFS algorithm utilizes the multi-view learning to effectively explore the complementary and related information contained in different views to enhance SSFS performance. We propose a valid iterative algorithm for optimizing the objective function, followed by the convergence analysis and the complexity analysis. To illustrate the effectiveness of the MASFS algorithm, some experiments are carried out on NUS-WIDE dataset and MSRA-MM2.0 dataset and the experimental results indicate that MASFS has better performance than other SSFS algorithms based on GSSL.

Joint Semi-Supervised Feature Selection and Classification through Bayesian Approach

With the increasing data dimensionality, feature selection has become a fundamental task to deal with high-dimensional data. Semi-supervised feature selection focuses on the problem of how to learn a relevant feature subset in the case of abundant unlabeled data with few labeled data. In recent years, many semi-supervised feature selection algorithms have been proposed. However, these algorithms are implemented by separating the processes of feature selection and classifier training, such that they cannot simultaneously select features and learn a classifier with the selected features. Moreover, they ignore the difference of reliability inside unlabeled samples and directly use them in the training stage, which might cause performance degradation. In this paper, we propose a joint semi-supervised feature selection and classification algorithm (JSFS) which adopts a Bayesian approach to automatically select the relevant features and simultaneously learn a classifier. Instead of using all unlabeled samples indiscriminately, JSFS associates each unlabeled sample with a self-adjusting weight to distinguish the difference between them, which can effectively eliminate the irrelevant unlabeled samples via introducing a left-truncated Gaussian prior. Experiments on various datasets demonstrate the effectiveness and superiority of JSFS.

Local adaptive learning for semi-supervised feature selection with group sparsity

Feature selection is often an important tool for many machine learning and data mining tasks. By largely removing the irrelevant features and reducing the complexity of the data processing, feature selection can significantly improve the performance of subsequent classification or clustering tasks. As a result of the rapid development of social networking, large amounts of high-dimensional data have been generated. Due to the high cost of collecting sufficient labels, graph-based semi-supervised feature selection algorithms have attracted the most research interest; however, these approaches neglect the local sparsity of data. Accordingly, motivated by the merits of adaptive learning and sparse learning, we propose a novel feature selection method with a local adaptive loss function and a global sparsity constraint in this paper. Our method can operate more flexibly to model data with different distributions. Moreover, when both the local and global sparsity of data is considered, our method is more capable of selecting the most discriminating features. Experimental results on various real-world applications demonstrate the effectiveness of the proposed feature selection method compared to several state-of-the-art methods.

GMDH-Based Semi-Supervised Feature Selection for Electricity Load Classification Forecasting

With the development of smart power grids, communication network technology and sensor technology, there has been an exponential growth in complex electricity load data. Irregular electricity load fluctuations caused by the weather and holiday factors disrupt the daily operation of the power companies. To deal with these challenges, this paper investigates a day-ahead electricity peak load interval forecasting problem. It transforms the conventional continuous forecasting problem into a novel interval forecasting problem, and then further converts the interval forecasting problem into the classification forecasting problem. In addition, an indicator system influencing the electricity load is established from three dimensions, namely the load series, calendar data, and weather data. A semi-supervised feature selection algorithm is proposed to address an electricity load classification forecasting issue based on the group method of data handling (GMDH) technology. The proposed algorithm consists of three main stages: (1) training the basic classifier; (2) selectively marking the most suitable samples from the unclassified label data, and adding them to an initial training set; and (3) training the classification models on the final training set and classifying the test samples. An empirical analysis of electricity load dataset from four Chinese cities is conducted. Results show that the proposed model can address the electricity load classification forecasting problem more efficiently and effectively than the FW-Semi FS (forward semi-supervised feature selection) and GMDH-U (GMDH-based semi-supervised feature selection for customer classification) models.

Multiview Hessian Semisupervised Sparse Feature Selection for Multimedia Analysis

Facing a large number of unlabeled data and a small number of labeled data, semisupervised sparse feature selection has received increasing attention in recent years. However, most semisupervised feature selection algorithms are developed for single-view data and cannot naturally handle multiview data. Moreover, most existing semisupervised sparse feature selection methods are based on Laplacian regularization, which is a lack of extrapolating power. To overcome the above-mentioned drawbacks, we present a multiview Hessian semi-supervised sparse feature selection (MHSFS) framework in this paper. MHSFS can directly accomplish multiview sparse feature selection by exploiting multiview learning to reveal and leverage the correlated and complemental information among different views. In addition, MHSFS can achieve better performance based on Hessian regularization, which favors functions whose values linearly vary with respect to geodesic distance and preserves the local manifold structure well. A simple yet efficient iterative method is proposed to solve the objective function, followed by convergence analysis. We apply the proposed method into different multimedia analysis tasks, such as image annotation, video concept detection, and 3D motion analysis. The results show that MHSFS outperforms the state-of-the-art sparse feature selection methods and achieves good performance.

GMDH-based semi-supervised feature selection for customer classification

Data dimension reduction is an important step for customer classification modeling, and feature selection has been a research focus of the data dimension reduction field. This study introduces the group method of data handling (GMDH), puts forward a GMDH-based semi-supervised feature selection (GMDH-SSFS) algorithm, and applies it to customer feature selection. The algorithm can utilize a few samples with class labels L, and a large number of samples without class labels U simultaneously. What is more, it considers the relationship between features and class labels, and that between features during feature selection. The GMDH-SSFS model mainly consists of three stages: 1) Train N basic classification models based on the dataset L with class labels; 2) Label samples selectively in the dataset U without class labels, and add them to L; 3) Train the GMDH neural network based on the new training set L, and select the optimal feature subset FS. Based on an empirical analysis of four customer classification datasets, results suggest that the features selected by the GMDH-SSFS model have a good explainability. Meanwhile, the customer classification performance of the classification model trained by the selected feature subset is superior to that of the models trained by the commonly used Laplacian score (an unsupervised feature selection algorithm), Fisher score (a supervised feature selection algorithm), and the FW-SemiFS and S3VM-FS (two semi-supervised feature selection algorithms).

Unified discriminating feature analysis for visual category recognition

Visual category recognition (VCR) is one of the most important tasks in image and video indexing. To deal with high dimension image/video data, feature analysis algorithms have been widely used for visual category recognition. In this paper, to enhance the flexibility regarding the exploitation of labeled or unlabeled data, we propose a unified feature analysis framework that can be applied to both supervised and semi-supervised scenarios. Furthermore, by revealing intrinsic relationships of traditional feature analysis methods, our framework not only integrates traditional methods, but also introduces an ℓ2,1-norm regularization term for sparse learning. Extensive experiments report that the proposed method obtains advantageous performance in comparison with other state-of-the-art supervised and semi-supervised feature selection algorithms.

Semi-supervised feature selection for audio classification based on constraint compensated Laplacian score

Audio classification, classifying audio segments into broad categories such as speech, non-speech, and silence, is an important front-end problem in speech signal processing. Dozens of features have been proposed for audio classification. Unfortunately, these features are not directly complementary and combining them does not improve classification performance. Feature selection provides an effective mechanism for choosing the most relevant and least redundant features for classification. In this paper, we present a semi-supervised feature selection algorithm named Constraint Compensated Laplacian score (CCLS), which takes advantage of the local geometrical structure of unlabeled data as well as constraint information from labeled data. We apply this method to the audio classification task and compare it with other known feature selection methods. Experimental results demonstrate that CCLS gives substantial improvement.

Compact representation for large-scale unconstrained video analysis

Recently, newly invented features (e.g. Fisher vector, VLAD) have achieved state-of-the-art performance in large-scale video analysis systems that aims to understand the contents in videos, such as concept recognition and event detection. However, these features are in high-dimensional representations, which remarkably increases computation costs and correspondingly deteriorates the performance of subsequent learning tasks. Notably, the situation becomes even worse when dealing with large-scale video data where the number of class labels are limited. To address this problem, we propose a novel algorithm to compactly represent huge amounts of unconstrained video data. Specifically, redundant feature dimensions are removed by using our proposed feature selection algorithm. Considering unlabeled videos that are easy to obtain on the web, we apply this feature selection algorithm in a semi-supervised framework coping with a shortage of class information. Different from most of the existing semi-supervised feature selection algorithms, our proposed algorithm does not rely on manifold approximation, i.e. graph Laplacian, which is quite expensive for a large number of data. Thus, it is possible to apply the proposed algorithm to a real large-scale video analysis system. Besides, due to the difficulty of solving the non-smooth objective function, we develop an efficient iterative approach to seeking the global optimum. Extensive experiments are conducted on several real-world video datasets, including KTH, CCV, and HMDB. The experimental results have demonstrated the effectiveness of the proposed algorithm.

Sketch-based human motion retrieval via selected 2D geometric posture descriptor

Sketch-based human motion retrieval is a hot topic in computer animation in recent years. In this paper, we present a novel sketch-based human motion retrieval method via selected 2-dimensional (2D) Geometric Posture Descriptor (2GPD). Specially, we firstly propose a rich 2D pose feature call 2D Geometric Posture Descriptor (2GPD), which is effective in encoding the 2D posture similarity by exploiting the geometric relationships among different human body parts. Since the original 2GPD is of high dimension and redundant, a semi-supervised feature selection algorithm derived from Laplacian Score is then adopted to select the most discriminative feature component of 2GPD as feature representation, and we call it as selected 2GPD. Finally, a posture-by-posture motion retrieval algorithm is used to retrieve a motion sequence by sketching several key postures. Experimental results on CMU human motion database demonstrate the effectiveness of our proposed approach.

A Convex Formulation for Semi-Supervised Multi-Label Feature Selection

Explosive growth of multimedia data has brought challenge of how to efficiently browse, retrieve and organize these data. Under this circumstance, different approaches have been proposed to facilitate multimedia analysis. Several semi-supervised feature selection algorithms have been proposed to exploit both labeled and unlabeled data. However, they are implemented based on graphs, such that they cannot handle large-scale datasets. How to conduct semi-supervised feature selection on large-scale datasets has become a challenging research problem. Moreover, existing multi-label feature selection algorithms rely on eigen-decomposition with heavy computational burden, which further prevent current feature selection algorithms from being applied for big data. In this paper, we propose a novel convex semi-supervised multi-label feature selection algorithm, which can be applied to large-scale datasets. We evaluate performance of the proposed algorithm over five benchmark datasets and compare the results with state-of-the-art supervised and semi-supervised feature selection algorithms as well as baseline using all features. The experimental results demonstrate that our proposed algorithm consistently achieve superiors performances.

A graph Laplacian based approach to semi-supervised feature selection for regression problems

Feature selection is a task of fundamental importance for many data mining or machine learning applications, including regression. Surprisingly, most of the existing feature selection algorithms assume the problems to address are either supervised or unsupervised, while supervised and unsupervised samples are often simultaneously available in real-world applications. Semi-supervised feature selection methods are thus necessary, and many solutions have been proposed recently. However, almost all of them exclusively tackle classification problems. This paper introduces a semi-supervised feature selection algorithm which is specifically designed for regression problems. It relies on the notion of Laplacian score, a quantity recently introduced in the unsupervised framework. Experimental results demonstrate the efficiency of the proposed algorithm.

A Semisupervised Feature Selection with Support Vector Machine

Feature selection has proved to be a beneficial tool in learning problems with the main advantages of interpretation and generalization. Most existing feature selection methods do not achieve optimal classification performance, since they neglect the correlations among highly correlated features which all contribute to classification. In this paper, a novel semisupervised feature selection algorithm based on support vector machine (SVM) is proposed, termed SENFS. In order to solve SENFS, an efficient algorithm based on the alternating direction method of multipliers is then developed. One advantage of SENFS is that it encourages highly correlated features to be selected or removed together. Experimental results demonstrate the effectiveness of our feature selection method on simulation data and benchmark data sets.

Efficient semi-supervised feature selection with noise insensitive trace ratio criterion

Feature selection is an effective method to deal with high-dimensional data. While in many applications such as multimedia and web mining, the data are often high-dimensional and very large scale, but the labeled data are often very limited. On these kind of applications, it is important that the feature selection algorithm is efficient and can explore labeled data and unlabeled data simultaneously. In this paper, we target on this problem and propose an efficient semi-supervised feature selection algorithm to select relevant features using both labeled and unlabeled data. First, we analyze a popular trace ratio criterion in the dimensionality reduction, and point out that the trace ratio criterion tends to select features with very small variance. To solve this problem, we propose a noise insensitive trace ratio criterion for feature selection with a re-scale preprocessing. Interestingly, the feature selection with the noise insensitive trace ratio criterion can be much more efficiently solved. Based on the noise insensitive trace ratio criterion, we propose a new semi-supervised feature selection algorithm. The algorithm fully explores the distribution of the labeled and unlabeled data with a special label propagation method. Experimental results verify the effectiveness of the proposed algorithm, and show improvement over traditional supervised feature selection algorithms.

Discriminative Semi-Supervised Feature Selection Via Manifold Regularization

Feature selection has attracted a huge amount of interest in both research and application communities of data mining. We consider the problem of semi-supervised feature selection, where we are given a small amount of labeled examples and a large amount of unlabeled examples. Since a small number of labeled samples are usually insufficient for identifying the relevant features, the critical problem arising from semi-supervised feature selection is how to take advantage of the information underneath the unlabeled data. To address this problem, we propose a novel discriminative semi-supervised feature selection method based on the idea of manifold regularization. The proposed approach selects features through maximizing the classification margin between different classes and simultaneously exploiting the geometry of the probability distribution that generates both labeled and unlabeled data. In comparison with previous semi-supervised feature selection algorithms, our proposed semi-supervised feature selection method is an embedded feature selection method and is able to find more discriminative features. We formulate the proposed feature selection method into a convex-concave optimization problem, where the saddle point corresponds to the optimal solution. To find the optimal solution, the level method, a fairly recent optimization method, is employed. We also present a theoretic proof of the convergence rate for the application of the level method to our problem. Empirical evaluation on several benchmark data sets demonstrates the effectiveness of the proposed semi-supervised feature selection method.

Locality sensitive semi-supervised feature selection

In many computer vision tasks like face recognition and image retrieval, one is often confronted with high-dimensional data. Procedures that are analytically or computationally manageable in low-dimensional spaces can become completely impractical in a space of several hundreds or thousands dimensions. Thus, various techniques have been developed for reducing the dimensionality of the feature space in the hope of obtaining a more manageable problem. The most popular feature selection and extraction techniques include Fisher score, Principal Component Analysis (PCA), and Laplacian score. Among them, PCA and Laplacian score are unsupervised methods, while Fisher score is supervised method. None of them can take advantage of both labeled and unlabeled data points. In this paper, we introduce a novel semi-supervised feature selection algorithm, which makes use of both labeled and unlabeled data points. Specifically, the labeled points are used to maximize the margin between data points from different classes, while the unlabeled points are used to discover the geometrical structure of the data space. We compare our proposed algorithm with Fisher score and Laplacian score on face recognition. Experimental results demonstrate the efficiency and effectiveness of our algorithm.