Class-specific Feature Selection Research Articles

Multiclass liver disease prediction with adaptive data preprocessing and ensemble modeling

Liver disease includes a range of health conditions resulting from various factors that impact the normal functioning of the liver over an extended period. Effective treatment of Hepatitis C, a type of liver disease, can be significantly enhanced by accurately and timely predicting risk factors and severity, covering different stages like fibrosis and cirrhosis. This research utilizes the Hepatitis C dataset from the UCI repository to present a comprehensive framework for the prediction of liver disease across various stages. We have proposed an adaptive data preprocessing technique designed to enhance the efficacy of our foundational ML models. This method incorporates class specific mean imputation, outlier rejection, log normalization, feature selection, feature scaling and data balancing. The proposed preprocessing technique underwent rigorous analysis. Furthermore, we also proposed a set of ensemble models by combining basic ML classifiers to further improve the accuracy of liver disease predictions. The best model underwent rigorous parameter optimization and exhibited remarkable training and testing accuracy, reaching up to 99.87% and 99.80% respectively, outperforming earlier research on the dataset. A user-friendly interface was developed to enhance user interaction and practical use, enabling medical professionals to effortlessly enter patient information and promptly receive assessments regarding the risk factors associated with liver disorders.

A Class Specific Feature Selection Method for Improving the Performance of Text Classification

Recently, a significant amount of research work has been carried out in the field of feature selection. Although these methods help to increase the accuracy of the machine learning classification, the selected subset of features considers all the classes and may not select recommendable features for a particular class. The main goal of our paper is to propose a new class-specific feature selection algorithm that is capable of selecting an appropriate subset of features for each class. In this regard, we first perform class binarization and then select the best features for each class. During the feature selection process, we deal with class imbalance problems and redundancy elimination. The Weighted Average Voting Ensemble method is used for the final classification. Finally, we carry out experiments to compare our proposed feature selection approach with the existing popular feature selection methods. The results prove that our feature selection method outperforms the existing methods with an accuracy of more than 37%.

Class-specific feature selection via maximal dynamic correlation change and minimal redundancy

Class-specific feature selection via maximal dynamic correlation change and minimal redundancy

A class-specific feature selection and classification approach using neighborhood rough set and K-nearest neighbor theories

Rough set theories are utilized in class-specific feature selection to improve the classification performance of continuous data while handling data uncertainty. However, most of those approaches are converting continuous data into discrete or fuzzy data before applying rough set theories. These data conversions can reduce or change the meaning of data, as well as introduce unnecessary complexity to the feature domain. Therefore, in this study, we use neighborhood rough sets in class-specific feature selection to improve the classification performance without data conversions. As the standard classification algorithms are capable of handling a single feature set, we propose a novel classification algorithm based on the K-Nearest Neighbour algorithm to use class-specific feature subsets. Experimental evaluations prove that the proposed approach outperforms most of the baselines, and the selected feature sets are more effective than using the full feature sets in classification. The approach highly reduces the number of selected features and hence, can be used for effective data analysis of continuous data with high performance.

A class-specific metaheuristic technique for explainable relevant feature selection

A significant amount of previous research into feature selection has been aimed at developing methods that can derive variables that are relevant to an entire dataset. Although these approaches have revealed substantial improvements in classification accuracy, they have failed to address the problem of explainability of outputs. This paper seeks to address this problem of identifying explainable features using a class-specific feature selection method based on genetic algorithms and the one-vs-all strategy. Our proposed method finds relevant features for each class in the dataset and uses these features to enable more accurate classification, and also interpretation of the outputs. The results of our experiments demonstrate that the proposed method provides descriptive insights into prediction outputs, and also outperforms popular global feature selection techniques in the classifications of high dimensional and noisy datasets. Since there are no known challenging benchmark datasets for evaluating class-specific feature selection algorithms, this paper also recommends an approach for combining disparate datasets for this purpose.

Using Class-Specific Feature Selection for Cancer Detection with Gene Expression Profile Data of Platelets.

A novel multi-classification method, which integrates the elastic net and probabilistic support vector machine, was proposed to solve this problem in cancer detection with gene expression profile data of platelets, whose problems mainly are a kind of multi-class classification problem with high dimension, small samples, and collinear data. The strategy of one-against-all (OVA) was employed to decompose the multi-classification problem into a series of binary classification problems. The elastic net was used to select class-specific features for the binary classification problems, and the probabilistic support vector machine was used to make the outputs of the binary classifiers with class-specific features comparable. Simulation data and gene expression profile data were intended to verify the effectiveness of the proposed method. Results indicate that the proposed method can automatically select class-specific features and obtain better performance of classification than that of the conventional multi-class classification methods, which are mainly based on global feature selection methods. This study indicates the proposed method is suitable for general multi-classification problems featured with high-dimension, small samples, and collinear data.

A Sparse-Modeling Based Approach for Class Specific Feature Selection.

In this work, we propose a novel Feature Selection framework called Sparse-Modeling Based Approach for Class Specific Feature Selection (SMBA-CSFS), that simultaneously exploits the idea of Sparse Modeling and Class-Specific Feature Selection. Feature selection plays a key role in several fields (e.g., computational biology), making it possible to treat models with fewer variables which, in turn, are easier to explain, by providing valuable insights on the importance of their role, and likely speeding up the experimental validation. Unfortunately, also corroborated by the no free lunch theorems, none of the approaches in literature is the most apt to detect the optimal feature subset for building a final model, thus it still represents a challenge. The proposed feature selection procedure conceives a two-step approach: (a) a sparse modeling-based learning technique is first used to find the best subset of features, for each class of a training set; (b) the discovered feature subsets are then fed to a class-specific feature selection scheme, in order to assess the effectiveness of the selected features in classification tasks. To this end, an ensemble of classifiers is built, where each classifier is trained on its own feature subset discovered in the previous phase, and a proper decision rule is adopted to compute the ensemble responses. In order to evaluate the performance of the proposed method, extensive experiments have been performed on publicly available datasets, in particular belonging to the computational biology field where feature selection is indispensable: the acute lymphoblastic leukemia and acute myeloid leukemia, the human carcinomas, the human lung carcinomas, the diffuse large B-cell lymphoma, and the malignant glioma. SMBA-CSFS is able to identify/retrieve the most representative features that maximize the classification accuracy. With top 20 and 80 features, SMBA-CSFS exhibits a promising performance when compared to its competitors from literature, on all considered datasets, especially those with a higher number of features. Experiments show that the proposed approach may outperform the state-of-the-art methods when the number of features is high. For this reason, the introduced approach proposes itself for selection and classification of data with a large number of features and classes.

Multi-task learning for intelligent data processing in granular computing context

Classification is a popular task in many application areas, such as decision making, rating, sentiment analysis and pattern recognition. In the recent years, due to the vast and rapid increase in the size of data, classification has been mainly undertaken in the way of supervised machine learning. In this context, a classification task involves data labelling, feature extraction, feature selection and learning of classifiers. In traditional machine learning, data is usually single-labelled by experts, i.e., each instance is only assigned one class label, since experts assume that different classes are mutually exclusive and each instance is clear-cut. However, the above assumption does not always hold in real applications. For example, in the context of emotion detection, there could be more than one emotion identified from the same person. On the other hand, feature selection has typically been done by evaluating feature subsets in terms of their relevance to all the classes. However, it is possible that a feature is only relevant to one class, but is irrelevant to all the other classes. Based on the above argumentation on data labelling and feature selection, we propose in this paper a framework of multi-task learning. In particular, we consider traditional machine learning to be single task learning, and argue the necessity to turn it into multi-task learning to allow an instance to belong to more than one class (i.e., multi-task classification) and to achieve class specific feature selection (i.e., multi-task feature selection). Moreover, we report two experimental studies in terms of fuzzy multi-task classification and rule learning based multi-task feature selection. The results show empirically that it is necessary to undertake multi-task learning for both classification and feature selection.

EEF: Exponentially Embedded Families With Class-Specific Features for Classification

In this letter, we present a novel exponentially embedded families (EEF) based classification method, in which the probability density function (PDF) on raw data is estimated from the PDF on features. With the PDF construction, we show that class-specific features can be used in the proposed classification method, instead of a common feature subset for all classes as used in conventional approaches. We apply the proposed EEF classifier for text categorization as a case study and derive an optimal Bayesian classification rule with class-specific feature selection based on the Information Gain (IG) score. The promising performance on real-life data sets demonstrates the effectiveness of the proposed approach and indicates its wide potential applications.

Semi-supervised class-specific feature selection for VHR remote sensing images

ABSTRACTFeatures relevant to a thematic class, that is, class-specific features are beneficial to thematic information extraction. However, existing class-specific feature selection methods require abundant labelled samples, while sample labelling is always labour intensive and time consuming. Therefore, it is necessary to select class-specific features with insufficient labelled objects. In this paper, we raise this problem as semi-supervised class-specific feature selection and propose a new two-stage method. First, a weight matrix fully integrates local geometrical structure and discriminative information. Second, the weight matrix is incorporated into a -norm minimization optimization problem of data reconstruction to objectively measure the effectiveness of features for a thematic class. Different from the explicit binarization in the label vector, the new method only implicitly employs binarization in the weight matrix. With area under receiver-operating characteristic curve, class-specific features result in an increase from 3% and 4% on average for Bayes and linear support vector machine, respectively.

Measuring the Effectiveness of Various Features for Thematic Information Extraction From Very High Resolution Remote Sensing Imagery

Generally, some object-based features are more relevant to a thematic class than other features. These strongly relevant features, termed as class-specific features, would significantly contribute to thematic information extraction for very high resolution (VHR) images. However, many existing feature selection methods have been designed to select a good feature subset for all classes, rather than an independent feature subset for the thematic class. The latter might better meet the requirement of thematic information extraction than the former. In addition, the lack of quantitative evaluation of the contribution of the selected features to thematic classes also weakens our understandability of these features. To address the problems, class-specific feature selection methods are developed to measure the effectiveness of features for extracting thematic information from VHR images. First, the one-versus-all scheme is combined with traditional feature selection methods, such as ReliefF and LeastC. Also, one-versus-one scheme is utilized for alleviating the negative impact of a class imbalance problem arising from the one-versus-all scheme. Then, the relative contributions of features to thematic classes are obtained by the class-specific feature selection methods to describe the effectiveness of features for thematic information extraction. Finally, the class-specific feature selection methods are compared with the original methods on three different VHR image data sets by the nearest neighbor and support vector machine. Experimental results show that the class-specific feature selection methods outperform the corresponding conventional methods, and the one-versus-one scheme surpasses one-versus-all scheme. Additionally, many features are evaluated by the class-specific feature selection methods, to provide end users advice on effectiveness of the features.

Class-Specific Feature Selection With Local Geometric Structure and Discriminative Information Based on Sparse Similar Samples

It is necessary while quite challenging to select features strongly relevant to a thematic class, i.e., class-specific features, from very high resolution (VHR) remote sensing images. To meet this challenge, a class-specific feature selection method based on sparse similar samples (CFS4) is proposed. Specifically, CFS4 incorporates the local geometrical structure and discriminative information of the data into a sparsity regularization problem. The experimental results on VHR satellite images well validate the effectiveness and practicability of the proposed method.

RELIEF-MM: effective modality weighting for multimedia information retrieval

Fusing multimodal information in multimedia data usually improves the retrieval performance. One of the major issues in multimodal fusion is how to determine the best modalities. To combine the modalities more effectively, we propose a RELIEF-based modality weighting approach, named as RELIEF-MM. The original RELIEF algorithm is extended for weaknesses in several major issues: class-specific feature selection, complexities with multi-labeled data and noise, handling unbalanced datasets, and using the algorithm with classifier predictions. RELIEF-MM employs an improved weight estimation function, which exploits the representation and reliability capabilities of modalities, as well as the discrimination capability, without any increase in the computational complexity. The comprehensive experiments conducted on TRECVID 2007, TRECVID 2008 and CCV datasets validate RELIEF-MM as an efficient, accurate and robust way of modality weighting for multimedia data.

Methods for pattern selection, class-specific feature selection and classification for automated learning

This paper presents methods for training pattern (prototype) selection, class-specific feature selection and classification for automated learning. For training pattern selection, we propose a method of sampling that extracts a small number of representative training patterns (prototypes) from the dataset. The idea is to extract a set of prototype training patterns that represents each class region in a classification problem. In class-specific feature selection, we try to find a separate feature set for each class such that it is the best one to separate that class from the other classes. We then build a separate classifier for that class based on its own feature set. The paper also presents a new hypersphere classification algorithm. Hypersphere nets are similar to radial basis function (RBF) nets and belong to the group of kernel function nets. Polynomial time complexity of the methods is proven. Polynomial time complexity of learning algorithms is important to the field of neural networks. Computational results are provided for a number of well-known datasets. None of the parameters of the algorithm were fine tuned for any of the problems solved and this supports the idea of automation of learning methods. Automation of learning is crucial to wider deployment of learning technologies.

Class Specific Feature Selection for Identity Validation using Dynamic Signatures

Classification of the biometrics data for identity validation can be modeled as a single-class problem, where the identity is confirmed by comparing the biometrics of the unknown person with those of the claimed identity. However, current feature selection techniques do not differentiate between single-class and multi-class problems when determining the suitable feature set and select the feature-set that is suitable for representing or discriminating for all the available classes. This may not be the best representation of the biometrics data of an individual because different people may have differences in the most suitable features to represent their biometrical data. In this paper, a class-specific feature selection method has been proposed and experimentally validated using dynamic signatures. This method is based on the coefficient of variance within the feature set, where the features with smaller variance are selected and the ones with larger variance are rejected. The proposed technique was compared with the other feature selection methods, and the results show that a significant improvement in the classification accuracy, specificity and sensitivity was obtained when using class-specific feature selection.

Retrieval of logically relevant 3D human motions by Adaptive Feature Selection with Graded Relevance Feedback

A system that can retrieve logically relevant 3D captured motions is useful in game and animation production. We presented a robust logical relevance metric based on the relative distances among the joints. Existing methods select a universal subset of features for all kinds of queries which may not well characterize the variations in different queries. To break through this limitation we proposed an Adaptive Feature Selection (AFS) method that abstracts the characteristics of the query by a Linear Regression Model, and different feature subsets can be selected according to the properties of the specific query. With a Graded Relevance Feedback (GRF) algorithm, we refined the feature subset that enhances the retrieval performance according to the graded relevance of the feedback samples. With an ontology that predefines the logical relevance between motion classes in terms of graded relevance, the performance of the proposed AFS–GRF algorithm is evaluated and shown to outperform other class-specific feature selection and motion retrieval methods.

General framework for class-specific feature selection

Commonly, when a feature selection algorithm is applied, a single feature subset is selected for all the classes, but this subset could be inadequate for some classes. Class-specific feature selection allows selecting a possible different feature subset for each class. However, all the class-specific feature selection algorithms have been proposed for a particular classifier, which reduce their applicability. In this paper, a general framework for using any traditional feature selector for doing class-specific feature selection, which allows using any classifier, is proposed. Experimental results and a comparison against traditional feature selectors showing the suitability of the proposed framework are included.