Huge Number Of Features Research Articles

Gene targeting in amyotrophic lateral sclerosis using causality-based feature selection and machine learning

BackgroundAmyotrophic lateral sclerosis (ALS) is a rare progressive neurodegenerative disease that affects upper and lower motor neurons. As the molecular basis of the disease is still elusive, the development of high-throughput sequencing technologies, combined with data mining techniques and machine learning methods, could provide remarkable results in identifying pathogenetic mechanisms. High dimensionality is a major problem when applying machine learning techniques in biomedical data analysis, since a huge number of features is available for a limited number of samples. The aim of this study was to develop a methodology for training interpretable machine learning models in the classification of ALS and ALS-subtypes samples, using gene expression datasets.MethodsWe performed dimensionality reduction in gene expression data using a semi-automated preprocessing systematic gene selection procedure using Statistically Equivalent Signature (SES), a causality-based feature selection algorithm, followed by Boosted Regression Trees (XGBoost) and Random Forest to train the machine learning classifiers. The SHapley Additive exPlanations (SHAP values) were used for interpretation of the machine learning classifiers. The methodology was developed and tested using two distinct publicly available ALS RNA-seq datasets. We evaluated the performance of SES as a dimensionality reduction method against: (a) Least Absolute Shrinkage and Selection Operator (LASSO), and (b) Local Outlier Factor (LOF).ResultsThe proposed methodology achieved 85.18% accuracy for the classification of cerebellum or frontal cortex samples as C9orf72-related familial ALS, sporadic ALS or healthy samples. Importantly, the genes identified as the most determinative have also been reported as disease-associated in ALS literature. When tested in the evaluation dataset, the methodology achieved 88.89% accuracy for the classification of sporadic ALS motor neuron samples. When LASSO was used as feature selection method instead of SES, the accuracy of the machine learning classifiers ranged from 74.07 to 96.30%, depending on tissue assessed, while LOF underperformed significantly (77.78% accuracy for the classification of pooled cerebellum and frontal cortex samples).ConclusionsUsing SES, we addressed the challenge of high dimensionality in gene expression data analysis, and we trained accurate machine learning ALS classifiers, specific for the gene expression patterns of different disease subtypes and tissue samples, while identifying disease-associated genes.

Understanding How CNNs Recognize Facial Expressions: A Case Study with LIME and CEM.

Recognizing facial expressions has been a persistent goal in the scientific community. Since the rise of artificial intelligence, convolutional neural networks (CNN) have become popular to recognize facial expressions, as images can be directly used as input. Current CNN models can achieve high recognition rates, but they give no clue about their reasoning process. Explainable artificial intelligence (XAI) has been developed as a means to help to interpret the results obtained by machine learning models. When dealing with images, one of the most-used XAI techniques is LIME. LIME highlights the areas of the image that contribute to a classification. As an alternative to LIME, the CEM method appeared, providing explanations in a way that is natural for human classification: besides highlighting what is sufficient to justify a classification, it also identifies what should be absent to maintain it and to distinguish it from another classification. This study presents the results of comparing LIME and CEM applied over complex images such as facial expression images. While CEM could be used to explain the results on images described with a reduced number of features, LIME would be the method of choice when dealing with images described with a huge number of features.

A screening method for ultra-high dimensional features with overlapped partition structures.

Ultra-high dimensional data, such as gene and neuroimaging data, are becoming increasingly important in biomedical science. Identifying important biomarkers from the huge number of features can help us gain better insights into further researches. Variable screening is an efficient tool to achieve this goal under the large scale cases, which reduces the dimension of features into a moderate size by removing the major part of inactive ones. Developing novel variable screening methods for high-dimensional features with group structures is challenging, especially under the overlapped cases. For example, the huge-scaled genes usually can be partitioned into hundreds of pathways according to background knowledge. One primary characteristic for this type of data is that many genes may appear across more than one pathway, which means that different pathways are overlapped. However, existing variable screening methods only could deal with disjoint group structure cases. To fill this gap, we propose a novel variable screening method for the generalized linear model by incorporating overlapped partition structures with theoretical guarantee. Besides the sure screening property, we also test the performance of the proposed method through a series of numerical studies and apply it to statistical analysis of a breast cancer data.

A Study on Facial Expression Change Detection Using Machine Learning Methods with Feature Selection Technique

Along with the fourth industrial revolution, research in the biomedical engineering field is being actively conducted. Among these research fields, the brain–computer interface (BCI) research, which studies the direct interaction between the brain and external devices, is in the spotlight. However, in the case of electroencephalograph (EEG) data measured through BCI, there are a huge number of features, which can lead to many difficulties in analysis because of complex relationships between features. For this reason, research on BCIs using EEG data is often insufficient. Therefore, in this study, we develop the methodology for selecting features for a specific type of BCI that predicts whether a person correctly detects facial expression changes or not by classifying EEG-based features. We also investigate whether specific EEG features affect expression change detection. Various feature selection methods were used to check the influence of each feature on expression change detection, and the best combination was selected using several machine learning classification techniques. As a best result of the classification accuracy, 71% of accuracy was obtained with XGBoost using 52 features. EEG topography was confirmed using the selected major features, showing that the detection of changes in facial expression largely engages brain activity in the frontal regions.

Sparse elastic net multi-label rank support vector machine with pinball loss and its applications

Multi-label rank support vector machine (RankSVM) is an effective technique to deal with multi-label classification problems, which has been widely used in various fields. However, it is sensitive to noise points and cannot delete redundant features for high dimensional problems. Therefore, to address the above two limitations, a sparse elastic net multi-label RankSVM with pinball loss (pin-ENR) is first proposed in this paper. On the one hand, pinball loss is employed to enhance the robustness. On the other hand, it adopts the sparse elastic net regularization, so that it can do variable selection. However, it still has challenges for large-scale problems with a huge number of features, samples, and labels. Therefore, motivated by the sparsity of pin-ENR, a safe simultaneous feature and label-pair elimination rule is further constructed for accelerating pin-ENR, which is termed as FLER-pin-ENR. Its main idea is to delete a large number of inactive features and label-pairs simultaneously before training without sacrificing accuracy. Numerical experiments on four synthetic and seven benchmark datasets demonstrate the feasibility and validity. Moreover, we apply our FLER-pin-ENR to the diagnosis of diabetes complications and the natural scene image classification problems, which further verifies the practicability of our proposed method.

Feature Mapping and Deep Long Short Term Memory Network-Based Efficient Approach for Parkinson’s Disease Diagnosis

In this paper, a novel approach was developed for Parkinson’s disease (PD) diagnosis based on speech disorders. When the literature about the speech disorders-based PD diagnosis was reviewed, it was seen that the most of approaches were concentrated on the feature selection as the datasets contained a huge number of features. In contrast, in the proposed approach, instead of eliminating some of the features by using any feature selection method, all features were initially used for forming a mapping procedure where the input feature vectors were converted to the input images. Then, a deep Long Short Term Memory (LSTM) network was employed for PD detection where the obtained images were used. The deep LSTM network carried out both feature extraction and classification processes and its training was carried out in an end-to-end fashion. The activations in the convolutional layer were converted to sequence data through the sequence-folding and sequence-unfolding layers. The activations in the LSTM output with learning parameters were conveyed to the Softmax layer for the classification process. A publically available PD dataset was used in the experimental works and classification accuracy, sensitivity, specificity, precision, and F-score metrics were used for performance evaluation. The obtained accuracy, sensitivity, specificity, precision and F-score values were 94.27%, 0.960, 0.960, 0.910 and 0.930, respectively. The obtained results were also compared with some of the published results and it had seen that most of the achievements of the proposed method are better than the compared methods.

A New Population Initialization of Particle Swarm Optimization Method Based on PCA for Feature Selection

In many fields such as signal processing, machine learning, pattern recognition and data mining, it is common practice to process datasets containing huge numbers of features. In such cases, Feature Selection (FS) is often involved. Meanwhile, owing to their excellent global search ability, evolutionary computation techniques have been widely employed to the FS. So, as a powerful global search method and calculation fast than other EC algorithms, PSO can solve features selection problems well. However, when facing a large number of feature selection, the efficiency of PSO drops significantly. Therefore, plenty of works have been done to improve this situation. Besides, many studies have shown that an appropriate population initialization can effectively help to improve this problem. So, basing on PSO, this paper introduces a new feature selection method with filter-based population. The proposed algorithm uses Principal Component Analysis (PCA) to measure the importance of features first, then based on the sorted feature information, a population initialization method using the threshold selection and the mixed initialization is proposed. The experiments were performed on several datasets and compared to several other related algorithms. Experimental results show that the accuracy of PSO to solve feature selection problems is significantly improved after using proposed method.

Machine Learning and Feature Selection Approaches for Categorizing Arabic Text: Analysis, Comparison, and Proposal

This work adopts some classification approaches for categorizing Arabic text. The approaches are operated on two datasets as test-beds. A comparative study is done to evaluate the performance of the adopted classifiers. Some feature selection methods are also analyzed, investigated, and evaluated. Selecting the most significant features is important because the huge number of features may cause performance degradation for text classification. A comparative study is done among the adopted feature selection methods for classifying Arabic documents. Moreover, a modification is done on the feature selection approaches by doing amalgamation for the chosen methods. A novel method is also proposed for selecting the most appropriate features. The method is based on the semantic fusion and multiple-words (SF-MW) for constructing the features. A comparison is done among the adopted feature selection methods and the proposed one. The experimental results show that the best performance was for the SVM classifier compared to the KNN and NB classifiers. The combination among the adopted feature selection methods presents better results compared to the individual adopted ones. The proposed feature selection method (SF-MW) is promising as it reduced the features and achieved higher classification accuracy. The accuracy improvement was about 22% for the two chosen Arabic test-beds which contain 1246 and 1500 documents respectively. The proposed method is expected to be also efficient for other Arabic and English datasets.

Online feature selection system for big data classification based on multi-objective automated negotiation

Feature Selection (FS) plays an important role in learning and classification tasks. Its objective is to select the relevant and non-redundant features. Considering the huge number of features in real-world applications, FS methods using batch learning technique cannot resolve big data problems especially when data arrive sequentially. In this paper, we proposed an online feature selection system which resolves this problem. The proposed OFS system called MOANOFS (Multi-Objective Automated Negotiation based Online Feature Selection) explore the recent advances of online machine learning techniques and a conflict resolution technique (Automated Negotiation) for the purpose of enhancing the classification performance of ultra-high dimensional databases. MOANOFS uses two decision levels. In the first level, we decided which k(s) among the learners (or OFS methods) are the trustful ones (with high confidence or trust value). These elected k learners would participate in the second level where we integrated our proposed Multilateral Automated Negotiation based OFS (MANOFS) method. This would enable us to finally decide which features are the most relevant. We showed that MOANOFS system achieves high accuracy with several real text classification datasets as 20Newsgroups, RCV1.

Binary BAT algorithm and RBFN based hybrid credit scoring model

Credit scoring is a process of calculating the risk associated with an applicant on the basis of applicant’s credentials such as social status, financial status, etc. and it plays a vital role to improve cash flow for financial industry. However, the credit scoring dataset may have a large number of irrelevant or redundant features which leads to poorer classification performances and higher complexity. So, by removing redundant and irrelevant features may overcome the problem with huge number of features. This work emphasized on the role of feature selection and proposed a hybrid model by combining feature selection by utilizing Binary BAT optimization technique with a novel fitness function and aggregated with for Radial Basis Function Neural Network (RBFN) for credit score classification. Further, proposed feature selection approach is aggregated with Support Vector Machine (SVM) & Random Forest (RF), and other optimization approaches namely: Hybrid Particle Swarm Optimization and Gravitational Search Algorithm (PSOGSA), Hybrid Particle Swarm Optimization and Genetic Algorithm (PSOGA), Improved Krill Herd (IKH), Improved Cuckoo Search (ICS), Firefly Algorithm (FF) and Differential Evolution (DE) are also applied for comparative analysis.

Review on Trait Selection of Tumor in the Field of Oncology With the Aid of Data Mining

Feature selection is the important technique which plays a pivotal role in the process of data mining. This is mainly needed to deal with the extreme number of features, normally which can turn into a computational over burden on the process of learning algorithms. Such type of algorithm is also necessary, even when the computational resources are not panic, since it improves the accuracy of the machine learning tasks. In the field of medical science there are various kinds of problem in the area of medical imaging as like extraction, classification, segmentation, selection and etc. On the other side Medical datasets are normally categorized by gigantic extent of disease measurements and comparatively little quantity of patient records. Such feature selection are not appropriate, where all such immaterial and redundancy features are extremely typical to evaluate. On the contrary, the huge number of features may cause the problem of memory storage in order to represent the data set. In such a situation various kinds of data mining algorithms or techniques can be implemented for dealing with such a huge amount of data with precision. This paper basically represents a brief review about the medical image feature selection for the diagnosis of the tumor by the help of data mining techniques.

A Multi-Objective Evolutionary Approach for Preprocessing Imbalanced Microarray Datasets

Microarray datasets are usually imbalanced with a huge number of features and few samples. Most of the existing methods consider classification accuracy as the performance measure while converting the imbalanced datasets into a balanced one and selecting the optimal feature subset, resulting in overfitting. To address the above-mentioned issue, in this paper, a multi-objective genetic algorithm approach is proposed that utilizes a combination of evaluation metrics for both imbalanced data processing and feature selection. The proposed methodology is implemented in seven real datasets taken from public repositories. The cross-validation results prove that the proposed methodology outperforms existing methods in all seven datasets. The proposed methodology serves as a simple and efficient method for preprocessing microarray datasets and helps clinical practitioners in making strategic decisions with the help of a few features. It will also greatly reduce the time spent on processing unnecessary features.

A Brief Conceptual View on Classification Using Support Vector Machine

Classifier is an important element for a classification process to classify features or samples of a dataset successfully. The existence of classifier is important, especially involved multiple class dataset with huge number of features or samples. Support Vector Machine (SVM) classifier catches more attention over other classifiers as it is widely used in various fields. Therefore, this paper aims to give a brief conceptual view of SVM in classification. The important elements and principle of SVM are also described. Along with that an appropriate evaluation and validation for SVM performance are also included. As a result, SVM concept in solving real world classification problem is identified for future reference.

Unsupervised Feature Selection by Pareto Optimization

Dimensionality reduction is often employed to deal with the data with a huge number of features, which can be generally divided into two categories: feature transformation and feature selection. Due to the interpretability, the efficiency during inference and the abundance of unlabeled data, unsupervised feature selection has attracted much attention. In this paper, we consider its natural formulation, column subset selection (CSS), which is to minimize the reconstruction error of a data matrix by selecting a subset of features. We propose an anytime randomized iterative approach POCSS, which minimizes the reconstruction error and the number of selected features simultaneously. Its approximation guarantee is well bounded. Empirical results exhibit the superior performance of POCSS over the state-of-the-art algorithms.

Hybrid Water Cycle Optimization Algorithm With Simulated Annealing for Spam E-mail Detection

Spam is defined as junk and unwanted e-mail. The implementation of a reliable spam email filter becomes more and more important for e-mail users since they have to face with the growing amount of uninvited e-mails. The faults of spam classifiers are characterized by being more and more insufficient to handle huge volumes of relevant emails and to identify and detect the new spam email as example with high performance. The problem in spam classifiers is a huge number of features. Feature selection is an important task in keyword content classification for being among the most popular and effective methods for feature reduction. Accordingly, irrelevant and redundant features that can impede performance would be eliminated. Meta-heuristic optimization is to choose the optimal solution between possible multi-solutions, which respect the aim of this research that is the performance. The other problem is related to ambiguity of the effect of optimization feature selection on multiple classifiers algorithm which are popular used by previous work namely; K-nearest Neighbor, Naive Bayesian and Support Vector Machine. Therefore, the aim of this research is to improve the accuracy of feature selection by applying hybrid Water Cycle and Simulated Annealing to optimize results and to evaluate the proposed Spam Detection. The methodology used in this study which consists of groundwork, induction, improvement, evaluation and comparison quality. The cross-validation was used for training and validation dataset and seven datasets were employed in testing the spam classification proposed. The results demonstrate that the meta-heuristic namely water cycle feature selection (WCFS) was employed and three ways of hybridization with Simulated Annealing as a feature selection employed. In comparison with other feature selection algorithms such as Harmony Search, Genetic Algorithm, and Particle Swarm, the hybridization interleaved hybridization outperformed other feature selection algorithms with accuracy 96.3%, on the other side the effect of using three classifier algorithms, the SVM was better than other of classifier algorithms with f-measurement 96.3%. The number of features using interleaved water cycle and Simulated Annealing the number of features has decreased to more than 50%.

A framework for event classification in tweets based on hybrid semantic enrichment

Twitter has become instrumental as a means of spreading information, opinions or awareness about real-world events. The classification of event-related tweets is a challenging problem since tweets are noisy and sparse pieces of text that lack contextual information. Related work proposes contextual enrichment techniques using external sources (e.g. semantic web, external documents), often considering underlying assumptions about the target events. However, they lack guidelines for determining the textual features to enrich, the external sources to use, the properties to explore, and how to prevent the inclusion of unrelated information. In this paper, we propose a hybrid semantic enrichment framework for the classification of event-related tweets. We contribute to this field by leveraging different contextual enrichment strategies into a unifying framework targeted at a broad range of event types, where each enrichment technique has a role in the improvement of event classification. The framework also encompasses a solution to deal with the huge number of features that result from semantic enrichment, which combines a pruning method to select domain relevant semantic features and general-purpose feature selection techniques. We assessed the contribution of each framework component to event classification improvement using a broad experimental setting. Using seven events of distinct natures, we outperformed a word embeddings baseline in 93.6% of cases, and a textual baseline in 60.3% of cases. In most cases, we improved the recall, with no significant impact on the precision.

An Hour Ahead Electricity Price Forecasting with Least Square Support Vector Machine and Bacterial Foraging Optimization Algorithm

<span lang="EN-US">Predicting electricity price has now become an important task in power system operation and planning. An hour-ahead forecast provides market participants with the pre-dispatch prices for the next hour. It is beneficial for an active bidding strategy where amount of bids can be reviewed or modified before delivery hours. However, only a few studies have been conducted in the field of hour-ahead forecasting. This is due to most power markets apply two-settlement market structure (day-ahead and real time) or standard market design rather than single-settlement system (real time). Therefore, a hybrid multi-optimization of Least Square Support Vector Machine (LSSVM) and Bacterial Foraging Optimization Algorithm (BFOA) was designed in this study to produce accurate electricity price forecasts with optimized LSSVM parameters and input features. So far, no works has been established on multistage feature and parameter optimization using LSSVM-BFOA for hour-ahead price forecast. The model was examined on the Ontario power market. A huge number of features were selected by five stages of optimization to avoid from missing any important features. The developed LSSVM-BFOA shows higher forecast accuracy with lower complexity than most of the existing models.</span>

A Comparative Study on using Principle Component Analysis with different Text Classifiers

Text categorization (TC) is the task of automatically organizing a set of documents into a set of pre-defined categories. Over the last few years, increased attention has been paid to the use of documents in digital form and this makes text categorization becomes a challenging issue. The most significant problem of text categorization is its huge number of features. Most of these features are redundant, noisy and irrelevant that cause over fitting with most of the classifiers. Hence, feature extraction is an important step to improve the overall accuracy and the performance of the text classifiers. In this paper, we will provide an overview of using principle component analysis (PCA) as a feature extraction with various classifiers. It was observed that the performance rate of the classifiers after using PCA to reduce the dimension of data improved. Experiments are conducted on three UCI data sets, Classic03, CNAE-9 and DBWorld e-mails. We compare the classification performance results of using PCA with popular and well-known text classifiers. Results show that using PCA encouragingly enhances classification performance on most of the classifiers.

A Comparative Study of Feature Selection Techniques for Bat Algorithm in Various Applications

Feature selection is a process to select the best feature among huge number of features in dataset, However, the problem in feature selection is to select a subset that give the better performs under some classifier. In producing better classification result, feature selection been applied in many of the classification works as part of preprocessing step; where only a subset of feature been used rather than the whole features from a particular dataset. This procedure not only can reduce the irrelevant features but in some cases able to increase classification performance due to finite sample size. In this study, Chi-Square (CH), Information Gain (IG) and Bat Algorithm (BA) are used to obtain the subset features on fourteen well-known dataset from various applications. To measure the performance of these selected features three benchmark classifier are used; k-Nearest Neighbor (kNN), Naïve Bayes (NB) and Decision Tree (DT). This paper then analyzes the performance of all classifiers with feature selection in term of accuracy, sensitivity, F-Measure and ROC. The objective of these study is to analyse the outperform feature selection techniques among conventional and heuristic techniques in various applications.

Credit Scoring Model based on Weighted Voting and Cluster based Feature Selection

Credit scoring concerns with developing empirical model to support financial decision making process for financial institutions. It makes use of applicants’ historical data and statistical or machine learning techniques to access the risk associated with an applicant. However, the data may have redundant and irrelevant information and features, which degrades the classification accuracy and increases the complexity. So, effective feature selection technique can resolve the problem of credit scoring dataset with huge number of features. In various studies, it is shown that ensemble classifier improves the classification performances as compared to its base classifiers. This study focuses to combine the benefits of feature selection and ensemble framework. For feature selection an approach based on feature clustering have been proposed in this study. Moreover, dataset with selected features is applied on five base classifiers and output obtained by base classifiers are aggregated by weighted voting approach for prediction of final output. For validating the proposed approach, three real world credit scoring datasets are utilized and results compared with some existing feature selection techniques in terms of classification accuracy and F1 -score.