Efficient Feature Selection Technique Research Articles

Hyperspectral image classification with spectral-spatial feature integration and ensemble learning

<p>Hyperspectral imaging (HSI) has emerged as a robust remote sensing and medical imaging tool. However, HSI classification remains a challenging problem due to the high-dimensional data and the need for efficient feature selection and enhancement techniques. The proposed work addresses the problem of spatial feature extraction in spectral-spatial HSI classification tasks. This paper introduces an innovative model addressing the intricacies of spatial feature extraction in spectral-spatial HSI classification tasks, employing a fusion of spectral and spatial features through an adaptive kernel-based Gaussian filtering mechanism to elevate the quality of HSI data and augment classification performance. The classification is executed using three distinct classifiers, whose decisions are harmoniously integrated within an ensemble learning framework to optimize outcomes. The effectiveness of the proposed system is meticulously evaluated across three diverse datasets, Indian Pine, Pavia, and Salinas. This study also compares the model's efficiency against the existing similar work presented in the literature. The results show that the proposed work outperforms existing methods with constantly showing 99% accuracy and kappa score for each dataset, demonstrating its potential applications in various domains such as remote sensing and medical imaging.</p>

Coherent Feature Extraction with Swarm Intelligence Based Hybrid Adaboost Weighted ELM Classification for Snoring Sound Classification.

For patients suffering from obstructive sleep apnea and sleep-related breathing disorders, snoring is quite common, and it greatly interferes with the quality of life for them and for the people surrounding them. For diagnosing obstructive sleep apnea, snoring is used as a screening parameter, so the exact detection and classification of snoring sounds are quite important. Therefore, automated and very high precision snoring analysis and classification algorithms are required. In this work, initially the features are extracted from six different domains, such as time domain, frequency domain, Discrete Wavelet Transform (DWT) domain, sparse domain, eigen value domain, and cepstral domain. The extracted features are then selected using three efficient feature selection techniques, such as Golden Eagle Optimization (GEO), Salp Swarm Algorithm (SSA), and Refined SSA. The selected features are finally classified with the help of eight traditional machine learning classifiers and two proposed classifiers, such as the Firefly Algorithm-Weighted Extreme Learning Machine hybrid with Adaboost model (FA-WELM-Adaboost) and the Capuchin Search Algorithm-Weighted Extreme Learning Machine hybrid with Adaboost model (CSA-WELM-Adaboost). The analysis is performed on the MPSSC Interspeech dataset, and the best results are obtained when the DWT features with the refined SSA feature selection technique and FA-WELM-Adaboost hybrid classifier are utilized, reporting an Unweighted Average Recall (UAR) of 74.23%. The second-best results are obtained when DWT features are selected with the GEO feature selection technique and a CSA-WELM-Adaboost hybrid classifier is utilized, reporting an UAR of 73.86%.

Email spam detection by deep learning models using novel feature selection technique and BERT

Due to the influx of advancements in technology and the increased simplicity of communication through emails, there has been a severe threat to the global economy and security due to upsurge in volume of unsolicited During the training of models, high-dimensional and redundant datasets may reduce the classification results of the model due to high memory costs and high computation. An important data processing technique is feature selection which helps in selecting relevant features and subsets of information from the dataset. Therefore, choosing efficient feature selection techniques is very important for the best performance of classification of a model. Moreover, most of the research has been performed using traditional machine learning techniques, which are not enough to deal with the huge amount of data and its variations. Also,spammers are becoming smarter with technological advancement. Therefore, there is a need for hybrid techniques consisting of deep learning and conventional algorithms to cope with these problems. We have proposed a novel scheme in this paper for email spam detection, which will result in an improved feature selection approach from the original dataset and increase the accuracy of the classifier as well. The literature has been studied to explore the efficient machine learning models that have been applied by different researchers for email spam detection and feature selection to acquire the best results. Our method, GWO-BERT, has given remarkable results with deep learning techniques such as CNN, biLSTM and LSTM. We have compared our models with RF and LSTM and used dataset: “Lingspam,” which is a publicly available dataset. With different experiments, our technique, GWO-BERT, obtained 99.14% accuracy, which is almost equal to 100 percent.

An Efficient Hybrid Feature Selection Technique Toward Prediction of Suspicious URLs in IoT Environment

An Efficient Hybrid Feature Selection Technique Toward Prediction of Suspicious URLs in IoT Environment

Intrusion detection approach for cloud and IoT environments using deep learning and Capuchin Search Algorithm

The Internet of Things (IoT) enabled technology will be adopted to develop smart cities, electronic commerce, electronic learning, electronic health, and other aspects of online activities. IoT enabled pervasive and wide connectivity to many objects and services. Therefore, it is easy to target IoT and cloud malware infection. Thus, cybersecurity is an essential problem to build robust IoT systems. This paper leverages the recent developments of the swarm intelligence (SI) algorithms combined with the advances of deep neural networks to build an efficient intrusion detection system for IoT-cloud based environments. First, deep neural networks are used to obtain optimal features from the IoT IDS data. Then, an efficient feature selection technique is proposed based on a recently developed SI optimizer called Capuchin Search Algorithm (CapSA). The performance of the developed model, called CNN-CapSA, is tested with four IoT-Cloud datasets, namely, NSL-KDD, BoT-IoT, KDD99, and CIC2017. Moreover, we consider extensive empirical comparisons to other optimization algorithms using several classification performance measures. The outcomes verified that the developed approach has a competitive performance overall datasets.

Stock index trend prediction based on TabNet feature selection and long short-term memory.

In this study, we propose a predictive model TabLSTM that combines machine learning methods such as TabNet and Long Short-Term Memory Neural Network (LSTM) with a complete factor library for stock index trend prediction. Our motivation is based on the notion that there are numerous interrelated factors in the stock market, and the factors that affect each stock are different. Therefore, a complete factor library and an efficient feature selection technique are necessary to predict stock index. In this paper, we first build a factor database that includes macro, micro and technical indicators. Successively, we calculate the factor importance through TabNet and rank them. Based on a prespecified threshold, the optimal factors set will include only the highest-ranked factors. Finally, using the optimal factors set as input information, LSTM is employed to predict the future trend of 4 stock indices. Empirical validation of the model shows that the combination of TabNet for factors selection and LSTM outperforms existing methods. Moreover, constructing a factor database is necessary for stock index prediction. The application of our method does not only show the feasibility to predict stock indices across different financial markets, yet it also provides an complete factor database and a comprehensive architecture for stock index trend prediction, which may provide some references for stock forecasting and quantitative investments.

An improved feature selection approach using global best guided Gaussian artificial bee colony for EMG classification

Electromyography (EMG) measures muscle relaxes or contractions during muscular activity through EMG signal. It plays a vital role in identifying muscle-related problems for clinical diagnosis. This paper presents an efficient EMG feature selection technique for classifying 17 different prosthetic hand movements recorded from 11 subjects. Two variants of the Artificial Bee Colony (ABC) algorithm, namely: i) Global Best Guided ABC (GbestABC) and ii) Gaussian ABC (GABC), are employed to propose an Improved Artificial Bee Colony (ABC) algorithm called Global Best Guided Gaussian ABC (GGABC) for solving global optimization problems. GbestABC performs better in the exploitation phase, whereas GABC performs better in the exploration phase in searching for the optimal solutions. The proposed GGABC takes advantage of GbestABC and GABC to counterbalance basic ABC's exploitation and exploration capability. Further, a binary version of GGABC known as binary GGABC (BGGABC) is developed to solve binary optimization problems and select optimal EMG signal classification features. Extensive experiments are carried out in three phases: i) GGABC for global optimization problems ii) BGGABC for EMG feature selection problems with other meta-heuristic-based competitors iii) BGGABC for EMG feature selection problems with well-known filter based techniques. K-nearest neighbor (KNN) classifier is used in experiments to validate and investigate the effectiveness of the proposed algorithm. Experimental result shows that the BGGABC-based EMG feature selection achieved 94.13% average classification accuracy and 97.06% best classification accuracy. Obtained results confirm that the proposed algorithm outperforms or is competitive with state-of-the-art algorithms in EMG feature selection and classification.

Efficient feature selection technique for brain tumor classification utilizing hybrid fruit fly based abc and ann algorithm

In medical science, information technology plays a main role in image processing for medicinal investigation and classification. For this reason “Magnetic Resonance Imaging” (MRI) can be useful today in the medical world which in turn is used to generate a comprehensive picture of the body's soft tissue and bones. The suggested technique utilizes an effective technique for brain tumor segmentation and classification. To segment and classify the brain tumor MR image by Modified fuzzy c means clustering (MFCM) and Artificial Neural Network (ANN). From the input image the proposed method extracts the shape, intensity and texture features. Here the extracted features are optimally selected by means of Hybrid Fruit fly and artificial bee colony (HFFABC). Both the segmentation and classification performance of the implemented technique is estimated in terms of “Accuracy, Sensitivity, Specificity, Ppv, Npv, Fpr And Fnr”. The implementation result shows the efficiency of proposed tumor segmentation and classification technique in segmenting the tissues perfectly from the MRI images of human nervous system. The classification performance of the implemented techniques is evaluated in terms of accuracy, sensitivity and specificity. The implementation results show the sensitivity, specificity & efficiency as 98.1%, 99.8% and 99.59%respectively.

Hellinger distance-based stable sparse feature selection for high-dimensional class-imbalanced data

BackgroundFeature selection in class-imbalance learning has gained increasing attention in recent years due to the massive growth of high-dimensional class-imbalanced data across many scientific fields. In addition to reducing model complexity and discovering key biomarkers, feature selection is also an effective method of combating overlapping which may arise in such data and become a crucial aspect for determining classification performance. However, ordinary feature selection techniques for classification can not be simply used for addressing class-imbalanced data without any adjustment. Thus, more efficient feature selection technique must be developed for complicated class-imbalanced data, especially in the context of high-dimensionality.ResultsWe proposed an algorithm called sssHD to achieve stable sparse feature selection applied it to complicated class-imbalanced data. sssHD is based on the Hellinger distance (HD) coupled with sparse regularization techniques. We stated that Hellinger distance is not only class-insensitive but also translation-invariant. Simulation result indicates that HD-based selection algorithm is effective in recognizing key features and control false discoveries for class-imbalance learning. Five gene expression datasets are also employed to test the performance of the sssHD algorithm, and a comparison with several existing selection procedures is performed. The result shows that sssHD is highly competitive in terms of five assessment metrics. In addition, sssHD presents limited differences between performing and not performing re-balance preprocessing.ConclusionssssHD is a practical feature selection method for high-dimensional class-imbalanced data, which is simple and can be an alternative for performing feature selection in class-imbalanced data. sssHD can be easily extended by connecting it with different re-balance preprocessing, different sparse regularization structures as well as different classifiers. As such, the algorithm is extremely general and has a wide range of applicability.

A Novel Hybrid Classification Method Based on the Opposition-Based Seagull Optimization Algorithm

In practice, classification problems have appeared in many scientific fields, including finance, medicine and industry. It is critically important to develop an effective and accurate classification model. Although numerous useful classifiers have been proposed, they are unstable, sensitive to noise and slow in computation. To overcome these drawbacks, the combination of feature selection techniques with traditional machine learning models is of great help. In this paper, a novel feature selection method called the opposition-based seagull optimization algorithm (OSOA) is proposed and studied. The OSOA is constructed based on an SOA whose population is determined by the opposition-based learning (OBL) algorithm. To evaluate its overall classification performance, some measures, including classification accuracy, number of selected features, receiver operating characteristic curve (ROC), and computation time, are adopted. The empirical results indicate that the suggested method exhibits higher or similar accuracy and computational efficiency in comparison with genetic algorithm (GA)-, simulated annealing (SA)-, and Fisher score (FS)-based classification models. The experimental results show that the OSOA is a computationally efficient feature selection technique that has the ability to select relevant variables. Furthermore, it performs well with high-dimensional data whose number of variables exceeds the number of samples. Thus, the OSOA is an effective approach for the enhancement of classification performance.

Efficient feature selection techniques for sentiment analysis

Sentiment analysis is a domain of study that focuses on identifying and classifying the ideas expressed in the form of text into positive, negative and neutral polarities. Feature selection is a crucial process in machine learning. In this paper, we aim to study the performance of different feature selection techniques for sentiment analysis. Term Frequency Inverse Document Frequency (TF-IDF) is used as the feature extraction technique for creating feature vocabulary. Various Feature Selection (FS) techniques are experimented to select the best set of features from feature vocabulary. The selected features are trained using different machine learning classifiers Logistic Regression (LR), Support Vector Machines (SVM), Decision Tree (DT) and Naive Bayes (NB). Ensemble techniques Bagging and Random Subspace are applied on classifiers to enhance the performance on sentiment analysis. We show that, when the best FS techniques are trained using ensemble methods achieve remarkable results on sentiment analysis. We also compare the performance of FS methods trained using Bagging, Random Subspace with varied neural network architectures. We show that FS techniques trained using ensemble classifiers outperform neural networks requiring significantly less training time and parameters thereby eliminating the need for extensive hyper-parameter tuning.

VARMAN: Multi-plane security framework for software defined networks

In the context of future networking technologies, Software-Defined paradigm offers compelling solutions and advantages for traffic orchestration and shaping, flexible and dynamic routing, programmable control and smart application-driven resource management. But the SDN operation has to confront critical issues and technical vulnerabilities, security problems and threats in the enabling technical architecture itself. To address the critical security problems in SDN enabled data centers, we propose a collaborative “Network Security and Intrusion Detection System(NIDS)” scheme called ‘VARMAN: adVanced multi-plAne secuRity fraMework for softwAre defined Networks’. The SDN security scheme comprises of coarse-grained flow monitoring algorithms on the dataplane for rapid anomaly detection and prediction of network-centric DDoS/botnet attacks. In addition, this is combined with a fine-grained hybrid deep-learning based classifier pipeline on the control plane. It is observed that existing ML-based classifiers improve the accuracy of NIDS, however, at the cost of higher processing power and memory requirement, thus unrealistic for real-time solutions. To address these problems and still achieve accuracy and speed, we designed a hybrid model, combining both deep and shallow learning techniques, that are implemented in an improved SDN stack. The data plane deploys attack prediction and behavioral trigger mechanisms, efficient data filtering, feature selection, and data reduction techniques. To demonstrate the practical feasibility of our security scheme in real modern datacenters, we utilized the popular NSL-KDD dataset, most recent CICIDS2017 dataset, and refined it to a balanced dataset containing a comparable number of normal traffic and malware samples. We further augmented the training by organically generating datasets from lab-simulated and public-network hosted hackathon websites. The results show that VARMAN framework is capable of detecting attacks in real-time with accuracy more than 98% under attack intensities up to 50k packets/second. In a multi-controller interconnected SDN domain, the flow setup time improves by 70% on an average, and controller response time reduces by 40%, without incurring additional latency due to security intelligence processing overhead in SDN stack. The comparisons of VARMAN under similar attack scenarios and test environment, with related recent works that utilized ML-based NIDS, demonstrate that our scheme offers higher accuracy, less than 5% false positive rate for various attack intensities and significant training space/time reduction.

Classification of Gene Expression Data using Efficient Feature Selection Technique and Resampling Method

Microarray technology has been developed as one of the powerful tools that have attracted many researchers to analyze gene expression level for a given organism. It has been observed that gene expression data have very large (in terms of thousands) of features and less number of samples (in terms of hundreds). This characteristic makes difficult to do an analysis of gene expression data. Hence efficient feature selection technique must be applied before we go for any kind of analysis. Feature selection plays a vital role in the classification of gene expression data. There are several feature selection techniques have been induced in this field. But Support Vector Machine with Recursive Feature Elimination (SVM-RFE) has been proven as the promising feature selection methods among others. SVM-RFE ranks the genes (features) by training the SVM classification model and with the combination of RFE method key genes are selected. Huge time consumption is the main issue of SVM-RFE. We introduced an efficient implementation of linier SVM to overcome this problem and improved the RFE with variable step size. Then, combined method was used for selecting informative genes. Effective resampling method is proposed to preprocess the datasets. This is used to make the distribution of samples balanced, which gives more reliable classification results. In this paper, we have also studied the applicability of common classifiers. Detailed experiments are conducted on four commonly used microarray gene expression datasets. The results show that the proposed method comparable classification performance.

A Stable SVM-RFE Feature Selection Method for Gene Expression Data

Feature Selection techniques are generally employed to remove the inessential attributes before machine learning technique could be applied. It thus plays an extremely important role by eliminating the unnecessary features that do not contribute and sometimes degrade the performance and prediction accuracy of the machine learning technique. With the growth of dimensionality of data, Feature Selection becomes even more important because it helps to reduce the dimensions of data and hence decreases the requisite memory and computational complexity of the machine learning techniques. Support vector machine-recursive feature elimination (SVM-RFE) has proven to be an efficient wrapper feature selection technique which continues to be widely utilized in many applications, especially in classification of gene expression data. From the perspective of this data, not only the precision in classification but also the stability of Feature Selection method plays an important role. Nonetheless, the topic of stability is ignored in study of feature selection algorithms. To improve the stability of RFE method, a fusion of Information Gain and RFE (IG-RFE-SVM) method is proposed in this paper. Experimental studies show that IG-RFE-SVM outperforms SVM-RFE method in terms of stability

A clustering‐based feature selection framework for handwritten Indic script classification

AbstractIn India, which has numerous officially recognized scripts, there is a primary need for categorizing the documents on the basis of the scripts used therein. Identification of script used in a document is essential for its effective handling both manually and digitally. Identification of script in a document image is an important research problem in the pattern recognition field, which, at times, suffers from the issue of growing dimensionality of the feature vector and requires an efficient feature selection technique. Keeping this fact in mind, in this paper, we propose a clustering‐based filter feature selection framework in order to extract an optimal and effective feature subset from the original feature vector. The present feature selection methodology is evaluated on a script classification problem involving handwritten documents in 12 major Indic scripts. Experiments are done at word‐level, text‐line‐level, and block‐level. Experiments demonstrate that a reasonable increment in classification accuracy has been realized using comparatively lesser number of features. The proposed framework for feature selection is computationally inexpensive and can be applied to other pattern recognition problems as well.

An evolutionary gravitational search-based feature selection

With recent advancements in data collection tools and the widespread use of intelligent information systems, a huge amount of data streams with lots of redundant, irrelevant, and noisy features are collected and a large number of features (attributes) should be processed. Therefore, there is a growing demand for developing efficient Feature Selection (FS) techniques. Gravitational Search algorithm (GSA) is a successful population-based metaheuristic inspired by Newton’s law of gravity. In this research, a novel GSA-based algorithm with evolutionary crossover and mutation operators is proposed to deal with feature selection (FS) tasks. As an NP-hard problem, FS finds an optimal subset of features from a given set. For the proposed wrapper FS method, both K-Nearest Neighbors (KNN) and Decision Tree (DT) classifiers are used as evaluators. Eighteen well-known UCI datasets are utilized to assess the performance of the proposed approaches. In order to verify the efficiency of proposed algorithms, the results are compared with some popular nature-inspired algorithms (i.e. Genetic Algorithm (GA), Particle Swarm Optimizer (PSO), and Grey Wolf Optimizer (GWO)). The extensive results and comparisons demonstrate the superiority of the proposed algorithm in solving FS problems.

A new steganalysis approach with an efficient feature selection and classification algorithms for identifying the stego images

The detection of stego images by using the steganalysis approach is one of the demanding task in the recent days. Because, it is used as an importer for the immoral activities by hiding the secrets in the messages. For this secret identification, the traditional works develop various steganographic techniques for steganalysis. But, it has some important drawbacks such as low detection percentage, inefficient results, and increased complexity. In order to solve these issues, this paper introduces a new steganalysis approach with an efficient feature selection and optimization techniques. The aim of this paper is to accurately detect the stego and clean images by implementing an efficient classification algorithm. Initially, a novel Coefficient based Walsh Hadamard Transform along with the Gray Level Co-occurrence Matrix (GLCM) is used for extracting the features of the image. Then, an efficient feature selection technique, namely, Pine Growth Optimization (PGO) is developed to select the optimal features from the extracted features. Finally, the Cross Integrated Machine Learning (CIML) classifier is implemented to classify the stego and clean images. The newness is provided during the feature extraction, selection and classification processes. In experiments, the performance results of the proposed steganalysis is evaluated and compared with the existing approaches by using different measures.

PyFeat: a Python-based effective feature generation tool for DNA, RNA and protein sequences.

Extracting useful feature set which contains significant discriminatory information is a critical step in effectively presenting sequence data to predict structural, functional, interaction and expression of proteins, DNAs and RNAs. Also, being able to filter features with significant information and avoid sparsity in the extracted features require the employment of efficient feature selection techniques. Here we present PyFeat as a practical and easy to use toolkit implemented in Python for extracting various features from proteins, DNAs and RNAs. To build PyFeat we mainly focused on extracting features that capture information about the interaction of neighboring residues to be able to provide more local information. We then employ AdaBoost technique to select features with maximum discriminatory information. In this way, we can significantly reduce the number of extracted features and enable PyFeat to represent the combination of effective features from large neighboring residues. As a result, PyFeat is able to extract features from 13 different techniques and represent context free combination of effective features. The source code for PyFeat standalone toolkit and employed benchmarks with a comprehensive user manual explaining its system and workflow in a step by step manner are publicly available. https://github.com/mrzResearchArena/PyFeat/blob/master/RESULTS.md. Toolkit, source code and manual to use PyFeat: https://github.com/mrzResearchArena/PyFeat/. Supplementary data are available at Bioinformatics online.

A hybrid approach using rough set theory and hypergraph for feature selection on high-dimensional medical datasets

‘Curse of Dimensionality’—massive generation of high-dimensional medical datasets from various biomedical applications hardens the data analytic process for precise medical diagnosis. The design of an efficient feature selection technique for finding the optimal feature subset can be devised as a prominent solution to the above-said challenge. Further, it also improves the accuracy and minimizes the computational complexity of the learning model. The state-of-the-art feature selection techniques based on heuristic and statistical functions suffer from significant challenges in terms of classification accuracy, time complexity, etc. Hence, this paper presents Rough Set Theory and Hypergraph (RSHGT)-based feature selection technique to identify the optimal feature subset for accurate medical diagnosis. Experimental validations using six medical datasets from the Kent Ridge Biomedical dataset repository prove the efficiency of RSHGT in terms of reduct size, accuracy, precision, recall, and time complexity.

Emotional Stress State Detection Using Genetic Algorithm-Based Feature Selection on EEG Signals.

In recent years, stress analysis by using electro-encephalography (EEG) signals incorporating machine learning techniques has emerged as an important area of research. EEG signals are one of the most important means of indirectly measuring the state of the brain. The existing stress algorithms lack efficient feature selection techniques to improve the performance of a subsequent classifier. In this paper, genetic algorithm (GA)-based feature selection and k-nearest neighbor (k-NN) classifier are used to identify stress in human beings by analyzing electro-encephalography (EEG) signals. GA is incorporated in the stress analysis pipeline to effectively select subset of features that are suitable to enhance the performance of the k-NN classifier. The performance of the proposed method is evaluated using the Database for Emotion Analysis using Physiological Signals (DEAP), which is a public EEG dataset. A feature set is extracted in 32 EEG channels, which consists of statistical features, Hjorth parameters, band power, and frontal alpha asymmetry. The selected features through GA are used as input to the k-NN classifier to distinguish whether each EEG datapoint represents a stress state. To further consolidate, the effectiveness of the proposed method is compared with that of a state-of-the-art principle component analysis (PCA) method. Experimental results show that the proposed GA-based method outperforms PCA, with GA demonstrating 71.76% classification accuracy compared with 65.3% for PCA. Thus, it can be concluded that the proposed method can be effectively used for stress analysis with high classification accuracy.