Feature Set Size Research Articles

An Improved Binary Crayfish Optimization Algorithm for Handling Feature Selection Task in Supervised Classification

Feature selection (FS) is a crucial phase in data mining (DM) and machine learning (ML) tasks, aimed at removing uncorrelated and redundant attributes to enhance classification accuracy. This study introduces an improved binary crayfish optimization algorithm (IBCOA) designed to tackle the FS problem. The IBCOA integrates a local search strategy and a periodic mode boundary handling technique, significantly improving its ability to search and exploit the feature space. By doing so, the IBCOA effectively reduces dimensionality, while improving classification accuracy. The algorithm’s performance was evaluated using support vector machine (SVM) and k-nearest neighbor (k-NN) classifiers on eighteen multi-scale benchmark datasets. The findings showed that the IBCOA performed better than nine recent binary optimizers, attaining 100% accuracy and decreasing the feature set size by as much as 0.8. Statistical evidence supports that the proposed IBCOA is highly competitive according to the Wilcoxon rank sum test (alpha = 0.05). This study underscores the IBCOA’s potential for enhancing FS processes, providing a robust solution for high-dimensional data challenges.

Enhancing IoT security: A comparative study of feature reduction techniques for intrusion detection system

Internet of Things (IoT) devices are extensively utilized but are susceptible to cyberattacks, posing significant security challenges. To mitigate these threats, machine learning techniques have been implemented for network intrusion detection in IoT environments. These techniques commonly employ various feature reduction methods, prior to inputting data into models, in order to enhance the efficiency of detection processes to meet real-time requirements. This study provides a comprehensive comparison of feature selection (FS) and feature extraction (FE) techniques for network intrusion detection systems (NIDS) in IoT environments, utilizing the TON-IoT and BoT-IoT datasets for both binary and multi-class classification tasks. We evaluated FS methods, including Pearson correlation and Chi-square, and FE methods, such as Principal Component Analysis (PCA) and Autoencoders (AE), across five classic machine learning models: Decision Tree (DT), Random Forest (RF), Naive Bayes (NB), k-Nearest Neighbors (kNN), and Multi-Layer Perceptron (MLP). Our analysis revealed that FE techniques generally achieve higher accuracy and robustness compared to FS methods, with RF paired with AE delivering superior performance despite higher computational demands. DTs are most effective with smaller feature sets, while MLPs excel with larger sets. Chi-square is identified as the most efficient FS method, balancing performance and computational efficiency, whereas PCA outperforms AE in runtime efficiency. The study also highlights that FE methods are more effective for complex datasets and less sensitive to feature set size, whereas FS methods show significant performance improvements with more informative features. Despite the higher computational costs of FE methods, they demonstrate a greater capability to detect diverse attack types, making them particularly suitable for complex IoT environments. These findings are crucial for both academic research and industry applications, providing insights into optimizing detection performance and computational efficiency in NIDS for IoT networks.

A novel feature selection method based on quantum support vector machine

Feature selection is critical in machine learning to reduce dimensionality and improve model accuracy and efficiency. The exponential growth in feature space dimensionality for modern datasets directly results in ambiguous samples and redundant features, which can severely degrade classification accuracy. Quantum machine learning offers potential advantages for addressing this challenge. In this paper, we propose a novel method, quantum support vector machine feature selection (QSVMF), integrating quantum support vector machines with multi-objective genetic algorithm. QSVMF optimizes multiple simultaneous objectives: maximizing classification accuracy, minimizing selected features and quantum circuit costs, and reducing feature covariance. We apply QSVMF for feature selection on a breast cancer dataset, comparing the performance of QSVMF against classical approaches with the selected features. Experimental results show that QSVMF achieves superior performance. Furthermore, the Pareto front solutions of QSVMF enable analysis of accuracy versus feature set size trade-offs, identifying extremely sparse yet accurate feature subsets. We contextualize the biological relevance of the selected features in terms of known breast cancer biomarkers. This work highlights the potential of quantum-based feature selection to enhance machine learning efficiency and performance on complex real-world data.

Predictive machine learning for gully susceptibility modeling with geo-environmental covariates: main drivers, model performance, and computational efficiency

Currently, machine learning (ML) based gully susceptibility prediction is a rapidly expanding research area. However, when assessing the predictive performance of ML models, previous research frequently overlooked the critical component of computational efficiency in favor of accuracy. This study aimed to evaluate and compare the predictive performance of six commonly used algorithms in gully susceptibility modeling. Artificial neural networks (ANN), partial least squares, regularized discriminant analysis, random forest (RF), stochastic gradient boosting, and support vector machine (SVM) were applied. The comparison was conducted under three scenarios of input feature set sizes: small (six features), medium (twelve features), and large (sixteen features). Results indicated that SVM was the most efficient algorithm with a medium-sized feature set, outperforming other algorithms across all overall accuracy (OA) metrics (OA = 0.898, F1-score = 0.897) and required a relatively short computation time (< 1 min). Conversely, ensemble-based algorithms, mainly RF, required a larger feature set to reach optimal accuracy and were computationally demanding, taking about 15 min to compute. ANN also showed sensitivity to the number of input features, but unlike RF, its accuracy consistently decreased with larger feature sets. Among geo-environmental covariates, NDVI, followed by elevation, TWI, population density, SPI, and LULC, were critical for gully susceptibility modeling. Therefore, using SVM and involving these covariates in gully susceptibility modeling in similar environmental settings is strongly suggested to ensure higher accuracy and minimal computation time.

A comprehensive evaluation framework for benchmarking multi-objective feature selection in omics-based biomarker discovery.

Machine learning algorithms have been extensively used for accurate classification of cancer subtypes driven by gene expression-based biomarkers. However, biomarker models combining multiple gene expression signatures are often not reproducible in external validation datasets and their feature set size is often not optimized, jeopardizing their translatability into cost-effective clinical tools. We investigated how to solve the multi-objective problem of finding the best trade-offs between classification performance and set size applying seven algorithms for machine learning-driven feature subset selection and analyse how they perform in a benchmark with eight large-scale transcriptome datasets of cancer, covering both training and external validation sets. The benchmark includes evaluation metrics assessing the performance of the individual biomarkers and the solution sets, according to their accuracy, diversity, and stability of the composing genes. Moreover, a new evaluation metric for cross-validation studies is proposed that generalizes the hypervolume, which is commonly used to assess the performance of multi-objective optimization algorithms. Biomarkers exhibiting 0.8 of balanced accuracy on the external dataset for breast, kidney and ovarian cancer using respectively 4, 2 and 7 features, were obtained. Genetic algorithms often provided better performance than other considered algorithms, and the recently proposed NSGA2-CH and NSGA2-CHS were the best performing methods in most cases.

Smart Multistage Privacy-Preserving Framework for Intrusion Detection in Multi-Domain SDN

SDN architectures are frequently used by organizations for the management of their networks and the detection of anomalous traffic in a single domain. However, in the real world, anomalous traffic might result in attacks like distributed denial of service (DDoS) that affect numerous domains. During intrusion detection, each SDN domain has to send real traffic data of a large volume to the multi-domain controller, exposing its sensitive information. This paper proposes a smart multistage framework for detecting attacks and ensuring privacy at no additional cost. This work utilized the recent unbalanced InSDN dataset for experimentation. It also uses an oversampling technique that reduces the imbalance rate for each attack type and selects the smallest possible training size and feature set size for an increase in detection accuracy and a reduction in computational complexity. Then, a multi-class classifier method for intrusion detection that does not require regularization or hyperparameter tuning, called ensemble-learning-based shallow decision tree (ELSDT) is proposed. Furthermore, the performance of the proposed classifier on the InSDN dataset is assessed on an SDN testbed. Experimental results show the ability of the proposed smart multistage privacy-preserving framework to make a significant reduction in the training sample size and feature set size to 87% and 76%, respectively. It also shows its outperformance in recent literature works by 5.67% improved accuracy.

MachineTFBS: Motif-based method to predict transcription factor binding sites with first-best models from machine learning library

This study aims to model the high-affinity transcription factor (TF) binding sites (TFBSs) in Saccharomyces cerevisiae promoters using high-throughput experimental TF binding data to train, test, and validate models. We introduce MachineTFBS, a machine learning based TFBS prediction tool consisting of a library of machine learning models generated by treating each TF's protein binding microarray (PBM) dataset as an individual optimization problem and selectively generating the first-best performing models. Since the modeling quality of machine learning methods varies for different TFs, we tested Random Forest, eXtreme Gradient Boosting, and up to 5-depth Deep Learning models. We extracted 274 high-resolution PBM datasets of 159 S. cerevisiae TFs from the UniProbe database. We designed an algorithm for the greedy selection of the first-best models and feature combinations from five feature extraction functions for each PBM data. The optimal subset of the functions was identified using a feature elimination algorithm. The feature elimination algorithm advances in successive cycles for each PBM data until the elimination of any feature(s) improves model performance or the feature set size is reduced to 1; in turn, reports the optimal subset. MachineTFBS predicts TFBSs with an average Matthews Correlation Coefficient score of 0.873.

Enhancing the efficacy of depression detection system using optimal feature selection from EHR

Diagnosing depression at an early stage is crucial and majorly depends on the clinician’s skill. The present work aims to develop an automated tool for assisting the diagnostic procedure of depression using multiple machine-learning techniques. The dataset of sample size 4184 used in this study contains biometric and demographic information of individuals with or without depression, accessed from the University of Nice Sophia-Antipolis. The Artificial Neural Network (ANN), Support Vector Machine (SVM), Random Forest (RF) and Extreme Gradient Boosting (XGBoost) are used for classifying the depressed from the control group. To enhance the computational efficiency, various feature selection algorithms like Recursive Feature Elimination (RFE), Mutual Information (MI) and three bio-inspired techniques, viz. Particle Swarm Optimization (PSO), Genetic Algorithm (GA) and Firefly Algorithms (FA) have been incorporated. To enhance the feature selection process further, majority voting is carried out in all possible combinations of three, four and five feature selection techniques. These feature selection techniques bring down the feature set size significantly to a mean of 33 from the actual size of 61 which is a reduction of 45.90%. The classification accuracy of the enhanced model varies between 84.18% and 88.46%, which is a significant improvement in performance as compared to the pre-existing models (83.76–85.89%). The proposed predictive models outperform the pre-existing classification models without feature selection and thereby enhancing both the performance and efficiency of the diagnostic process.

Adaptive Window Based 3-D Feature Selection for Multispectral Image Classification Using Firefly Algorithm

Feature extraction is the most critical step in classification of multispectral image. The classification accuracy is mainly influenced by the feature sets that are selected to classify the image. In the past, handcrafted feature sets are used which are not adaptive for different image domains. To overcome this, an evolutionary learning method is developed to automatically learn the spatial-spectral features for classification. A modified Firefly Algorithm (FA) which achieves maximum classification accuracy with reduced size of feature set is proposed to gain the interest of feature selection for this purpose. For extracting the most efficient features from the data set, we have used 3-D discrete wavelet transform which decompose the multispectral image in all three dimensions. For selecting spatial and spectral features we have studied three different approaches namely overlapping window (OW-3DFS), non-overlapping window (NW-3DFS) adaptive window cube (AW-3DFS) and Pixel based technique. Fivefold Multiclass Support Vector Machine (MSVM) is used for classification purpose. Experiments conducted on Madurai LISS IV multispectral image exploited that the adaptive window approach is used to increase the classification accuracy.

Evolutionary Computing to solve product inconsistencies in Software Product Lines

• Inconsistencies prevent and delay the configuration of an SPL product. • Evolutionary computation is proposed to resolve inconsistencies in SPL products. • ga-SPLIS (genetic algorithm-Software Product LIne Solution) is developed. • Results show the applicability of GA to resolve the SPL product inconsistencies. In Software Product Lines (SPLs), multiple design teams work collectively to configure products. Often, having multiple sub-designs leads to inconsistencies which prevent and delay the configuration of the final product at the end. Solutions are available to identify and resolve these inconsistencies for limited scale SPLs. For large scale and industrial SPLs, resolution of inconsistencies is still a potential research topic. In this paper, we propose, implement and validate an application of evolutionary computation, particularly genetic algorithms, to resolve inconsistencies in large scale SPL products. We label our approach ga -SPLIS (genetic algorithm-Software Product LIne Solution). Using a concrete experimental methodology with different feature set sizes and multiple standard performance metrics, we generate results which validate the applicability of ga -SPLIS for the SPL community.

A Modified Firefly Deep Ensemble for Microarray Data Classification

Abstract Many researchers are using microarray technology to examine and investigate the levels of gene expression in a specific organism, which is an emerging trend in the field of genetic research. Microarray studies have a wide range of applications in the health sector, including disease prediction and diagnostics, as well as cancer research. Due to the existence of irrelevant or duplicated data in microarray datasets, it is difficult to correctly and immediately capture possible patterns using existing algorithms. Feature selection (FS) has evolved into a critical approach for identifying and eliminating the most pertinent qualities. The enormous dimensionality of microarray datasets, on the other hand, presents a significant barrier to the majority of available FS techniques. In this research, we propose a Modified Firefly Feature Selection (MFFS) algorithm that will reduce the irrelevant attributes needed for classification and a Deep Learning Model for classifying the microarray data. The experimental outcomes show that the proposed MFFS algorithm combined with a Hybrid Deep Learning Algorithm outperforms the existing methods in terms of feature set size, accuracy, precision, recall, F-measure and AUC for a dataset with larger number of features.

Evolving Hybrid Partial Genetic Algorithm Classification Model for Cost-effective Frailty Screening: Investigative Study.

BackgroundA commonly used method for measuring frailty is the accumulation of deficits expressed as a frailty index (FI). FIs can be readily adapted to many databases, as the parameters to use are not prescribed but rather reflect a subset of extracted features (variables). Unfortunately, the structure of many databases does not permit the direct extraction of a suitable subset, requiring additional effort to determine and verify the value of features for each record and thus significantly increasing cost.ObjectiveOur objective is to describe how an artificial intelligence (AI) optimization technique called partial genetic algorithms can be used to refine the subset of features used to calculate an FI and favor features that have the least cost of acquisition.MethodsThis is a secondary analysis of a residential care database compiled from 10 facilities in Queensland, Australia. The database is comprised of routinely collected administrative data and unstructured patient notes for 592 residents aged 75 years and over. The primary study derived an electronic frailty index (eFI) calculated from 36 suitable features. We then structurally modified a genetic algorithm to find an optimal predictor of the calculated eFI (0.21 threshold) from 2 sets of features. Partial genetic algorithms were used to optimize 4 underlying classification models: logistic regression, decision trees, random forest, and support vector machines.ResultsAmong the underlying models, logistic regression was found to produce the best models in almost all scenarios and feature set sizes. The best models were built using all the low-cost features and as few as 10 high-cost features, and they performed well enough (sensitivity 89%, specificity 87%) to be considered candidates for a low-cost frailty screening test.ConclusionsIn this study, a systematic approach for selecting an optimal set of features with a low cost of acquisition and performance comparable to the eFI for detecting frailty was demonstrated on an aged care database. Partial genetic algorithms have proven useful in offering a trade-off between cost and accuracy to systematically identify frailty.

Hybridized Dimensionality Reduction Method for Machine Learning based Web Pages Classification

Feature space high dimensionality is a well-known problem in text classification and web mining domains, it is caused mainly by the large number of vocabularies contained within web documents. Several methods were applied to select the most useful and important features over the years; however, the performance of such methods is still improvable from different aspects such as the computational cost and accuracy. This research presents an enhanced cosine similarity-based hybridization of two efficient feature selection methods for higher classification performance. The reduced feature sets are generated using the Random Projection (RP) and the Principal Component Analysis (PCA) methods, individually, then hybridized based on the cosine similarity values between features’ vectors. The performance of the proposed method in terms of accuracy and F-measure was tested on a dataset of web pages based on several term weighting schemes. As compared to relevant methods, results of the proposed method show significantly higher accuracy and f-measure performance based on less feature set size. Index Terms— Cosine similarity, Dimensionality Reduction, Feature selection, PCA, Random Projection.

A dynamic feature selection and intelligent model serving for hybrid batch-stream processing

The increasing complexity of real-world applications, especially those related to the Internet of Things and cloud computing, highlights the importance of using hybrid batch-stream processing techniques to analyze big data. Hybrid processing combines the accuracy of batch processing and the speed of stream processing. Among the important challenges of this approach are selecting relevant and diverse features to build the base models and intelligently choosing which of those models to use in computing the final results. We present the H-DIFS architecture, a dynamic and intelligent feature selection approach to address these two challenges of hybrid batch-stream processing for anomaly detection. The proposed architecture employs a dynamic feature selection method based on the genetic algorithm that is fully compatible with the nature of hybrid processing and dynamically changes the models’ features over time based on the input data stream. Additionally, two components, the model evaluator and intelligent model recommender, allow for offering various batch and stream models and selecting from them based on the defined policies. The experimental results, on the datasets with various feature set sizes, indicate that the proposed architecture increases the speed of hybrid processing and anomaly detection by eliminating irrelevant and redundant features. Moreover, it increases detection accuracy by selecting appropriate models and aggregating their results.

Hybrid feature selection based on SLI and genetic algorithm for microarray datasets.

One of the major problems in microarray datasets is the large number of features, which causes the issue of “the curse of dimensionality” when machine learning is applied to these datasets. Feature selection refers to the process of finding optimal feature set by removing irrelevant and redundant features. It has a significant role in pattern recognition, classification, and machine learning. In this study, a new and efficient hybrid feature selection method, called Garank&rand, is presented. The method combines a wrapper feature selection algorithm based on the genetic algorithm (GA) with a proposed filter feature selection method, SLI-γ. In Garank&rand, some initial solutions are built regarding the most relevant features based on SLI-γ, and the remaining ones are only the random features. Eleven high-dimensional and standard datasets were used for the accuracy evaluation of the proposed SLI-γ. Additionally, four high-dimensional well-known datasets of microarray experiments were used to carry out an extensive experimental study for the performance evaluation of Garank&rand. This experimental analysis showed the robustness of the method as well as its ability to obtain highly accurate solutions at the earlier stages of the GA evolutionary process. Finally, the performance of Garank&rand was also compared to the results of GA to highlight its competitiveness and its ability to successfully reduce the original feature set size and execution time.

BLINC: lightweight bimodal learning for low-complexity VVC intra-coding

The latest video coding standard, versatile video coding (VVC), achieves almost twice coding efficiency compared to its predecessor, the high efficiency video coding (HEVC). However, achieving this efficiency (for intra coding) requires 31 × computational complexity compared to HEVC, which makes it challenging for low power and real-time applications. This paper, proposes a novel machine learning approach that jointly and separately employs two modalities of features, to simplify the intra coding decision. To do so, first a set of features are extracted that use the existing DCT core of VVC, to assess the texture characteristics, and forms the first modality of data. This produces high-quality features with almost no extra computational overhead. The distribution of intra modes at the neighboring blocks is also used to form the second modality of data, which provides statistical information about the frame, unlike the first modality. Second, a two-step feature reduction method is designed that reduces the size of feature set, such that a lightweight model with a limited number of parameters can be used to learn the intra mode decision task. Third, three separate training strategies are proposed (1) an offline training strategy using the first (single) modality of data, (2) an online training strategy that uses the second (single) modality, and (3) a mixed online–offline strategy that uses bimodal learning. Finally, a low-complexity encoding algorithms is proposed based on the proposed learning strategies. Extensive experimental results show that the proposed methods can reduce up to 24% of encoding time, with a negligible loss of coding efficiency. Moreover, it is demonstrated how a bimodal learning strategy can boost the performance of learning. Lastly, the proposed method has a very low computational overhead (0.2%), and uses existing components of a VVC encoder, which makes it much more practical compared to competing solutions.

Towards Applicability of Artificial Intelligence in Healthcare, Banking and Education Sector

Machine learning is the result of a combination of computational and artificial intelligence. They use the human mind as a model to construct intelligent computers capable of solving real-world problems. It encompasses neuro computing, fuzzy logic, and genetic computing for the creation and deployment of intelligent instruments. Genetic algorithms, belief networks, and learning theory are also included in the probabilistic reasoning. Medical and genetic data classification is a fundamental problem in biomedical informatics. Small medical data categorization may be used to forecast a patient's illness state. Feature extraction or feature selection techniques may be used to minimize the size of feature sets, and then the smaller sets can be classified using effective classifiers. Using educational data mining, it is possible to predict student performance and identify pupils at risk. Using educational data mining, students' learning habits can be analyzed, growing pass rates can be boosted, and topic curriculum renewal may be maximized.. It is feasible to employ machine learning in banking for a wide range of purposes including fraud detection and improving client happiness. In this papar, we'll take a look at how machine learning is being used in the healthcare, education, and banking industries.

A HYBRID SENTIMENT ANALYSIS APPROACH USING BLACK WIDOW OPTIMIZATION BASED FEATURE SELECTION

This paper proposes a novel hybrid framework with BWO based feature reduction technique which combines the merits of both machine learning and lexicon-based approaches to attain better scalability and accuracy. The scalability problem arises due to noisy, irrelevant and unique features present in the extracted features from proposed approach, which can be eliminated by adopting an effective feature reduction technique. In our proposed BWO approach, without changing the accuracy (90%), the feature-set size is reduced up to 43%. The proposed feature selection technique outperforms other commonly used PSO and GAbased feature selection techniques with reduced computation time of 21 sec. Moreover, our sentiment analysis approach is analysed using performance metrices such as precision, recall, F-measure, and computation time. Many organizations can use these online reviews to make well-informed decisions towards the users’ interests and preferences to enhance customer satisfaction, product quality and to find the aspects to improve the products, thereby to generate more profits.

A hybrid sentiment analysis approach using black widow optimization based feature selection

With the massive development of Internet technologies and e-commerce technology, people rely on the product reviews provided by users through web. Sentiment analysis of online reviews has become a mainstream way for businesses on e-commerce platforms to satisfy the customers. This paper proposes a novel hybrid framework with Black Widow Optimization (BWO) based feature reduction technique which combines the merits of both machine learning and lexicon-based approaches to attain better scalability and accuracy. The scalability problem arises due to noisy, irrelevant and unique features present in the extracted features from proposed approach, which can be eliminated by adopting an effective feature reduction technique. In our proposed BWO approach, without changing the accuracy (90%), the feature-set size is reduced up to 43%. The proposed feature selection technique outperforms other commonly used Particle Swarm Optimization (PSO) and Genetic Algorithm (GA) based feature selection techniques with reduced computation time of 21 sec. Moreover, our sentiment analysis approach is analyzed using performance metrics such as precision, recall, F-measure, and computation time. Many organizations can use these online reviews to make well-informed decisions towards the users’ interests and preferences to enhance customer satisfaction, product quality and to find the aspects to improve the products, thereby to generate more profits.

An ensemble filter-based heuristic approach for cancerous gene expression classification

Gene expression data of cancer has a huge feature set size, making its categorization a challenge for the existing classification methods. It contains redundancy, noise, and irrelevant genes. Therefore, feature selection/reduction plays a crucial role in the classification of such gene expression datasets. This work presents an ensemble of three filter methods, namely, Symmetrical Uncertainty (SU), chi square (X2), and Relief to reduce the feature dimensions by eliminating redundant and noisy genes. The present work designs a novel heuristic called Local Search-based Feature Selection (LSFS) that further reduces noise generated by the ensemble method. The resulting selected features are then optimized using a genetic algorithm. Afterwards, the optimal set of features is classified using three models; Support Vector Machine (SVM), k-NN (k-nearest neighbor), and Random Forest (RF) to find cancer relevant genes. Experiments are conducted using six benchmark datasets. The obtained results are compared with five state-of-the-art algorithms based on accuracy, sensitivity, specificity, F-measure, entropy, and precision. Additional experiments are carried out by manipulating the SVM kernel as a fitness value as well as using multiple distance measures and various values of k for k-NN. Prediction accuracy of the proposed system on the six benchmark datasets is 99%, 90%, 98%, 94%, 98%, and 99%. Significant outcomes obtained from experimental analysis indicate that the proposed approach improves classification of cancerous gene expression data and can be used as a practical tool for the analysis of gene expression data.