Feature Selection Mechanism Research Articles

Weighted mean of vectors algorithm with neighborhood information interaction and vertical and horizontal crossover mechanism for feature selection

Weighted mean of vectors algorithm with neighborhood information interaction and vertical and horizontal crossover mechanism for feature selection

Habitat selection ecology of the aquatic beetle community using explainable machine learning

The aim of our work is to determine the importance of habitat features for the selection of the aquatic beetle community. Insects are represented by their general ecological traits such as body size, ecological element and trophic type, which are categorised into four body size ranges, four ecological groups and four trophic types. To determine the importance of habitat selection of the studied insects, we analysed the relationships between the above categories and the set of habitat features of the lake and its surroundings. Ensemble machine learning modelling (XGBoost-SHAP) revealed the mechanism of habitat feature selection in relation to the general ecological traits. We found strong interactions between the body size, ecological element and trophic type of beetles, suggesting that these general traits control the structure and functioning of the beetle community studied. The area of the lake and the features of beetle occurrence in the aquatic environment play an important but secondary role, and the importance of the characteristics of the lake’s riparian zone was minimised. We found several categories of beetles as they select the number of the same habitat features. The study can provide valuable information for the practical conservation and management of lake ecosystems.

Adaptive feature fusion and disturbance correction for accurate remaining useful life prediction of rolling bearings

As a key component in the transmission system of high-speed trains, bearings need to withstand certain loads while rotating at high speeds. Once a failure occurs, it can directly affect the safety of train operations. Therefore, it is of great significance to establish a reliable remaining useful life model to ensure train operation safety. Addressing the problems of redundant features in the existing multi-feature fusion process, which affects diagnostic performance, and the spurious fluctuations in the fusion features, which cause inaccurate determination of the start fault time and lead to low prediction accuracy of remaining useful life, we propose a method based on adaptive feature fusion and the autoregressive integrated moving average model for rolling bearing prediction. Firstly, features from the time domain, frequency domain, and entropy are extracted. A feature selection mechanism with minimum redundancy is constructed to screen the optimal sensitive feature set. Secondly, based on adaptive feature fusion, the optimal sensitive feature set is dynamically fused, and the spurious fluctuations of the health index are corrected using linear regression and the 3σ principle. Next, a bottom-up time series segmentation method is employed to divide the health status of the Improved Health Indicators. Finally, a remaining useful life prediction model based on the autoregressive integrated moving average model is established. This study demonstrates that the proposed method effectively identifies features that are most sensitive to degradation trends, accurately determines the initial fault moment of bearings, and achieves effective prediction of the remaining useful life of bearings.

Enhancing gait recognition in lower limb exoskeletons: adaptive feature selection and random forest with Bayesian optimization

Abstract To improve human-machine cooperation and enhance the accuracy of gait recognition in wearable lower limb exoskeletons, an enhancement method of gait recognition based on adaptive feature selection and an optimized machine learning algorithm was proposed. In this study, surface electromyography (sEMG) signals of rectus femoris, medialis femoris, lateralis femoris, semitendinosus and biceps femoris were recorded during level-ground walking. Then, time domain, frequency domain, time-frequency domain and nonlinear features were extracted. The integrated value of electromyography, variance, root mean square and wavelength were selected as the time domain features, and the mean power frequency and median frequency were selected as the frequency domain features. Wavelet packet energy was selected as the time-frequency domain feature. Nonlinear features including approximate entropy, sample entropy and fuzzy entropy of sEMG were extracted. To reduce feature dimension and improve the calculation efficiency, adaptive feature selection was performed by particle swarm optimization combined with sigmoid function. Then, the feature matrix was determined as the input for a random forest classifier to recognize different gait phases. An adaptive optimization mechanism based on Bayesian optimization was applied to random forest. Compared with random forest, the overall performance of the optimized model was improved. Its accuracy was increased by 3.57%. The feature selection and adaptive optimization mechanisms in gait recognition not only enhance the accuracy of random forest algorithms applied to sEMG for gait prediction, but also facilitate the flexibility and consistency required for lower limb exoskeleton gait control.

A Collaborative Approach to Detecting DDoS Attacks in SDN Using Entropy and Deep Learning

Software-defined networking (SDN) is an approach to network management allowing to enhance the performance of the network and making it more flexible. The centralized architecture of SDN makes it vulnerable to cyberattacks, especially distributed denial of service (DDoS) attacks. Existing research investigates the detection of DDoS attacks separately on the control plane and data plane. However, there is a need for efficient and accurate detection of these attacks using features obtained from both control and data planes. Therefore, we present a mechanism for identifying DDoS attacks using entropy, multiple feature selection mechanisms, and deep learning. Initially, we use entropy on the control plane to detect anomalous activity and identify suspicious switches. Next, we capture traffic on the suspicious switches to detect DDoS attacks. To detect these attacks, we utilize multi-layer perceptron (MLP) deep learning models, convolutional neural network (CNN), and the long short-term memory (LSTM) approach. An InSDN dataset is used to train the model and test data are generated using Mininet emulation and the Ryu controller. The results reveal that LSTM outperforms MLP and CNN, achieving an accuracy of 99.83%.

Efficient prediction framework for large-scale nonlinear petrochemical process based on feature selection and temporal-attention LSTM: Applied to fluid catalytic cracking

Data-driven modeling plays a vital role in petrochemical industry, especially fluid catalytic cracking (FCC). However, the long operational cycles, large-scale measurements, multivariate data, and intricate temporal correlations in FCC units may lead to the problem of low prediction accuracy when only use a single time series data-driven modeling neural network such as long short-term memory (LSTM) network. To address these challenges, an effective prediction framework is proposed that integrates LSTM network with extreme gradient boosting (XGBOOST)-based feature selection and temporal-attention (TA) mechanisms. XGBOOST is applied to filter features related to the predictive variables in order to eliminate redundant variables. TA mechanisms within an LSTM network is used to capture the relevant historical time steps of the current moment. The results of our efficient prediction framework, applied to FCC process and three additional petrochemical processes, have proven to be superior to other methods.

Intrusion detection system using classification algorithms with feature selection mechanism over real-time data traffic

Intrusion detection system using classification algorithms with feature selection mechanism over real-time data traffic

CellRegNet: Point Annotation-Based Cell Detection in Histopathological Images via Density Map Regression.

Recent advances in deep learning have shown significant potential for accurate cell detection via density map regression using point annotations. However, existing deep learning models often struggle with multi-scale feature extraction and integration in complex histopathological images. Moreover, in multi-class cell detection scenarios, current density map regression methods typically predict each cell type independently, failing to consider the spatial distribution priors of different cell types. To address these challenges, we propose CellRegNet, a novel deep learning model for cell detection using point annotations. CellRegNet integrates a hybrid CNN/Transformer architecture with innovative feature refinement and selection mechanisms, addressing the need for effective multi-scale feature extraction and integration. Additionally, we introduce a contrastive regularization loss that models the mutual exclusiveness prior in multi-class cell detection cases. Extensive experiments on three histopathological image datasets demonstrate that CellRegNet outperforms existing state-of-the-art methods for cell detection using point annotations, with F1-scores of 86.38% on BCData (breast cancer), 85.56% on EndoNuke (endometrial tissue) and 93.90% on MBM (bone marrow cells), respectively. These results highlight CellRegNet's potential to enhance the accuracy and reliability of cell detection in digital pathology.

Zero-shot stance detection based on multi-expert collaboration

Zero-shot stance detection is pivotal for autonomously discerning user stances on novel emerging topics. This task hinges on effective feature alignment transfer from known to unseen targets. To address this, we introduce a zero-shot stance detection framework utilizing multi-expert cooperative learning. This framework comprises two core components: a multi-expert feature extraction module and a gating mechanism for stance feature selection. Our approach involves a unique learning strategy tailored to decompose complex semantic features. This strategy harnesses the expertise of multiple specialists to unravel and learn diverse, intrinsic textual features, enhancing transferability. Furthermore, we employ a gating-based mechanism to selectively filter and fuse these intricate features, optimizing them for stance classification. Extensive experiments on standard benchmark datasets demonstrate that our model significantly surpasses existing baseline models in performance.

Pest-ConFormer: A hybrid CNN-Transformer architecture for large-scale multi-class crop pest recognition

Crop pests are acknowledged as the major factors in reducing the yield and quality of agricultural production worldwide. It is an urgent necessity to recognize crop pests accurately to protect the crop in the early stage to reduce the loss for the agricultural economy. Due to the ecological characteristics of the crop pests and the complex background in fields, the crop pests show high inter-class similarity and significant intra-class variation in external morphology appearance, which makes current recognition methods suffer from low classification accuracy and poor generalization ability in complex natural environment recognition tasks. To tackle this problem, a hybrid convolutional neural network and transformer-based model, namely Pest-ConFormer, featured with multi-scale weakly supervised feature selection mechanisms is proposed, which has shown excellent multi-scale discriminative feature extraction in fine-grained visual classification (FGVC) tasks. This method employs a hybrid convolution-transformer encoder architecture pre-training in a self-supervised masked autoencoder manner as a backbone to learn pests’ highly discriminative features across various scales. Next, a dual-path feature aggregation structure with a top-down FPN-like feature pathway and a bottom-up PANet-like feature pathway based on attention mechanisms is designed to learn high-level global context information and low-level local detailed feature representation. Thirdly, a fine-grained classification module using weakly supervised learning is introduced to select the discriminative feature points in different pyramidal levels. Then, these feature points are fed into a graph convolutional network to accomplish classification. Several experiments are conducted on the large-scale multi-class IP102 benchmark dataset, and the proposed method achieves an accuracy of 77.81 % regarding crop pest recognition. The experimental results indicate that our approach outperforms other state-of-the-art methods by nearly 2 percent points, demonstrating that the proposed hybrid architecture with dual-path feature aggregation and fine-grained classification modules can be more effective in the crop pest recognition field than CNN-based methods and can be deployed in the practical natural environment. The source code will be available at https://github.com/mwfang/pestconformer.

Computational Cost Reduction in Multi-Objective Feature Selection Using Permutational-Based Differential Evolution

Feature selection is a preprocessing step in machine learning that aims to reduce dimensionality and improve performance. The approaches for feature selection are often classified according to the evaluation of a subset of features as filter, wrapper, and embedded approaches. The high performance of wrapper approaches for feature selection is associated at the same time with the disadvantage of high computational cost. Cost-reduction mechanisms for feature selection have been proposed in the literature, where competitive performance is achieved more efficiently. This work applies the simple and effective resource-saving mechanisms of the fixed and incremental sampling fraction strategies with memory to avoid repeated evaluations in multi-objective permutational-based differential evolution for feature selection. The selected multi-objective approach is an extension of the DE-FSPM algorithm with the selection mechanism of the GDE3 algorithm. The results showed high resource savings, especially in computational time and the number of evaluations required for the search process. Nonetheless, it was also detected that the algorithm’s performance was diminished. Therefore, the results reported in the literature on the effectiveness of the strategies for cost reduction in single-objective feature selection were only partially sustained in multi-objective feature selection.

An interpretable Bayesian clustering approach with feature selection for analyzing spatially resolved transcriptomics data.

Recent breakthroughs in spatially resolved transcriptomics (SRT) technologies have enabled comprehensive molecular characterization at the spot or cellular level while preserving spatial information. Cells are the fundamental building blocks of tissues, organized into distinct yet connected components. Although many non-spatial and spatial clustering approaches have been used to partition the entire region into mutually exclusive spatial domains based on the SRT high-dimensional molecular profile, most require an ad hoc selection of less interpretable dimensional-reduction techniques. To overcome this challenge, we propose a zero-inflated negative binomial mixture model to cluster spots or cells based on their molecular profiles. To increase interpretability, we employ a feature selection mechanism to provide a low-dimensional summary of the SRT molecular profile in terms of discriminating genes that shed light on the clustering result. We further incorporate the SRT geospatial profile via a Markov random field prior. We demonstrate how this joint modeling strategy improves clustering accuracy, compared with alternative state-of-the-art approaches, through simulation studies and 3 real data applications.

Correction: Enhanced coati optimization algorithm using elite opposition-based learning and adaptive search mechanism for feature selection

Correction: Enhanced coati optimization algorithm using elite opposition-based learning and adaptive search mechanism for feature selection

Progressive Adapting and Pruning: Domain-Incremental Learning for Saliency Prediction

Saliency prediction (SAP) plays a crucial role in simulating the visual perception function of human beings. In practical situations, humans can quickly grasp saliency extraction in new image domains. However, current SAP methods mainly concentrate on training models in single domains, which do not effectively handle diverse content and styles present in real-world images. As a result, it would be of great significance if SAP models could efficiently adjust to new image domains. To this end, this article aims to design SAP models that can imitate the incremental learning ability of human beings on multiple image domains and name domain-incremental saliency prediction (DISAP). To make a tradeoff between preventing the forgetting of historical domains and achieving high performance on new domains, we propose a progressively updated domain incremental encoder. This encoder consists of a domain-sharing branch and a domain-specific branch. The domain-sharing branch includes a feature selection mechanism to preserve crucial parameters after fine-tuning the model on each current domain. The remaining parameters are reserved to absorb knowledge from future domains. Furthermore, to capture the unique characteristics of each domain with relatively low computational overhead, we introduce a lightweight design to construct the domain-specific branch, enabling effective adaptation to new domains. Extensive experiments are conducted on multiple domain-incremental learning settings formed by four saliency prediction datasets, including Salicon, MIT1003, the art subset of CAT2000, and WebSal. The results demonstrate that our method outperforms existing methods significantly. The code is available at https://github.com/KaIi-github/DIL4SAP .

Enhanced coati optimization algorithm using elite opposition-based learning and adaptive search mechanism for feature selection

Enhanced coati optimization algorithm using elite opposition-based learning and adaptive search mechanism for feature selection

Feature selection for sentiment analysis using hybrid multiobjective evolutionary algorithm

As the volume of data continues to grow, the significance of text classification is on the rise. This vast amount of data majorly exists in the form of texts. Effective data preparation is essential to extract sentiment data from this vast amount of text, as irrelevant and redundant information can impede valuable insights. Feature selection is an important step in the data preparation phase as it eliminates irrelevant and insignificant features from the huge features set. There exist a large body of work related to feature selection for image processing but limited research is done for text data. While some studies recognize the significance of feature selection in text classification, but there is still need for more efficient sentiment analysis models that optimize feature selection and reduce computational. This manuscript aims to bridge these gaps by introducing a hybrid multi-objective evolutionary algorithm as a feature selection mechanism, combining the power of multiple objectives and evolutionary processes. The approach combines two feature selection techniques within a binary classification model: a filter method, Information Gain (IG), and an evolutionary wrapper method, Binary Multi-Objective Grey Wolf Optimizer (BMOGWO). Experimental evaluations are conducted across six diverse datasets. It achieves a reduction of over 90 percent in feature size while improving accuracy by nearly nine percent. These results showcase the model’s efficiency in terms of computational time and its efficacy in terms of higher classification accuracy which improves sentiment analysis performance. This improvement can be beneficial for various applications, including recommendation systems, reviews analysis, and public opinion observation. However, it’s crucial to acknowledge certain limitations of this study. These encompass the need for broader classifier evaluation, and scalability considerations with larger datasets. These identified limitations serve as directions for future research and the enhancement of the proposed approach.

Predictive model for a product without history using LightGBM. Pricing model for a new product

The article focuses on developing a predictive product pricing model using LightGBM. Also, the goal was to adapt the LightGBM method for regression problems and, especially, in the problems of forecasting the price of a product without history, that is, with a cold start.The article contains the necessary concepts to understand the working principles of the light gradient boosting machine, such as decision trees, boosting, random forests, gradient descent, GBM (Gradient Boosting Machine), GBDT (Gradient Boosting Decision Trees). The article provides detailed insights into the algorithms used for identifying split points, with a focus on the histogram-based approach.LightGBM enhances the gradient boosting algorithm by introducing an automated feature selection mechanism and giving special attention to boosting instances characterized by more substantial gradients. This can lead to significantly faster training and improved prediction performance. The Gradient-based One-Side Sampling (GOSS) and Exclusive Feature Bundling (EFB) techniques used as enhancements to LightGBM are vividly described. The article presents the algorithms for both techniques and the complete LightGBM algorithm.This work contains an experimental result. To test the lightGBM, a real dataset of one Japanese C2C marketplace from the Kaggle site was taken. In the practical part, a performance comparison between LightGBM and XGBoost (Extreme Gradient Boosting Machine) was performed. As a result, only a slight increase in estimation performance (RMSE, MAE, R-squard) was found by applying LightGBM over XGBoost, however, there exists a notable contrast in the training procedure’s time efficiency. LightGBM exhibits an almost threefold increase in speed compared to XGBoost, making it a superior choice for handling extensive datasets.This article is dedicated to the development and implementation of machine learning models for product pricing using LightGBM. The incorporation of automatic feature selection, a focus on highgradient examples, and techniques like GOSS and EFB demonstrate the model’s versatility and efficiency. Such predictive models will help companies improve their pricing models for a new product. The speed of obtaining a forecast for each element of the database is extremely relevant at a time of rapid data accumulation.

Analyzing Longitudinal Health Screening Data with Feature Ensemble and Machine Learning Techniques: Investigating Diagnostic Risk Factors of Metabolic Syndrome for Chronic Kidney Disease Stages 3a to 3b.

Longitudinal data, while often limited, contain valuable insights into features impacting clinical outcomes. To predict the progression of chronic kidney disease (CKD) in patients with metabolic syndrome, particularly those transitioning from stage 3a to 3b, where data are scarce, utilizing feature ensemble techniques can be advantageous. It can effectively identify crucial risk factors, influencing CKD progression, thereby enhancing model performance. Machine learning (ML) methods have gained popularity due to their ability to perform feature selection and handle complex feature interactions more effectively than traditional approaches. However, different ML methods yield varying feature importance information. This study proposes a multiphase hybrid risk factor evaluation scheme to consider the diverse feature information generated by ML methods. The scheme incorporates variable ensemble rules (VERs) to combine feature importance information, thereby aiding in the identification of important features influencing CKD progression and supporting clinical decision making. In the proposed scheme, we employ six ML models-Lasso, RF, MARS, LightGBM, XGBoost, and CatBoost-each renowned for its distinct feature selection mechanisms and widespread usage in clinical studies. By implementing our proposed scheme, thirteen features affecting CKD progression are identified, and a promising AUC score of 0.883 can be achieved when constructing a model with them.

A machine learning based depression screening framework using temporal domain features of the electroencephalography signals.

Depression is a serious mental health disorder affecting millions of individuals worldwide. Timely and precise recognition of depression is vital for appropriate mediation and effective treatment. Electroencephalography (EEG) has surfaced as a promising tool for inspecting the neural correlates of depression and therefore, has the potential to contribute to the diagnosis of depression effectively. This study presents an EEG-based mental depressive disorder detection mechanism using a publicly available EEG dataset called Multi-modal Open Dataset for Mental-disorder Analysis (MODMA). This study uses EEG data acquired from 55 participants using 3 electrodes in the resting-state condition. Twelve temporal domain features are extracted from the EEG data by creating a non-overlapping window of 10 seconds, which is presented to a novel feature selection mechanism. The feature selection algorithm selects the optimum chunk of attributes with the highest discriminative power to classify the mental depressive disorders patients and healthy controls. The selected EEG attributes are classified using three different classification algorithms i.e., Best- First (BF) Tree, k-nearest neighbor (KNN), and AdaBoost. The highest classification accuracy of 96.36% is achieved using BF-Tree using a feature vector length of 12. The proposed mental depressive classification scheme outperforms the existing state-of-the-art depression classification schemes in terms of the number of electrodes used for EEG recording, feature vector length, and the achieved classification accuracy. The proposed framework could be used in psychiatric settings, providing valuable support to psychiatrists.

PAR-Net: An Enhanced Dual-Stream CNN-ESN Architecture for Human Physical Activity Recognition.

Physical exercise affects many facets of life, including mental health, social interaction, physical fitness, and illness prevention, among many others. Therefore, several AI-driven techniques have been developed in the literature to recognize human physical activities. However, these techniques fail to adequately learn the temporal and spatial features of the data patterns. Additionally, these techniques are unable to fully comprehend complex activity patterns over different periods, emphasizing the need for enhanced architectures to further increase accuracy by learning spatiotemporal dependencies in the data individually. Therefore, in this work, we develop an attention-enhanced dual-stream network (PAR-Net) for physical activity recognition with the ability to extract both spatial and temporal features simultaneously. The PAR-Net integrates convolutional neural networks (CNNs) and echo state networks (ESNs), followed by a self-attention mechanism for optimal feature selection. The dual-stream feature extraction mechanism enables the PAR-Net to learn spatiotemporal dependencies from actual data. Furthermore, the incorporation of a self-attention mechanism makes a substantial contribution by facilitating targeted attention on significant features, hence enhancing the identification of nuanced activity patterns. The PAR-Net was evaluated on two benchmark physical activity recognition datasets and achieved higher performance by surpassing the baselines comparatively. Additionally, a thorough ablation study was conducted to determine the best optimal model for human physical activity recognition.