Feature Selection Research Articles

Bagging machine learning algorithms for rapid identification, classification, evaluation and upscaling in unconventional reservoir

Reservoir classification is a critical aspect of oil and gas fields development and management. It provides detailed geological data and serves as a scientific basis for optimizing well placement, reservoir fracturing stages, and perforation locations. Traditional reservoir classification relies on manual stratification based on formation attributes. However, this approach is time-consuming, subjective, and has a limited scope. To achieve more efficient and accurate classification, we develop a machine learning classification prediction model based on a large dataset of real-time drilling logs. By analyzing the raw logging data, performing feature extraction and selection, and applying dimensionality reduction, the study enhances data relevance. Various clustering algorithms are compared and optimized to develop the final model, which is then used to predict reservoir classification. The model is tested on over 500,000 field data points from 50 wells in five well areas. It demonstrates an average accuracy of 92.8% and achieves a maximum accuracy of 95%. These results indicate that the model is significant for sweet spot evaluation and for planning further development.

Complex product quality prediction based on ResNeSt model combining class-aware attention mechanism and dynamically updating class weights

Abstract The processing state data of complex products often exhibits high dimensionality, intricate feature relationships, and imbalanced distributions, hindering the accuracy of data-driven quality prediction models. To address these challenges, this paper proposes a complex product quality prediction model that integrates a class-aware attention mechanism and dynamic class weight update strategy. To handle the high dimensionality and complex correlations of the data, Pearson Correlation Coefficient and Gradient Boosting Decision Tree-based Recursive Feature Elimination (GBDT-RFE) algorithms are employed for feature selection. In response to the imbalanced distribution of data categories, a class-aware attention mechanism module is built upon the ResNeSt network, utilizing prior knowledge of data distribution to adjust the model’s focus. Furthermore, a loss function is designed to dynamically update class weights based on classification error rates, enabling the model to adaptively adjust the weight allocation for different classes, thereby enhancing its generalization capability. Experimental results on a semiconductor industry dataset demonstrate that the proposed model outperforms the original ResNeSt model in multiple metrics, with accuracy improved by 3.5%, AUC increased by 12.6%, F1 score raised by 18.9%, and recall enhanced by 24%, ultimately achieving an overall prediction accuracy of 98.7%. These multidimensional improvements make the model suitable for various complex product data scenarios such as electronics manufacturing, automotive industry, and biopharmaceuticals, demonstrating broad applicability.

Machine learning-based performance prediction for energy storage medium-deep borehole ground source heat pump systems

With the rapid development of artificial intelligence, data-driven prediction models play an important role in energy demand forecasting and performance prediction. This study established performance prediction models for medium-deep borehole ground source heat pump (MDB-GSHP) systems to predict the coefficient of performance (COP) of GSHP, utilizing back propagation neural network (BPNN), extreme learning machine (ELM), support vector machine (SVM), and particle swarm optimization-support vector machine (PSO-SVM) method that optimizes the penalty parameter (C), insensitive loss parameter (ε), and kernel parameter (γ) of SVM using PSO. Operational data were collected through field experiments, and typical parameters were selected as features to establish the feature set. The feature set was then optimized using Pearson correlation analysis and recursive feature elimination (RFE), resulting in the creation of five distinct feature sets. Error analysis and trend evaluation results indicate that the combined feature set (FS5), which incorporates the advantages of multiple feature selection methods, demonstrates the best performance. The PSO-SVM model exhibits outstanding performance under various input conditions, achieving an accuracy of over 90% within the error range. When using FS5, the PSO-SVM model achieves a mean absolute percentage error (MAPE) of 0.037, mean absolute error (MAE) of 0.103, root mean square error (RMSE) of 0.125, coefficient of multiple determinations (R2) of 0.970, and explained variance score (EVS) of 0.967, showcasing excellent predictive performance. The research results can provide a basis for the formulation of storage-related operational parameters and the optimization of system operation for energy storage in MDB-GSHP heating systems.

Principal Component Analysis With Fuzzy Elastic Net for Feature Selection

Principal Component Analysis With Fuzzy Elastic Net for Feature Selection

Enhancing Leaf Area Index Estimation in Southern Xinjiang Fruit Trees: A Competitive Adaptive Reweighted Sampling-Successive Projections Algorithm and Three-Band Index Approach with Fractional-Order Differentiation

The Leaf Area Index (LAI) is a key indicator for assessing fruit tree growth and productivity, and accurate estimation using hyperspectral technology is essential for monitoring plant health. This study aimed to improve LAI estimation accuracy in apricot, jujube, and walnut trees in Xinjiang, China. Canopy hyperspectral data were processed using fractional-order differentiation (FOD) from 0 to 2.0 orders to extract spectral features. Three feature selection methods—Competitive Adaptive Reweighted Sampling (CARS), Successive Projections Algorithm (SPA), and their combination (CARS-SPA)—were applied to identify sensitive spectral bands. Various band combinations were used to construct three-band indices (TBIs) for optimal LAI estimation. Random forest (RF) models were developed and validated for LAI prediction. The results showed that (1) the reflectance spectra of jujube and walnut trees were similar, while apricot spectra differed. (2) The correlation between fractional-order differential spectra and LAI was highest at orders 1.4 and 1.7, outperforming integer-order spectra. (3) The CARS-SPA selected a smaller set of feature bands in the 1100~2500 nm, reducing collinearity and improving spectral index construction. (4) The RF model using TBI4 demonstrated high R², low RMSE, and an RPD value > 2, indicating optimal prediction accuracy. This approach holds promise for hyperspectral LAI monitoring in fruit trees.

Evolutionary Multitasking for Multiobjective Feature Selection in Classification

Evolutionary Multitasking for Multiobjective Feature Selection in Classification

RBD-EVIO: optimized event-based visual-inertial odometry for a large field of view with a rotated binary DART descriptor.

Event-based cameras offer unique advantages over traditional cameras, such as high dynamic range, absence of motion blur, and microsecond-level latency. This paper introduces an innovative approach to visual odometry, to our knowledge, by integrating the newly proposed Rotated Binary DART (RBD) descriptor within a Visual-Inertial Navigation System (VINS)-based event visual odometry framework. Our method leverages event optical flow and RBD for precise feature selection and matching, ensuring robust performance in dynamic environments. We further validate the effectiveness of RBD in scenarios captured by a large field-of-view (FoV) fisheye event camera under high dynamic range and high-speed rotation conditions. Our results demonstrate significant improvements in tracking accuracy and robustness, setting what we believe to be a new benchmark for event-based visual odometry. This work paves the way for advanced applications in high-speed, high dynamic range, and large FoV visual sensing.

Creation of the MaaS readiness index with a modified AHP-ISM method

Smart mobility solutions are trending in the mobility domain realized through pilot activities and commercial solutions, but there is a lack of a broad framework defining the readiness to introduce such mobility solutions in a specific area. In this research, smart mobility solutions are examined in the perspective of the Mobility-as-a-Service (MaaS) scheme that is an adequate representation of the maturity of a region regarding smart mobility solutions including technology, business, and coopetition aspects. These three aspects define the feature selection, whereas surveys are used to collect input data from local experts (LEs). For weighting the features, the analytic hierarchy process (AHP) is used with a modified interpretive structural modeling (ISM). With this modification, an expert-friendly process is developed without affecting the results. The elaborated MaaS readiness index (MRI) is applied to six regions with different types of mobility related pilot activities to demonstrate the MRI as a comparison tool between regions and between ex-ante and ex-post pilot activities. The developed interpretive structural modeling with Graph (ISM-G) methodology requites remarkably less work from the evaluators compared to the ISM, while no important difference appeared in the results. The MRI can support smart mobility related pilot evaluations, whereas the ISM-G can be used widely in decision-making.

A novel deep anomaly detection approach for intrusion detection in futurisitic network

<span lang="EN-US">In an era where networks are increasingly heterogeneous and multi-domain, establishing robust security models to protect data and network infrastructure is becoming ever more complex. Traditional intrusion detection systems (IDS) often struggle with novel or variant attacks that fall outside predefined rule sets, resulting in significant detection challenges. This paper proposes a methodologically refined approach leveraging data-driven insights and statistically robust feature selection to enhance the training dataset. The study presents a long short-term memory-autoencoder (LSTM-AE) based learning model designed for multi-class anomaly detection. The model's novelty lies in its application of distance metrics to define distinct thresholds for varied attack classifications, a strategy that significantly amplifies detection precision. Experimental results validate the superior performance of the proposed system, achieving 94.82% accuracy rate, outperforming similar existing works. The study also proactively addresses common issues of class imbalance and skewed data representation in benchmark datasets by strategically training the model on normal traffic, enhancing its capability to generalize and identify anomalies effectively.</span>

Symmetrical Self-Representation and Data-Grouping Strategy for Unsupervised Feature Selection

Symmetrical Self-Representation and Data-Grouping Strategy for Unsupervised Feature Selection

Balancing Different Optimization Difficulty Between Objectives in Multiobjective Feature Selection

Balancing Different Optimization Difficulty Between Objectives in Multiobjective Feature Selection

Stacking based ensemble learning framework for identification of nitrotyrosine sites

Protein nitrotyrosine is an essential post-translational modification that results from the nitration of tyrosine amino acid residues. This modification is known to be associated with the regulation and characterization of several biological functions and diseases. Therefore, accurate identification of nitrotyrosine sites plays a significant role in the elucidating progress of associated biological signs. In this regard, we reported an accurate computational tool known as iNTyro-Stack for the identification of protein nitrotyrosine sites. iNTyro-Stack is a machine-learning model based on a stacking algorithm. The base classifiers in stacking are selected based on the highest performance. The feature map employed is a linear combination of the amino composition encoding schemes, including the composition of k-spaced amino acid pairs and tri-peptide composition. The recursive feature elimination technique is used for significant feature selection. The performance of the proposed method is evaluated using k-fold cross-validation and independent testing approaches. iNTyro-Stack achieved an accuracy of 86.3% and a Matthews correlation coefficient (MCC) of 72.6% in cross-validation. Its generalization capability was further validated on an imbalanced independent test set, where it attained an accuracy of 69.32%. iNTyro-Stack outperforms existing state-of-the-art methods across both evaluation techniques. The github repository is create to reproduce the method and results of iNTyro-Stack, accessible on: https://github.com/waleed551/iNTyro-Stack/

Multi-domain analysis of ultra-short-term HRV for breathing pattern classification in wearable health devices

This research paper introduces an innovative approach to classify heart rate variability (HRV) time series into paced and spontaneous breathing patterns to reflect changes in the autonomic nervous system. This type of classification is beneficial in wearable devices for stress/relaxation level detection and in deciding therapeutic interventions. The “Multi-Domain Approach” methodology integrates three different techniques: standard HRV features, fuzzy recurrence plot (FRP)-based FRP_GLCM, and empirical mode decomposition-based IMF_FRP_GLCM. The study concentrates on analyzing HRV time series within shorter data segments, aligning with the requirements of contemporary wearable health devices and biofeedback systems. HRV data collected during spontaneous and slow-paced breathing were analyzed across data segments of 5, 4, 3, 2, and 1 min, incorporating feature selection and reduction methods. Results demonstrated that standard HRV features yielded optimal performance for 5-min segments, achieving an average accuracy of 90%. Interestingly, IMF_FRP features achieved comparable accuracy even for 1-min segments. As segment duration decreased, standard HRV feature accuracy declined while IMF_FRP accuracy stayed intact, eventually matching 5-min segment accuracy levels. The study underscores the surging demand for shorter data segment HRV analysis, driven by advancements in wearable smart watches technology and mobile applications for monitoring health and managing stress.

Cloud-Cyber Physical Systems: Enhanced Metaheuristics with Hierarchical Deep Learning-based Cyberattack Detection

Cyber-Physical Systems (CPS) integrate several interconnected physical processes, networking units, and computing resources, along with monitoring the processes of the computing system. The connection between the cyber and physical world creates threatening security problems, particularly with the growing complexities of transmission networks. Despite efforts to overcome this challenge, it remains challenging to analyze and detect cyber-physical attacks in CPS. This study mainly focuses on the development of Enhanced Metaheuristics with Hierarchical Deep Learning-based Attack Detection (EMHDL-AD) method in a cloud-based CPS environment. The proposed EMHDL-AD method identifies various types of attacks to protect CPS. In the initial stage, data preprocessing is implemented to convert the input dataset into a useful format. Then, the Quantum Harris Hawks Optimization (QHHO) algorithm is used for feature selection. An Improved Salp Swarm Algorithm (ISSA) is used to optimize the hyperparameters of the HDL technique to recognize several attacks. The performance of the EMHDL-AD algorithm was examined using two benchmark intrusion datasets, and the experimental results indicated improvements over other existing approaches.

Patterns of Gene Expression Profiles Associated with Colorectal Cancer in Colorectal Mucosa by Using Machine Learning Methods.

Colorectal cancer (CRC) has a very high incidence and lethality rate and is one of the most dangerous cancer types. Timely diagnosis can effectively reduce the incidence of colorectal cancer. Changes in para-cancerous tissues may serve as an early signal for tumorigenesis. Comparison of the differences in gene expression between para-cancerous and normal mucosa can help in the diagnosis of CRC and understanding the mechanisms of development. This study aimed to identify specific genes at the level of gene expression, which are expressed in normal mucosa and may be predictive of CRC risk. A machine learning approach was used to analyze transcriptomic data in 459 samples of normal colonic mucosal tissue from 322 CRC cases and 137 non-CRC, in which each sample contained 28,706 gene expression levels. The genes were ranked using four ranking methods based on importance estimation (LASSO, LightGBM, MCFS, and mRMR) and four classification algorithms (decision tree [DT], K-nearest neighbor [KNN], random forest [RF], and support vector machine [SVM]) were combined with incremental feature selection [IFS] methods to construct a prediction model with excellent performance. The top-ranked genes, namely, HOXD12, CDH1, and S100A12, were associated with tumorigenesis based on previous studies. This study summarized four sets of quantitative classification rules based on the DT algorithm, providing clues for understanding the microenvironmental changes caused by CRC. According to the rules, the effect of CRC on normal mucosa can be determined.

Evidence from Machine Learning, Diagnostic Hub Genes in Sepsis and Diagnostic Models based on Xgboost Models, Novel Molecular Models for the Diagnosis of Sepsis.

Systemic multi-organ dysfunction resulting from dysregulated immune responses in the host triggered by microbial infection or other factors is a major cause of death in sepsis, and secretory pathways play an important role in it. GSE57065, GSE65682, GSE145227, and GSE54514 from Gene Expression Omnibus (GEO) were derived for this study. Secretory pathways single sample gene set enrichment analysis (ssGSEA) scores in sepsis and normal samples were exposed. Gene modules associated with secretory pathways were selected by weighted gene coexpression network analysis (WGCNA) for Protein-Protein Interaction Networks (PPI) assessment, and crossover genes in both were evaluated by eXtreme Gradient Boosting (XGBoost) model in feature selection to identify hub genes in sepsis. In addition, we explored the immune cells and signaling pathways regulated by hub genes. Remarkable dysregulation of secretory pathways was demonstrated in sepsis. The secretory pathways-associated gene modules were intimately involved in cytokine and immune responses in infection. Four crossover genes (CD163, FCER1G, C3AR1, ARG1) were present in WGCNA and PPI, and training in the XGBoost model revealed the best diagnostic performance of these 4 genes, meaning that these genes were the hub genes for sepsis. The 4-hub genes showed a significant negative correlation with T cell activity and a significant positive correlation with inflammatory immune cells. In addition, we found that the 4-hub genes markedly positively regulated INFLAMMATORY RESPONSE, IL6 JAK STAT3 SIGNALING. Based on WGCNA, PPI, and XGBoost models, we identified hub genes that play an important regulatory role in sepsis. We also developed novel molecular models for the diagnosis of sepsis.

GD-Net: An Integrated Multimodal Information Model Based on Deep Learning for Cancer Outcome Prediction and Informative Feature Selection.

Multimodal information provides valuable resources for cancer prognosis and survival prediction. However, the computational integration of this heterogeneous data information poses significant challenges due to the complex interactions between molecules from different biological modalities and the limited sample size. Here, we introduce GD-Net, a Graph Deep learning algorithm to enhance the accuracy of survival prediction with an average accuracy of 72% by early fusing of multimodal information, which includes an interpretable and lightweight XGBoost module to efficiently extract informative features. First, we applied GD-Net to eight cancer datasets and achieved superior performance compared to benchmarking methods, with an average 7.9% higher C-index value. The ablation experiments strongly supported that multi-modal integration could significantly improve accuracy over the single-modality model. In the deep case study of liver cancer, 319 differential genes, 15 differential miRNAs and 155 methylated differential genes based on the predicted risk subgroups are identified as the informative features, and then we have statistically and biologically validated the efficacy of these key molecules in internal and external test datasets. The comprehensive independent validations demonstrated that GD-Net is accurate and competitive in predicting different cancer outcomes in real-time, and it is an effective tool for identifying new multimodal prognosis biomarkers.

Hyperparameter recommendation via automated meta-feature selection embedded with kernel group Lasso learning

Hyperparameter recommendation via meta-learning relies on the characterization and quality of meta-features. These meta-features provide critical information about the underlying datasets but are often selected manually based on the practitioner’s experience and preference, which can be inefficient and ineffective in many applications. In this paper, we propose a novel hyperparameter recommendation approach that integrates with a Lasso-based multivariate kernel group (KGLasso) model. The developed KGLasso model automatically identifies primary meta-features through model training. By selecting the most explanatory meta-features for a specific meta-learning task, the recommendation performance becomes much more effective. Our KGLasso model builds on a group-wise generalized multivariate Lasso approach. Within this framework, we establish a minimization algorithm using a corresponding auxiliary function, which is mathematically proven to be convergent and robust. As an application, we develop a hyperparameter recommendation system using our built KGLasso model on 120 UCI datasets for the well-known support vector machine (SVM) algorithm. This system efficiently provides competent hyperparameter recommendations for new tasks. Extensive experiments, including comparisons with popular meta-learning baselines and search algorithms, demonstrate the superiority of our proposed approach. Our results highlight the benefits of integrating model learning and feature selection to construct an automated meta-learner for hyperparameter recommendation in meta-learning.

Picking point identification and localization method based on swin-transformer for high-quality tea

In the nature scene, because of the high degree of similarity between the background and the tea buds, as well as the different growth postures of the tea buds, finding and precisely identifying the picking point is challenging. To solve these issues, this paper proposes a precise way to find the best picking point for tea buds by combining traditional algorithms with Swin-Transformer-based target detection and semantic segmentation algorithms, namely SORC-SFT. Firstly, an improved target detection algorithm, Swin-Oriented R-CNN (SORC), is used to realize the recognition of four types of high-quality tea. The mean Average Precision (mAP) of the four categories was 82.3% after replacing the feature fusion network FPN with PAFPN and adding the Coordinate Attention (CA) mechanism. Secondly, the corresponding segmentation mask of the four recognized categories is obtained by adding Semask, Feature Alignment Module (FAM), and Feature Selection Module (FSM) to the improved semantic segmentation algorithm Semask-Fa-Transformer (SFT). The mean Intersection over Union (mIoU) of the semantic segmentation algorithm for each category is 89.83%, 91.97%, 88.85%, and 89.68%, respectively. Finally, the morphology of different categories of tea buds is analyzed, and the traditional algorithm is used to realize the accurate localization of the identified tea buds. For the four tested categories, the proportion of correct samples in locating picking points is 96.18%, 91.28%, 93.85%, and 90.58%, respectively. The experimental results show that, out of all the algorithms, the proposed picking point identification and localization approach has the best performance and will make a strong contribution to the accurate identification of tea leaves during the intelligent picking process.

Diabetic retinopathy detection via deep learning based dual features integrated classification model

Background The primary recognition of diabetic retinopathy (DR) is a pivotal requirement to prevent blindness and vision impairment. This deadly condition is identified by highly qualified professionals by examining colored retinal images. Objective The physical diagnostics for this condition was time-consuming and prone to fault. The development of computer-vision based intelligent systems has develop a main research area to effectually diagnosis the pathologies from an image. Methods In this research, a novel Deep learning based Dual Features Integrated classification (DD-FIC) framework is designed to detect the DR from a color retinal image. Initially, the fundus images are denoised by Wavelet integrated Retinex (WIR) algorithm to remove the noise artifacts which provide high contrast image. This DD-FIC model contains two phases of feature extraction module to evaluation of several retinal areas. Initially, global features of the fundus image are retrieved by the assist of attention fused efficient model, whereas the attention module dynamically highlights the important features. Afterwards, the segmented retinal vessels data is converted into features for learning the local features. Results Finally, the collective of features is processed into the Random Forest based feature selection model for the optimal prediction with five different classes using multi-class support vector machine (MCSVM). The efficacy of the proposed DD-FIC framework is estimated by Kaggle dataset with the detection accuracy of 98.6%. Conclusions The proposed framework rises the accuracy of 1.54%, 3.65%, 13.79% and 6.28% for Multi-channel CNN, CNN, VGG NiN and Shallow CNN respectively.