Wrapper Method Research Articles

On the Use of Kullback–Leibler Divergence for Kernel Selection and Interpretation in Variational Autoencoders for Feature Creation

This study presents a novel approach for kernel selection based on Kullback–Leibler divergence in variational autoencoders using features generated by the convolutional encoder. The proposed methodology focuses on identifying the most relevant subset of latent variables to reduce the model’s parameters. Each latent variable is sampled from the distribution associated with a single kernel of the last encoder’s convolutional layer, resulting in an individual distribution for each kernel. Relevant features are selected from the sampled latent variables to perform kernel selection, which filters out uninformative features and, consequently, unnecessary kernels. Both the proposed filter method and the sequential feature selection (standard wrapper method) were examined for feature selection. Particularly, the filter method evaluates the Kullback–Leibler divergence between all kernels’ distributions and hypothesizes that similar kernels can be discarded as they do not convey relevant information. This hypothesis was confirmed through the experiments performed on four standard datasets, where it was observed that the number of kernels can be reduced without meaningfully affecting the performance. This analysis was based on the accuracy of the model when the selected kernels fed a probabilistic classifier and the feature-based similarity index to appraise the quality of the reconstructed images when the variational autoencoder only uses the selected kernels. Therefore, the proposed methodology guides the reduction of the number of parameters of the model, making it suitable for developing applications for resource-constrained devices.

Measuring vertical track irregularities from instrumented heavy haul railway vehicle data using machine learning

This paper proposes a data-driven approach to estimating geometric track irregularities from instrumented railway vehicle (IRV) data. Machine learning is used to find the nonlinear mapping between IRV data and track irregularities. A dynamic model of the BRA1 railway vehicle was used to generate an artificial dataset that contains variables that are measured by the real BRA1 IRV and other variables measured by IRVs found in the literature. An extensive data analysis step was done to verify if the current instrumentation of the BRA1 IRV is sufficient for obtaining both lateral and vertical track irregularities. Feature engineering based on wagon movements, signal integration and time domain statistical metrics were applied to extract features and then the best features were selected using a wrapper method. Eight different regression ML models were trained and optimized after the feature selection using Optuna. The results show that, with the current instrumentation of the BRA1 IRV, obtaining lateral track irregularities is unlikely due to low correlation, however, vertical irregularities can be obtained with a root mean squared error (RMSE) of 0.556 mm. With postprocessing, the RMSE was further reduced to 0.410 mm.

Gynecological Healthcare: Unveiling Pelvic Masses Classification through Evolutionary Gravitational Neocognitron Neural Network Optimized with Nomadic People Optimizer.

Accurate and early detection of malignant pelvic mass is important for a suitable referral, triage, and for further care for the women diagnosed with a pelvic mass. Several deep learning (DL) methods have been proposed to detect pelvic masses but other methods cannot provide sufficient accuracy and increase the computational time while classifying the pelvic mass. To overcome these issues, in this manuscript, the evolutionary gravitational neocognitron neural network optimized with nomadic people optimizer for gynecological abdominal pelvic masses classification is proposed for classifying the pelvic masses (EGNNN-NPOA-PM-UI). The real time ultrasound pelvic mass images are augmented using random transformation. Then the augmented images are given to the 3D Tsallis entropy-based multilevel thresholding technique for extraction of the ROI region and its features are further extracted with the help of fast discrete curvelet transform with the wrapping (FDCT-WRP) method. Therefore, in this work, EGNNN optimized with nomadic people optimizer (NPOA) was utilized for classifying the gynecological abdominal pelvic masses. It was executed in PYTHON and the efficiency of the proposed method analyzed under several performance metrics. The proposed EGNNN-NPOA-PM-UI methods attained 99.8%. Ultrasound image analysis using the proposed EGNNN-NPOA-PM-UI methods can accurately predict pelvic masses analyzed with the existing methods.

Research on the construction of enterprise financial management information based on feature set

Abstract This study suggests a PSO-SVM financial crisis early warning model aid organizations in developing an efficient financial crisis early warning model for early financial crisis avoidance. For feature set selection, a wrapper method is utilized, in which the classifier’s learning algorithm and feature set selection are combined. The SVM is utilized as the classifier, and its feature set and kernel function parameters (C and δ 2) are employed as the placements of the particles in the PSO, with the SVM’s classification results being used as the adaption values. By achieving the near-optimal feature subset and kernel function parameters, the results are close to the model’s ideal predictions. The PSO technique is used to concurrently optimize the feature set and the kernel function parameters to remove unnecessary or redundant features. According to the findings of the model comparison, the SVM’s accuracy is 80.31%, which is less than the accuracy of the PSO-SVM model, which is 94.55%. The accuracy and recall of the PSO-SVM training set are 71.39% and 81.87%, respectively, while the accuracy and recall of the test set are both 70.53% and 72.29%, respectively. In comparison, the SVM model’s test set accuracy rate is just 34.57%, while its training set accuracy rate is only 44.01%. This suggests that the PSO-SVM model put forward in this research has seen significant improvements in both the accuracy of indicator selection and indicator interpretability.

New Feature Selection Approach for Photovoltaïc Power Forecasting Using KCDE

Feature selection helps improve the accuracy and computational time of solar forecasting. However, FS is often passed by or conducted with methods that do not suit the solar forecasting issue, such as filter or linear methods. In this study, we propose a wrapper method termed Sequential Forward Selection (SFS), with a Kernel Conditional Density Estimator (KCDE) named SFS-KCDE, as FS to forecast day-ahead regional PV power production in French Guiana. This method was compared to three other FS methods used in earlier studies: the Pearson correlation method, the RReliefF (RRF) method, and SFS using a linear regression. It has been shown that SFS-KCDE outperforms other FS methods, particularly for overcast sky conditions. Moreover, Wrapper methods show better forecasting performance than filter methods and should be used.

The Development of Data Warehouse and Data Mining System for Serious Mental Illness with High Risk to Violence (SMI-V) Psychiatric Patients: A Case Study of Thailand

This study aimed to develop the data warehouse and data mining system for serious mental illness with high risk to violence (SMI-V) psychiatric patients of Nakhon Ratchasima Rajanagarindra Psychiatric Hospital, and develop a classification model for SMI-V psychiatric patients. Star schema design and database management software package were used for development of data warehouse system. Moreover, data mining technique, i.e., feature selection by Wrapper method, along with various algorithms were applied for constructing the classification model to classify SMI-V patients from other psychiatric patients. Software programming were used to create web-based task with graphical user interface (GUI) that would allow users to work with the system via Internet. The results of data warehouse design and development showed that it consisted of one fact table surrounding with six dimensional tables. There were six classification algorithms analyzed in this study which were Random Forest, Random Tree, Decision Tree J48, ZeroR, OneR, and Multilayer Perceptron. The results of developing classification models revealed that the model using Decision Tree J48 algorithm achieved higher performance than other models in comparison. Keywords: Data Warehouse; Data Mining; Classification; Serious Mental Illness with High Risk to Violence; Psychiatric Patients

A modified recurrent neural network (MRNN) model for and breast cancer classification system

Breast cancer is most dangerous cancer among women. Image processing techniques are used for Breast cancer detection. A Block-based cross diagonal texture matrix (BCDTM) method is used first to extract Haralick’s features from each mammography ROI. Likewise, wrapper method is utilized to choose the crucial features from the condensed feature vector. There are lot of factors that affects the quality of the images such as salt or pepper noise. As a result, this is less precise and more prone to mistakes because of human. In order to address the problems, input breast image is first pre-processed via median filtering to reduce noise. ROI segmentation is done using weighted K means clustering. Feature extraction, texture and form descriptors based on Centroid Distance Functions (CDF) and BCDTM are used. Kernel Principal Component Analysis (KPCA) is used as dimensionality reduction on the extracted features. Improved Cuckoo Search Optimization (ICSO) is used to compute relevant feature selection. Modified Recurrent Neural Network (MRNN) is utilized to classify breast cancer into benign and malignant. Results show that the suggested model achieved highest accuracy, precision and recall values compared with other state-of-the-art approaches.

Derandomised knockoffs: leveraging e-values for false discovery rate control

Abstract Model-X knockoffs is a flexible wrapper method for high-dimensional regression algorithms, which provides guaranteed control of the false discovery rate (FDR). Due to the randomness inherent to the method, different runs of model-X knockoffs on the same dataset often result in different sets of selected variables, which is undesirable in practice. In this article, we introduce a methodology for derandomising model-X knockoffs with provable FDR control. The key insight of our proposed method lies in the discovery that the knockoffs procedure is in essence an e-BH procedure. We make use of this connection and derandomise model-X knockoffs by aggregating the e-values resulting from multiple knockoff realisations. We prove that the derandomised procedure controls the FDR at the desired level, without any additional conditions (in contrast, previously proposed methods for derandomisation are not able to guarantee FDR control). The proposed method is evaluated with numerical experiments, where we find that the derandomised procedure achieves comparable power and dramatically decreased selection variability when compared with model-X knockoffs.

Nested ensemble selection: An effective hybrid feature selection method

It has been shown that while feature selection algorithms are able to distinguish between relevant and irrelevant features, they fail to differentiate between relevant and redundant and correlated features. To address this issue, we propose a highly effective approach, called Nested Ensemble Selection (NES), that is based on a combination of filter and wrapper methods. The proposed feature selection algorithm differs from the existing filter-wrapper hybrid methods in its simplicity and efficiency as well as precision. The new algorithm is able to separate the relevant variables from the irrelevant as well as the redundant and correlated features. Furthermore, we provide a robust heuristic for identifying the optimal number of selected features which remains one of the greatest challenges in feature selection. Numerical experiments on synthetic and real-life data demonstrate the effectiveness of the proposed method. The NES algorithm achieves perfect precision on the synthetic data and near optimal accuracy on the real-life data. The proposed method is compared against several popular algorithms including mRMR, Boruta, genetic, recursive feature elimination, Lasso, and Elastic Net. The results show that NES significantly outperforms the benchmarks algorithms especially on multi-class datasets.

Modeling and simulation of the flash sintering: An approach to study the influence of process attributes on the flash phenomenon and microstructure

Field-assisted sintering still has gaps in explaining and modeling the flash phenomenon, as well as in enabling control over the sintering process to achieve desirable densification and microstructure control in ceramic materials.This study investigates methods for constructing computational models based on machine learning for predicting the onset temperature of the flash event (Tonset) and bulk density (BD) from a combination of various process attributes: powder composition, green sample geometry, electric field configuration, and heating. Samples of 3 mol% and 8 mol% yttria-stabilized zirconia were shaped into cylindrical geometry and were sintered using flash sintering and multi-step flash sintering processes, employing the following variations: current density; DC and AC electric field; and holding time. The experimental data were used for constructing and evaluating prediction models for Tonset and BD using k-fold cross-validation and the following algorithms: Random Forest, Support Vector Machine, Artificial Neural Network, K-Nearest Neighbors, and Multivariate Linear Regression Analysis. The model performance was assessed by comparing the measured values with the respective predicted values based on the Pearson correlation coefficient (r), mean absolute error (MAE), and root mean square error (RMSE). The models based on the Artificial Neural Network algorithm showed the best performance for predicting Tonset (r = 0.98; MAE = 11.88; RMSE = 14.88) and BD (r = 0.91; MAE = 1.86; RMSE = 2.30). The best models were used in conjunction with the Wrapper method for selecting attributes that significantly impact the sintering process, allowing the identification of 6 main attributes that impact Tonset and 9 attributes that impact BD in the flash-sintered samples.

A survey on multi-label feature selection from perspectives of label fusion

With the rapid advancement of big data technology, high-dimensional datasets comprising multi-label data have become prevalent in various fields. However, these datasets often contain more relevant and redundant features, which can adversely affect the performance of machine learning algorithms. Multi-label feature selection (MLFS) has emerged as a crucial pre-processing step in multi-label learning to address this issue. This survey provides an overview of multi-label learning and its algorithms, including problem transformation and algorithm adaptation. We also introduced three traditional strategies for MLFS: filter, wrapper, and embedded-based methods. Furthermore, we categorize existing research on multi-label feature selection into six aspects based on label fusion: label transformation-based (Binary Relevance-based and Label Powerset-based), label correlation-based (second and high-order, high and hybrid order), label specific-based, semi-supervised-learning-based, missing and noisy labels-based, and label enhancement-based approaches. We provide a detailed introduction to each method’s common approaches and theories. Additionally, we conduct experimental comparisons on practical multi-label learning datasets to evaluate the advantages and disadvantages of different algorithms. We discuss the application of multi-label feature selection in various domains, such as data mining, computer vision, natural language processing, and bio-informatics. Finally, we outline potential future research directions in multi-label feature selection, including MLFS with online learning, active learning, label distribution learning, partial label learning, granular computing, and class-imbalanced learning.

Predicting the Cognitive Ability of Young Women Using a New Feature Selection Algorithm.

Cognitive abilities are the capabilities to perform mental processes that include executive function, comprehension, decision-making, work performance, and educational attainment. This study aimed to investigate the relationship between several biomarkers and individuals' cognitive ability using various machine learning methods. A total of 144 young women aged between 18 and 24years old were recruited into the study. Cognitive performance was assessed using a standard questionnaire. A panel of biochemical, hematological, inflammatory, and oxidative stress biomarkers in serum and urine was measured for all participants. A novel combination of feature selection and feature scoring techniques within a hierarchical ensemble structure has been proposed to identify the most effective features in recognizing the importance of various biomarker signatures in cognitive abilities classification. Multiple feature selection methods were employed in conjunction with different classifiers to construct this model. In this manner, using three filter methods, the scores of each feature were considered. The combination of high-scoring features for each filter method was stored as the primary feature subset. A high-accuracy feature subset was selected by using a wrapper method. The collection of highly scored features from each filter method formed the primary feature subset. A wrapper method was also employed to select a feature subset with high accuracy. To ensure robustness and minimize random variations in the feature subset search process, a repeative tenfold cross-validation was conducted. The most frequently recurring features were determined. This iterative step facilitated the identification of an optimal feature subset, effectively reducing the dimensionality of features while maintaining accuracy. Among the 47 extracted factors, serum level of NOx (nitrite ± nitrate), alkaline phosphatase (ALP), and phosphate as well as blood platelet count (PLT) was entered into the model of cognitive abilities with the highest accuracy of approximately 70.9% using a decision tree classifier. Therefore, the serum levels of NOx, ALP, phosphate, and blood PLT count may be important markers of the cognitive abilities in apparently healthy young women. These factors my provide a simple procedure to identify mental abilities and earlier cognitive decline in healthy adults.

A Modified Binary Arithmetic Optimization Algorithm for Feature Selection

Feature selection chooses the optimal subset from the feature set without scarifying the information carried by the dataset. It is considered a complex combinatorial problem, so classical optimization techniques fail to solve it when the feature set becomes larger. Meta-heuristic approaches are well known to solve complex optimization problems; hence these algorithms have been successfully applied to extract optimal feature subsets. The arithmetic Optimization Algorithm is a newly proposed mathematics-based meta-heuristic search algorithm successfully applied to solve optimization problems. However, it has been observed that AOA experiences a poor exploration phase. Hence in the present work, a Modified Binary Arithmetic Optimization Algorithm (MB-AOA) is proposed, which solves the poor exploration problem of standard AOA. In the MB-AOA, instead of utilizing a single best solution, an optimal solution set that gradually shrinks after each successive iteration is applied for better exploration during initial iterations. Also, instead of a fixed search parameter (μ), the MB-AOA utilizes a variable parameter suitable for binary optimization problems. The proposed method is evaluated over seven real-life datasets from the UCI repository as a feature selection wrapper method and compared with standard AOA over two performance metrics, Average Accuracy, F-score, and the generated feature subset size. MB-AOA has performed better in six datasets regarding F-score and average accuracy. The obtained results from the simulation process demonstrate that the MB-AOA can select the relevant features, thus improving the classification task’s overall accuracy levels.

A comparative investigation of machine learning algorithms for predicting safety signs comprehension based on socio-demographic factors and cognitive sign features

This study examines whether the socio-demographic factors and cognitive sign features can be used for envisaging safety signs comprehensibility using predictive machine learning (ML) techniques. This study will determine the role of different machine learning components such as feature selection and classification to determine suitable factors for safety construction signs comprehensibility. A total of 2310 participants were requested to guess the meaning of 20 construction safety signs (four items for each of the mandatory, prohibition, emergency, warning, and firefighting signs) using the open-ended method. Moreover, the participants were asked to rate the cognitive design features of each sign in terms of familiarity, concreteness, simplicity, meaningfulness, and semantic closeness on a 0–100 rating scale. Subsequently, all eight features (age, experience, education level, familiarity, concreteness, meaningfulness, semantic closeness, and simplicity) were used for classification. Furthermore, the 14 most popular supervised classifiers were implemented and evaluated for safety sign comprehensibility prediction using these eight features. Also, filter and wrapper methods were used as feature selection techniques. Results of feature selection techniques indicate that among the eight features considered in this study, familiarity, simplicity, and meaningfulness are found to be the most relevant and effective components in predicting the comprehensibility of selected safety signs. Further, when these three features are used for classification, the K-NN classifier achieves the highest classification accuracy of 94.369% followed by medium Gaussian SVM which achieves a classification accuracy of 76.075% under hold-out data division protocol. The machine learning (ML) technique was adopted as a promising approach to addressing the issue of comprehensibility, especially in terms of determining factors affecting the safety signs' comprehension. The cognitive sign features of familiarity, simplicity, and meaningfulness can provide useful information in terms of designing user-friendly safety signs.

TF-Predictor: Transformer-Based Prerouting Path Delay Prediction Framework

Timing mismatch between different stages of physical design poses great challenges for circuit optimization to achieve the desired performance, power and area (PPA) tradeoff. The inaccurate timing estimation prior to routing may lead to over-design with unwanted power and area consumption or iterating back to cell placement at the cost of design turn-around time. Existing learning models could not predict post-routing circuit timing with satisfying accuracy and efficiency due to the limitations of the ignorance of delay correlation along the timing path and the empirical feature selection solutions. In this work, an accurate and efficient pre-routing path delay prediction framework is proposed by utilizing transformer network and residual model with an ensemble feature selection mechanism. Owing to the combined filter and wrapper methods, an ensemble feature selection mechanism is implemented to determine the optimal feature subset based on the timing and physical information at the placement stage for path delay prediction, which is extracted as feature sequences for each cell along the timing path to be trained by transformer network. With the residual model, the predicted timing mismatch between the placement and routing stages by transformer network is further calibrated to estimate the post-routing path delay. The proposed framework has been validated with ISCAS’85 and OpenCores benchmark circuits for the prediction of post-routing path delay, where the perdition error in terms of rRMSE is limited within 1.3% and 3.0% and the correlation coefficient R is higher than 0.999 and 0.995 for seen and unseen circuits respectively, indicating an error reduction by 2.3 10.6 times compared by prior learning-based models. In addition, the framework achieves average three orders of magnitude speedup compared with the commercial tools and is accelerated by a factor of 14 128 as against the competitive learning models, which is promising to be applied to guide design optimization prior to time-consuming routing stage.

Comparison of the Sedation Method Using Chloral Hydrate and Feed and the Wrap Method in Neonatal Brain Magnetic Resonance Imaging

Comparison of the Sedation Method Using Chloral Hydrate and Feed and the Wrap Method in Neonatal Brain Magnetic Resonance Imaging

Utilization of ultraviolet‐visible spectrophotometry in conjunction with wrapper method and correlated component regression for nitrite prediction outside the Beer–Lambert domain

AbstractThe determination of nitrite concentration is crucial due to its toxicity. A novel model has been developed to accurately determine nitrite concentration within the non‐linear range, utilizing the Zambelli method. Previously, techniques for measure nitrite concentration were primarily restricted to the linear range. This new method employs UV‐Visible absorption spectra and correlated component regression (CCR) to determine nitrite concentration within the range of 0.27–11.34 ppm. A wavelength selection strategy in conjunction with partial least squares (PLS) was implemented prior to applying CCR. The spectral data underwent pre‐processing using standard normal variant (SNV) and Savitzky Golay (SG) techniques, and a backward selection (BS) strategy with PLS was applied to select wavelengths. The 15 most sensitive wavelengths, determined through the RMSECV criterion, were utilized to create a PLS model within the range 377–497 nm, resulting in a model with R2C = 0.9999 and R2CV = 0.9999, RMSEC = 0.006, RMSECV = 0.027. A CCR model was then established using the 15selected wavelengths and nitrite concentration. The results yielded strong correlation between predicted and measured nitrite values with R2C = 0.9996, RMSEC = 4.7491 E‐15, RMSECV = 0.0004, and MAPE = 0.68%. The method has been validated through an accuracy profile, which demonstrates that 80% of future results will fall within the 10% acceptability limit within the validation range of 1.30–8.83 mg/L.

Binary dynamic stochastic search algorithm with support vector regression for feature selection in low-velocity impact localization problem

Locating low-velocity impacts (LVIs) on composite plates precisely is necessary. Support vector regression (SVR) is an effective method in addressing the LVI localization problem, whereas a large number of impact features may lead to a slow computational rate and over-fitting of the SVR model. In this paper, a binary dynamic stochastic search algorithm (BDSS) is proposed by introducing a threshold factor and the encoding concept of genetic algorithm into the original dynamic stochastic search algorithm (DSS). Then, a feature selection method is proposed by combining BDSS with SVR, which is called BDSS-SVR. BDSS-SVR as a wrapper method can simultaneously implement dimensionality reduction of impact features and optimize the SVR model’s parameters. Furthermore, a novel LVI localization method based on BDSS-SVR and multi-domain features is designed for accurately detecting the LVIs on a carbon fiber reinforced plastic (CFRP) plate. To analyze the performance of BDSS-SVR, a LVI localization system based on fiber Bragg grating (FBG) sensors is applied to the conduction of two experiments. Two additional control parameters of BDSS-SVR, including the threshold factor and weight coefficient, are tuned in the first experiment. Moreover, the statistical results in the second experiment illustrate that BDSS-SVR outperforms optimized SVR using DSS, feature selection methods based on five state-of-the-art algorithms and SVR, and five machine learning methods. For fifteen random LVIs on the CFRP plate, BDSS-SVR effectively reduces the number of impact features and provides satisfactory localization accuracy. The maximum, minimum, and average errors are 4.659 mm, 0.024 mm, and 3.065 mm, respectively.

Multi-filter semi-supervised transformer model for fault diagnosis

Dissolved Gas Analysis (DGA) is the most commonly used method for power transformer fault diagnosis. However, very few reliable and labeled fault DGA samples are available in the transformer substation whilst DGA data without labels is easier to obtain, which makes it difficult to train fault detectors in high-dimensional input space or select features using wrapper methods. Therefore, in order to improve the fault diagnosis accuracy using limited labeled DGA samples but more unlabeled DGA data, this paper proposes a novel multi-filter semi-supervised feature selection method for selecting optimal DGA features and building effective fault diagnosis models. A confidence criterion is also proposed for selecting high confidence unlabeled data to expand the training data set. Five filter techniques based on different evaluation criteria are employed to rank input DGA features, and a feature combination method is then applied to aggregate feature ranks by multiple filters and form a lower-dimensional candidate feature subset. The proposed method has been tested by using the IEC T10 dataset and compared with traditional supervised diagnostic models. The results show that the proposed method works well in optimizing DGA features and improving fault diagnosis accuracy significantly. Besides, the robustness of the selection of optimal feature subset is validated by testing DGA samples from the local power utility.

Performance Analysis of a new Filter and Wrapper Sequence for the Survivability Prediction of Breast Cancer Patients

Feature selection is an essential preprocessing step for removing redundant or irrelevant features from multidimensional data to improve predictive performance. Currently, medical clinical datasets are increasingly large and multidimensional and not every feature helps in the necessary predictions. So, feature selection techniques are used to determine relevant feature set that can improve the performance of a learning algorithm. This study presents a performance analysis of a new filter and wrapper sequence involving the intersection of filter methods, Mutual Information and Chi-Square followed by one of the wrapper methods: Sequential Forward Selection and Sequential Backward Selection to obtain a more informative feature set for improved prediction of the survivability of breast cancer patients from the clinical breast cancer dataset, SEER. The improvement in performance due to this filter and wrapper sequence in terms of Accuracy, False Positive Rate, False Negative Rate and Area under the Receiver Operating Characteristics curve is tested using the Machine learning algorithms: Logistic Regression, K-Nearest Neighbour, Decision Tree, Random Forest, Support Vector Machine and Multilayer Perceptron. The performance analysis supports the Sequential Backward Selection of the new filter and wrapper sequence over Sequential Forward Selection for the SEER dataset.