Robust Model Selection Research Articles

Enhancing Bank Credit Card Transaction Fraud Detection with Machine Learning Techniques

Abstract: Because of the increasing prevalence and sophistication of credit card theft, standard detection measures frequently fail. This study investigates the use of several machine learning algorithms to improve fraud detection in bank credit card transactions. The research aims to develop enhanced fraud detection systems by employing an integrated strategy involving data preprocessing, the application of various classification algorithms, and performance evaluation. The study thoroughly examines algorithms such as Random Forest, Logistic Regression, and Neural Networks, focusing on their predictive capabilities and practical applications. The findings indicate that machine learning provides a robust framework for increasing the accuracy and efficiency of fraud detection systems, consequently assisting financial institutions in protecting customer transactions and strengthening security measures. Moreover, by prioritizing robust model selection and feature engineering, banks can significantly enhance their fraud detection performance. This research could greatly influence how financial institutions handle fraud, potentially reducing losses, improving operational efficiency, and securing transaction environments. The insights gained from this study are also valuable for informing broader financial security strategies and guiding future research in the field.

Refining heart disease prediction accuracy using hybrid machine learning techniques with novel metaheuristic algorithms

Early diagnosis of heart disease is crucial, as it's one of the leading causes of death globally. Machine learning algorithms can be a powerful tool in achieving this goal. Therefore, this article aims to increase the accuracy of predicting heart disease using machine learning algorithms. Five classification models are explored: eXtreme Gradient Boosting (XGBC), Random Forest Classifier (RFC), Decision Tree Classifier (DTC), K-Nearest Neighbors Classifier (KNNC), and Logistic Regression Classifier (LRC). Additionally, four optimizers are evaluated: Slime mold Optimization Algorithm, Forest Optimization Algorithm, Pathfinder algorithm, and Giant Armadillo Optimization. To ensure robust model selection, a feature selection technique utilizing k-fold cross-validation is employed. This method identifies the most relevant features from the data, potentially improving model performance. The top three performing models are then coupled with the optimization algorithms to potentially enhance their generalizability and accuracy in predicting heart failure. In the final stage, the shortlisted models (XGBC, RFC, and DTC) were assessed using performance metrics like accuracy, precision, recall, F1-score, and Matthews Correlation Coefficient (MCC). This rigorous evaluation identified the XGGA hybrid model as the top performer, demonstrating its effectiveness in predicting heart failure. XGGA achieved impressive metrics, with an accuracy, precision, recall, and F1-score of 0.972 in the training phase, underscoring its robustness. Notably, the model's predictions deviated by less than 5.5 % for patients classified as alive and by less than 1.2 % for those classified as deceased compared to the actual outcomes, reflecting minimal error and high predictive reliability. In contrast, the DTC base model was the least effective, with an accuracy of 0.840 and a precision of 0.847. Overall, the optimization using the GAO algorithm significantly enhanced the performance of the models, highlighting the benefits of this approach.

Green Capital Dynamics: Investigating the Impact of Capital Structure on Profitability among IDX LQ45 Low Carbon Leaders in Indonesia

Research Aims: This study explores the nexus between capital structure and financial performance among firms listed in Indonesia's IDX LQ45 Low Carbon Leaders. The study focuses on capital structure, operationalized through leverage (Total Liabilities/Total Assets), and financial performance metrics, specifically Return on Assets (ROA) and Return on Equity (ROE). Design/methodology/approach: The research employs the Hausman test to select appropriate panel data models, subsequently validated by the Breusch-Pagan test. This rigorous approach ensures robust model selection between fixed-effect and random-effect specifications. Research Findings: Based on ROA and ROE measurements, leverage's influence on financial performance does not consistently show significant statistical importance. Theoretical Contribution/Originality: This study enriches financial literature by supporting the findings of Miller and Modigliani, which indicate that capital structure does not directly impact profitability, particularly in the context of companies listed in the IDX LQ45 Low Carbon Leaders in Indonesia.

Predicting the Liquid Steel End-Point Temperature during the Vacuum Tank Degassing Process Using Machine Learning Modeling

The present work focuses on predicting the steel melt temperature following the vacuum treatment step in a vacuum tank degasser (VTD). The primary objective is to establish a comprehensive methodology for developing and validating machine learning (ML) models within this context. Another objective is to evaluate the model by analyzing the alignment of the SHAP values with metallurgical domain expectations, thereby validating the model’s predictions from a metallurgical perspective. The proposed methodology employs a Random Forest model, incorporating a grid search with domain-informed variables grouped into batches, and a robust model-selection criterion that ensures optimal predictive performance, while keeping the model as simple and stable as possible. Furthermore, the Shapley Additive Explanations (SHAP) algorithm is employed to interpret the model’s predictions. The selected model achieved a mean adjusted R2 of 0.631 and a hit ratio of 75.3% for a prediction error within ±5 °C. Despite the moderate predictive performance, SHAP highlighted several aspects consistent with metallurgical domain expertise, emphasizing the importance of domain knowledge in interpreting ML models. Improving data quality and refining the model framework could enhance predictive performance.

Estimation of instantaneous peak flows in Canadian rivers: an evaluation of conventional, nonlinear regression, and machine learning methods.

Instantaneous peak flows (IPFs) are often required to derive design values for sizing various hydraulic structures, such as culverts, bridges, and small dams/levees, in addition to informing several water resources management-related activities. Compared to mean daily flows (MDFs), which represent averaged flows over a period of 24 h, information on IPFs is often missing or unavailable in instrumental records. In this study, conventional methods for estimating IPFs from MDFs are evaluated and new methods based on the nonlinear regression framework and machine learning architectures are proposed and evaluated using streamflow records from all Canadian hydrometric stations with natural and regulated flow regimes. Based on a robust model selection criterion, it was found that multiple methods are suitable for estimating IPFs from MDFs, which precludes the idea of a single universal method. The performance of machine learning-based methods was also found reasonable compared to conventional and regression-based methods. To build on the strengths of individual methods, the fusion modeling concept from the machine learning area was invoked to synthesize outputs of multiple methods. The study findings are expected to be useful to the climate change adaptation community, which currently heavily relies on MDFs simulated by hydrologic models.

Accelerating Adversarially Robust Model Selection for Deep Neural Networks via Racing

Recent research has introduced several approaches to formally verify the robustness of neural network models against perturbations in their inputs, such as the ones that occur in adversarial attacks. At the same time, this particular verification task is known to be computationally challenging. More specifically, assessing the robustness of a neural network against input perturbations can easily take several hours of compute time per input vector, even when using state-of-the-art verification approaches. In light of this, it becomes challenging to select from a given set of neural network models the one that is best in terms of robust accuracy, i.e., the fraction of instances for which the model is known to be robust against adversarial perturbations, especially when given limited computing resources. To tackle this problem, we propose a racing method specifically adapted to the domain of robustness verification. This racing method utilises Delta-values, which can be seen as an efficiently computable proxy for the distance of a given input to a neural network model to the decision boundary. We present statistical evidence indicating significant differences in the empirical cumulative distribution between robust and non-robust inputs as a function of Delta-values. Using this information, we show that it is possible to reliably expose vulnerabilities in the model with relatively few input iterations. Overall, when applied to selecting the most robust network from sets of 31 MNIST and 27 CIFAR-10 networks, our proposed method achieves speedups of a factor of 108 and 42, respectively, in terms of cumulative running time compared to standard local robustness verification on the complete testing sets.

Robust Model Selection for Plant Leaf Image Recognition Based on Evolutionary Ant Colony Optimization With Learning Rate Schedule

Robust Model Selection for Plant Leaf Image Recognition Based on Evolutionary Ant Colony Optimization With Learning Rate Schedule

Gaussian Mixture Model for Marine Reverberations

Ocean reverberations, a significant interference source in active sonar, arise as a response generated by random scattering at the receiving end, a consequence of randomly distributed clutter or irregular interfaces. Statistical analysis of reverberation data has revealed a predominant adherence to the Rayleigh distribution, signifying its departure from specific distribution forms like the Gaussian distribution. This study introduces the Gaussian mixture model, capable of simulating random variables conforming to a wide array of distributions through the integration of an adequate number of components. Leveraging the unique statistical attributes of reverberation, we initiate the Gaussian mixture model’s parameters via the frequency histogram of the reverberation data. Subsequently, model parameters are estimated using the expectation–maximization (EM) algorithm and the most suitable statistical model is selected based on robust model selection criteria. Through a comprehensive evaluation that encompasses both simulated and observed data, our results underscore the Gaussian mixture model’s effectiveness in accurately characterizing the distribution of reverberation data, yielding a mean squared error of less than 4‰.

Multi‐aspect permutation tests for model selection

AbstractIn typical machine learning frameworks, model selection is of fundamental importance: commonly, multiple models have to be trained and compared in order to identify the one with the best predictive performances. The aim of this study is to provide a new tool to improve the model selection process, allowing the user to identify the algorithm which significantly outperforms the other candidates. It proposes a robust model selection procedure based on a multi‐aspect permutation test which makes it possible to detect differences in both location and variability between two paired samples of prediction errors. A new extension of the nonparametric combination (NPC) methodology is therefore introduced and is integrated with an appropriate ranking procedure in order to deal with the comparison of candidate models. A simulation study is conducted to evaluate the performances of this testing procedure in 2‐sample and ‐sample problems, by generating data from various well‐known distributions and simulating several possible null and alternative scenarios. The adoption of the proposed technique in machine learning model selection problems is then discussed by means of multiple real data applications. These applications confirm what emerges from the simulation study: the introduced NPC‐based approach performs well under several different scenarios and represents a valuable tool for robust machine learning model selection.

Model Selection for independent not identically distributed observations based on Rényi’s pseudodistances

Model selection criteria are rules used to select the best statistical model among a set of candidate models, striking a trade-off between goodness of fit and model complexity. Most popular model selection criteria measures the goodness of fit trough the model log-likelihood function, yielding to non-robust criteria. This paper presents a new family of robust model selection criteria for independent but not identically distributed observations (i.n.i.d.o.) based on the Rényi’s pseudodistance (RP). The RP-based model selection criterion is indexed with a tuning parameter α controlling the trade-off between efficiency and robustness. Some theoretical results about the RP criterion are derived and the theory is applied to the multiple linear regression model, obtaining explicit expressions of the model selection criterion. Moreover, restricted models are considered and explicit expressions under the multiple linear regression model with nested models are accordingly derived. Finally, a simulation study empirically illustrates the robustness advantage of the method.

Robust Transformations for Multiple Regression via Additivity and Variance Stabilization

ABSTRACT Outliers can have a major effect on the estimated transformation of the response in linear regression models, as they can on the estimates of the coefficients of the fitted model. The effect is more extreme in the Generalized Additive Models (GAMs) that are the subject of this article, as the forms of terms in the model can also be affected. We develop, describe and illustrate robust methods for the nonparametric transformation of the response and estimation of the terms in the model. Numerical integration is used to calculate the estimated variance stabilizing transformation. Robust regression provides outlier free input to the polynomial smoothers used in the calculation of the response transformation and in the backfitting algorithm for estimation of the functions of the GAM. Our starting point was the AVAS (Additivity and VAriance Stabilization) algorithm of Tibshirani. Even if robustness is not required, we have made four further general optional improvements to AVAS which greatly improve the performance of Tibshirani’s original Fortran program. We provide a publicly available and fully documented interactive program for our procedure which is a robust form of Tibshirani’s AVAS that allows many forms of robust regression. We illustrate the efficacy of our procedure through data analyses. A refinement of the backfitting algorithm has interesting implications for robust model selection. Supplementary materials for this article are available online.

Establishing semantic relatedness through ratings, reaction times, and semantic vectors: A database in Polish.

This study presents a Polish semantic priming dataset and semantic similarity ratings for word pairs obtained with native Polish speakers, as well as a range of semantic spaces. The word pairs include strongly related, weakly related, and semantically unrelated word pairs. The rating study (Experiment 1) confirmed that the three conditions differed in semantic relatedness. The semantic priming lexical decision study with a carefully matched subset of the stimuli (Experiment 2), revealed strong semantic priming effects for strongly related word pairs, whereas weakly related word pairs showed a smaller but still significant priming effect relative to semantically unrelated word pairs. The datasets of both experiments and those of SimLex-999 for Polish were then used in a robust semantic model selection from existing and newly trained semantic spaces. This database of semantic vectors, semantic relatedness ratings, and behavioral data collected for all word pairs enable future researchers to benchmark new vectors against this dataset. Furthermore, the new vectors are made freely available for researchers. Although similar semantically strongly and weakly related word pairs are available in other languages, this is the first freely available database for Polish, that combines measures of semantic distance and human data.

Integrated data-driven modeling and experimental optimization of granular hydrogel matrices

Integrated data-driven modeling and experimental optimization of granular hydrogel matrices

On robustness of model selection criteria based on divergence measures: Generalizations of BHHJ divergence-based method and comparison

In model selection problems, robustness is one important feature for selecting an adequate model from the candidates. We focus on statistical divergence-based selection criteria and investigate their robustness. We mainly consider BHHJ divergence and related classes of divergence measures. BHHJ divergence is a representative robust divergence measure that has been utilized in, for example, parametric estimation, hypothesis testing, and model selection. We measure the robustness against outliers of a selection criterion by approximating the difference of values of the criterion between the population with outliers and the non-contaminated one. We derive and compare the conditions to guarantee robustness for model selection criteria based on BHHJ and related divergence measures. From the results, we find that conditions for robust selection differ depending on the divergence families, and that some expanded classes of divergence measures require stricter conditions for robust model selection. Moreover, we prove that robustness in estimation does not always guarantee robustness in model selection. Through numerical experiments, we confirm the advantages and disadvantages of each divergence family, asymptotic behavior, and the validity for employing criteria on the basis of robust divergence. Especially, we reveal the superiority of BHHJ divergence in robust model selection for extensive cases.

Robust model selection for positive and unlabeled learning with constraints

Robust model selection for positive and unlabeled learning with constraints

A deep, multi-epoch Chandra HETG study of the ionized outflow from NGC 4051

ABSTRACT Actively accreting supermassive black holes significantly impact the evolution of their host galaxies, truncating further star formation by expelling large fractions of gas with wide-angle outflows. The X-ray band is key to understanding how these black hole winds affect their environment, as the outflows have high temperatures (∼105–8 K). We have developed a Bayesian framework for characterizing active galactic nucleus outflows with an improved ability to explore parameter space and perform robust model selection. We applied this framework to a new 700 ks and an archival 315 ks Chandra High Energy Transmission Gratings observation of the Seyfert galaxy NGC 4051. We have detected six absorbers intrinsic to NGC 4051. These wind components span velocities from 400 to 30 000 km s−1. We have determined that the most statistically significant wind component is purely collisionally ionized, which is the first detection of such an absorber. This wind has T ≈ 107 K and v ≈ 880 km s−1 and remains remarkably stable between the two epochs. Other slow components also remain stable across time. Fast outflow components change their properties between 2008 and 2016, suggesting either physical changes or clouds moving in and out of the line of sight. For one of the fast components, we obtain one of the tightest wind density measurements to date, log n/(cm−3) = 13.0$^{+0.01}_{-0.02}$, and determine that it is located at ∼240 gravitational radii. The estimated total outflow power surpasses 5 per cent of the bolometric luminosity (albeit with large uncertainties) making it important in the context of galaxy–black hole interactions.

Developing a novel stock index trend predictor model by integrating multiple criteria decision-making with an optimized online sequential extreme learning machine.

It has always been the goal of many researchers to gain a thorough understanding of the patterns in the stock market and forecast the trends it will follow. The use of an advanced forecasting model can assist with accurately forecasting the future price of stocks, their fluctuations in the markets, as well as make profits in trading. With this motivation, in this study, a novel stock index trend predictor model is designed by integrating Multiple Criteria Decision-Making (MCDM) with an optimized Online Sequential Extreme Learning Machine (OSELM). Forecasting the future stock index prices and analyzing the upward or downward trends of these price forecasts are the two objectives of the proposed model. As the performance of OSELM is heavily dependent on the activation functions used in it, suitable selection of the activation function for OSELM is addressed as a MCDM problem. According to this approach, the trend prediction performance of six popular activation functions is assessed based on five regression-based and five classification-based criteria. In this investigation, three MCDM approaches are used to assess the performance matrix and determine which activation function is the best for OSELM based on six alternative models and ten criteria. To further optimize OSELM's performance, a hybrid crow search algorithm (hCSA) is incorporated in its training phase. By introducing the chaotic map and mutation operator in position update scheme and catfish behavior in the search process of original CSA, the proposed hCSA is able to achieve the right balance between exploration and exploitation improving the convergence. The proposed trend predictor model is empirically evaluated over historical data of three stock indices such as BSE SENSEX, S&P 500 and DJIA collected during pre-COVID and COVID time frame. In most of the test cases, the hCSA-OSELM model outperforms the state-of-the-art baseline models in terms of all evaluation criteria. When compared to the second-best baseline model, the suggested model is able to achieve the MSE improvements of 4–6%, 25–31%, and accuracy improvements of 0.4–0.8%, 0.9–1.3% over the pre-COVID and COVID time-frames, respectively. The statistical test also reveals the better performance of the proposed model. The robust and reliable MCDM-based model selection, superior prediction and classification outcomes clearly reveal that the proposed model can be used for financial time-series forecasting amid daily volatility as well as highly volatile markets.

Factorization of Binary Matrices: Rank Relations, Uniqueness and Model Selection of Boolean Decomposition

The application of binary matrices are numerous. Representing a matrix as a mixture of a small collection of latent vectors via low-rank decomposition is often seen as an advantageous method to interpret and analyze data. In this work, we examine the factorizations of binary matrices using standard arithmetic (real and nonnegative) and logical operations (Boolean and ℤ 2 ). We examine the relationships between the different ranks, and discuss when factorization is unique. In particular, we characterize when a Boolean factorization X = W ∧ H has a unique W , a unique H (for a fixed W ), and when both W and H are unique, given a rank constraint. We introduce a method for robust Boolean model selection, called BMF k , and show on numerical examples that BMF k not only accurately determines the correct number of Boolean latent features but reconstruct the pre-determined factors accurately.

MPSN: Motion-aware Pseudo-Siamese Network for indoor video head detection in buildings

Indoor video head detection is an essential component of building occupancy detection. While deep learning models have achieved remarkable progress in general object detection, their performance is limited in complex indoor scenes. Indoor surveillance videos often contain cluttered background objects, among which heads have small scales and diverse poses. In this paper, we propose Motion-aware Pseudo-Siamese Network (MPSN), an end-to-end approach that leverages head motion information to guide deep learning models to extract effective head features in indoor scenarios. By taking the pixel-wise difference between adjacent frames as the auxiliary input, MPSN effectively enhances human head motion information and removes irrelevant background objects. Its performance is compared with that of existing methods. Compared with existing methods, MPSN achieves superior performance on two indoor video datasets. It successfully suppresses static background objects and highlights the moving instances, especially human heads in indoor videos. Different methods to capture head motion also were compared. MPSN demonstrates simplicity and flexibility compared to other methods. We validate its robustness through adversarial experiments with a mathematical solution of small perturbations for robust model selection. To confirm its potential in building control systems, we apply MPSN to occupancy counting. The code is available at https://github.com/pl-share/MPSN.

A robust model selection framework for fault detection and system health monitoring with limited failure examples: Heterogeneous data fusion and formal sensitivity bounds

Fault detection models play a fundamental role in monitoring the health state of engineering systems subject to degradation processes. Data-driven fault detection models, albeit very effective when trained on large databases of failures, fail to perform well under a lack of failure examples. Because reliable engineering systems seldom fail, data shortage is often inevitable. To overcomes failure data scarcity, this work proposes a new model selection framework for the robust selection of fault detection and system health monitoring models. We combine heterogeneous sources of information (time-to-failure and sensors data), a model of the system structure, and mathematical bounds on the sensitivity and specificity of components fault classifiers. We use Support Vector Machines (SVMs) to detect components faults and Scenario theory to derive formal sensitivity and specificity bounds. The component predictions are combined within a system structure-function for system health states estimation. A novel model selection strategy optimizes the hyper-parameters of the SVM ensemble by minimizing system-level prediction errors and generalization error bounds of the individual fault classifiers. One of the main advantages of the proposed framework is a set of formal epistemic bounds on fault detection and false alarm probabilities quantifying the lack of data uncertainty affecting the fault detection rate. We test the method on three representative case studies: (1) On randomized fault detection experiments with synthetic data, (2) on ten SVM models for predictive maintenance of industrial health care imaging systems, and (3) on a real-world PHM challenge problem. The results prove the efficacy of the proposed approach and its usefulness for solving fault detection.