Hyperparameter Optimization Methods Research Articles

Improving classification accuracy of fine-tuned CNN models: Impact of hyperparameter optimization

The immense popularity of convolutional neural network (CNN) models has sparked a growing interest in optimizing their hyperparameters. Discovering the ideal values for hyperparameters to achieve optimal CNN training is a complex and time-consuming task, often requiring repetitive numerical experiments. As a result, significant attention is currently being devoted to developing methods aimed at tailoring hyperparameters for specific CNN models and classification tasks. While existing optimization methods often yield favorable image classification results, they do not provide guidance on which hyperparameters are worth optimizing, the appropriate value ranges for those hyperparameters, or whether it is reasonable to use a subset of training data for the optimization process. This work is focused on the optimization of hyperparameters during transfer learning, with the goal of investigating how different optimization methods and hyperparameter selections impact the performance of fine-tuned models. In our experiments, we assessed the importance of various hyperparameters and identified the ranges within which optimal CNN training can be achieved. Additionally, we compared four hyperparameter optimization methods—grid search, random search, Bayesian optimization, and the Asynchronous Successive Halving Algorithm (ASHA). We also explored the feasibility of fine-tuning hyperparameters using a subset of the training data. By optimizing the hyperparameters, we observed an improvement in CNN classification accuracy of up to 6%. Furthermore, we found that achieving a balance in class distribution within the subset of data used for parameter optimization is crucial in establishing the optimal set of hyperparameters for CNN training. The results we obtained demonstrate that hyperparameter optimization is highly dependent on the specific task and dataset at hand.

Prediction of transient NOX emission from a non-road diesel engine using a model combining Bayesian search and Population-based training

Due to the rising environmental concerns, particularly air quality, the emission regulations for non-road mobile machinery are becoming increasingly strict. Real-time emission prediction from diesel engines is significant for emission control and regional pollution estimation. This study aims to develop a machine learning model and optimize its hyperparameters by using a hyperparameter optimization method to NOX emission. Firstly, we collected NOX emission data from test under the non-road transient test cycle (NRTC) and built a significant dataset to choose a best model. Then, the model was trained by dataset and the hyperparameters were automatically optimized by combining Bayesian and Population based training. The accuracy of the optimized was indicated by an R2 value of 0.9784 with the 8 input features. The relative error in the cycle level was 1.3%. Lastly, the quality of NOX emissions during the cycle and the effect of each parameter on NOX emissions were analyzed. The results show that the model is able to predict the real-time concentration changes of NOX more accurately. It can provide a reference for the research and development of emission control technology for non-road mobile machinery.

Optimizing classification of diseases through language model analysis of symptoms

This paper investigated the use of language models and deep learning techniques for automating disease prediction from symptoms. Specifically, we explored the use of two Medical Concept Normalization—Bidirectional Encoder Representations from Transformers (MCN-BERT) models and a Bidirectional Long Short-Term Memory (BiLSTM) model, each optimized with a different hyperparameter optimization method, to predict diseases from symptom descriptions. In this paper, we utilized two distinct dataset called Dataset-1, and Dataset-2. Dataset-1 consists of 1,200 data points, with each point representing a unique combination of disease labels and symptom descriptions. While, Dataset-2 is designed to identify Adverse Drug Reactions (ADRs) from Twitter data, comprising 23,516 rows categorized as ADR (1) or Non-ADR (0) tweets. The results indicate that the MCN-BERT model optimized with AdamP achieved 99.58% accuracy for Dataset-1 and 96.15% accuracy for Dataset-2. The MCN-BERT model optimized with AdamW performed well with 98.33% accuracy for Dataset-1 and 95.15% for Dataset-2, while the BiLSTM model optimized with Hyperopt achieved 97.08% accuracy for Dataset-1 and 94.15% for Dataset-2. Our findings suggest that language models and deep learning techniques have promise for supporting earlier detection and more prompt treatment of diseases, as well as expanding remote diagnostic capabilities. The MCN-BERT and BiLSTM models demonstrated robust performance in accurately predicting diseases from symptoms, indicating the potential for further related research.

Exploring the Molecular Interaction of PCOS and Endometrial Carcinoma through Novel Hyperparameter-Optimized Ensemble Clustering Approaches

Polycystic ovary syndrome (PCOS) and endometrial carcinoma (EC) are gynecological conditions that have attracted significant attention due to the higher prevalence of EC in patients with PCOS. Even with this proven association, little is known about the complex molecular pathways that connect PCOS to an increased risk of EC. In order to address this, our study presents two main innovations. To provide a solid basis for our analysis, we have first created a dataset of genes linked to EC and PCOS. Second, we start by building fixed-size ensembles, and then we refine the configuration of a single clustering algorithm within the ensemble at each step of the hyperparameter optimization process. This optimization evaluates the potential performance of the ensemble as a whole, taking into consideration the interactions between each algorithm. All the models in the ensemble are individually optimized with the suitable hyperparameter optimization method, which allows us to tailor the strategy to the model’s needs. Our approach aims to improve the ensemble’s performance, significantly enhancing the accuracy and robustness of clustering outcomes. Through this approach, we aim to enhance our understanding of PCOS and EC, potentially leading to diagnostic and treatment breakthroughs.

Theoretical Followings and Practical Leadings of Network Ideological and Political Education in Colleges and Universities in the New Era

Abstract With the Internet era’s development, information technology’s role in education has become more prominent. In this paper, we firstly design the quality index of college network ideological and political education from the two broad dimensions of the goal (network ideological and political education work) and goal achievement (network ideological and political education effectiveness) of the quality of network ideological and political education in colleges and universities in the new era. Then, combining XGBoost algorithm and TPE hyperparameter optimization method, XGBoost is used as a classification model, and its iterative optimization is carried out by TPE hyperparameter optimization method, to establish the prediction model of the quality of ideological and political education based on XGBoost-TPE. Then, the online ideological and political education courses of university X are taken as the research object, and data are collected and calculated to study the prediction of the effectiveness of each index on the Civic and Political Education. Finally, the predicted and actual scores of the two ideology and politics courses are compared to explore the influence of teachers’ teaching quality factors on the quality of ideology and politics education. The analysis shows that the quality of course parenting, the quality of research parenting, and the home-school-society linkage parenting mechanism have the most significant influence on the predicted direct, indirect, and spillover effectiveness of ideological education, respectively, which are more than 60%. Teacher 1 and Teacher 2 contributed 2.38 and -1.13 to the achievement of Course D, and Teacher 3 and Teacher 4 contributed 2.47 and -2.53 to the accomplishment of Course E, respectively.

Two-Step Hyperparameter Optimization Method: Accelerating Hyperparameter Search by Using a Fraction of a Training Dataset

Abstract Hyperparameter optimization (HPO) is an important step in machine learning (ML) model development, but common practices are archaic—primarily relying on manual or grid searches. This is partly because adopting advanced HPO algorithms introduces added complexity to the workflow, leading to longer computation times. This poses a notable challenge to ML applications, as suboptimal hyperparameter selections curtail the potential of ML model performance, ultimately obstructing the full exploitation of ML techniques. In this article, we present a two-step HPO method as a strategic solution to curbing computational demands and wait times, gleaned from practical experiences in applied ML parameterization work. The initial phase involves a preliminary evaluation of hyperparameters on a small subset of the training dataset, followed by a reevaluation of the top-performing candidate models postretraining with the entire training dataset. This two-step HPO method is universally applicable across HPO search algorithms, and we argue it has attractive efficiency gains. As a case study, we present our recent application of the two-step HPO method to the development of neural network emulators for aerosol activation. Although our primary use case is a data-rich limit with many millions of samples, we also find that using up to 0.0025% of the data—a few thousand samples—in the initial step is sufficient to find optimal hyperparameter configurations from much more extensive sampling, achieving up to 135× speedup. The benefits of this method materialize through an assessment of hyperparameters and model performance, revealing the minimal model complexity required to achieve the best performance. The assortment of top-performing models harvested from the HPO process allows us to choose a high-performing model with a low inference cost for efficient use in global climate models (GCMs).

Stock Price Movement Classification Using Ensembled Model of Long Short-Term Memory (LSTM) and Random Forest (RF)

Stock investing is known worldwide as a passive income available for everyone. To increase the profit possibly gained, many researchers and investors brainstorm to gain a strategy with the most profit. Machine learning and deep learning are two of these approaches to predicting the stock's movement and deciding the strategy to gain as much as possible. To reach this goal, the researcher experiments with Random Forest (RF) and Long Short-Term Memory (LSTM) by trying them individually and merging them into an ensembled model. The researcher used RF to classify the results from LSTM models obtained throughout the Hyperparameter Optimization (HPO) process. This idea is implemented to lessen the time needed to train and optimize each LSTM model inside the ensembled model. Another anticipation done in this research to overcome the time needed to train the model is classifying the return for longer periods. The dataset used in this model is 45 stocks listed in LQ45 as of August 2021 This research results in showing that LSTM gives better results than RF model especially when using Bayesian Optimization as the HPO method, and that the ensembled model can return better precision in predicting stocks in comparison to the LSTM model itself. Future improvement can focus on the model structure, additional model types as the ensemble model estimator, improvement on the model efficiency, and datasets research to be used in predicting the stock movement prediction

Research on driver posture recognition and prediction in vehicle

The statistics show that more than 90% of traffic accidents are caused by the individual factors of the driver. Therefore, real-time monitoring of the current driver’s posture in the car and further predicting of the future driving posture are of great significance for designing the Adaptive Restraint System (ARS) to improve the safety protection of drivers. In this study, a real-time vehicle driver posture recognition and prediction method is proposed. The traditional Kinect-OpenPose algorithm, which has the problems such as lack of depth value, key point pulsation and measurement error of Kinect, was optimized. The OpenPose network structure and skeleton model are modified to improve the algorithm. In this study, a key point motion trajectory prediction network based on Long Short Term Memory (LSTM) is developed. The hyperparameter optimization method is combined with multi-layer grid search to obtain the best prediction effect in braking and steering conditions, and the driver’s future posture can be obtained by predicting the subsequent movement trajectory of key points. It has been verified that the method in this study has high accuracy, high robustness and low cost in key point recognition and prediction. And the improved processing speed of the algorithm did not affect the accuracy of driving posture recognition and prediction. This method predicted the driver’s posture in future of the collision, which can be used to guide the development of controlling system of the ARS.

A powerful Peripheral Arterial Disease detection using machine learning-based severity level classification model and hyper parameter optimization methods

This study addresses the critical need for improved detection and assessment of Peripheral Arterial Disease (PAD) using Artificial Intelligence (AI) algorithms, aiming to reduce global mortality rates associated with this condition. The research focuses on enhancing the efficiency of machine learning (ML) models for PAD detection and severity categorization by employing optimization techniques. Notably, this study introduces the application of Synthetic Minority Oversampling Technique (SMOTE) to mitigate imbalanced data challenges in PAD detection. The primary objective of this research is to develop highly efficient ML models utilizing six distinct AI classifiers and Hyper Parameter Optimization (HPO), specifically targeting PAD identification and severity level categorization. Leveraging the Cleveland and Statlog datasets, the study constructs and evaluates these models using all relevant features. The principal findings indicate remarkable advancements in PAD detection accuracy. The proposed models, optimized using SMOTE and Enhanced Decision Tree (EDT) through HPO, surpass state-of-the-art methodologies, achieving exceptional accuracies of 99.20% and 98.52% on the Cleveland and Statlog datasets, respectively. These models outperform existing approaches, assessed using comprehensive metrics such as F1 score, Accuracy, Precision, Recall, Specificity, and MCC. In conclusion, this study's innovative approach significantly improves the capability to assess PAD and categorize its severity levels accurately. The proposed models hold promise in aiding physicians to evaluate patients' cardiac conditions effectively, enabling early interventions that have the potential to mitigate mortality rates associated with cardiovascular diseases.

Stock Price Movement Classification Using Ensembled Model of Long Short-Term Memory (LSTM) and Random Forest (RF)

Stock investing is known worldwide as a passive income available for everyone. To increase the profit possibly gained, many researchers and investors brainstorm to gain a strategy with the most profit. Machine learning and deep learning are two of these approaches to predicting the stock's movement and deciding the strategy to gain as much as possible. To reach this goal, the researcher experiments with Random Forest (RF) and Long Short-Term Memory (LSTM) by trying them individually and merging them into an ensembled model. The researcher used RF to classify the results from LSTM models obtained throughout the Hyperparameter Optimization (HPO) process. This idea is implemented to lessen the time needed to train and optimize each LSTM model inside the ensembled model. Another anticipation done in this research to overcome the time needed to train the model is classifying the return for longer periods. The dataset used in this model is 45 stocks listed in LQ45 as of August 2021 This research results in showing that LSTM gives better results than RF model especially when using Bayesian Optimization as the HPO method, and that the ensembled model can return better precision in predicting stocks in comparison to the LSTM model itself. Future improvement can focus on the model structure, additional model types as the ensemble model estimator, improvement on the model efficiency, and datasets research to be used in predicting the stock movement prediction

An intelligent fault detection and diagnosis monitoring system for reactor operational resilience: Fault diagnosis

Various advanced reactor designs proposed in recent years envision deployment scenarios which feature reactor operations with significantly reduced staffing or even completely autonomous frameworks to reduce the operations and management costs of the plants. Many SMR and microreactor designs feature extended fuel cycles which limit inspection intervals, reduced access to critical components, and load-following capabilities that expose the reactor unit to different transients. Safe and reliable semi- or fully autonomous operations under these challenging operational regimes must be enabled through an on-line monitoring (OLM) system that effectively detects and diagnoses malfunctions in the reactor plant. This work presents the development of the Fault Diagnosis Module of the previously proposed data-driven OLM system for reactor operations: the Fault Detection and Diagnosis Monitoring System (FDDMS). The Fault Diagnosis Module monitors various sensor signatures from multiple systems and components at a nuclear power plant and accurately diagnoses the type and location of a malfunction. When integrated into the complete FDDMS methodology, the Fault Diagnosis Module can provide power transient dependent fault characterization by using separate convolutional neural network (CNN) models for steady state, ramping up in power, and ramping down in power operations, making the FDDMS especially applicable to load-following operational strategies. Two separate diagnosis approaches were explored in this paper for the Fault Diagnosis Module architecture: Hierarchical and End-to-End. The Hierarchical approach is a two-stage architecture in which the first stage uses a single CNN to classify the plant subsystem where the malfunction initiated. Subsequently in the second stage, a separate CNN is used for each subsystem to describe the specific fault type. The End-to-End approach uses a single CNN to directly classify the fault type from the overall list. To efficiently utilize computational resources, the hyperband intelligent hyperparameter optimization method to generate optimal CNN architectures. The diagnosis approaches were compared in terms of precision rate, recall rate, F1-score, and total accuracy for the possible fault types. Both diagnosis approaches produced good and satisfactory performance by generating total diagnosis accuracies above 99% on 17 different malfunction scenarios for all three power transient datasets. Additionally, robustness against noisy sensor data was tested, with models maintaining 99–100% accuracy at various levels of noise, and an illustrative real-time application of the methodology is provided.

Advanced hyperparameter optimization of deep learning models for wind power prediction

The uncertainty of wind power as the main obstacle of its integration into the power grid can be addressed by an accurate and efficient wind power forecast. Among the various wind power forecasting methods, machine learning (ML) algorithms, are recognized as a powerful wind power forecasting tool, however, their performance is highly dependent on the proper tuning of their hyperparameters. Common hyperparameter tuning methods such as grid search or random search are time-consuming, computationally expensive, and unreliable for complex models such as deep learning neural networks. Therefore, there is an urgent need for automatic methods to discover optimal hyperparameters for higher accuracy and efficiency of prediction models. In this study, a novel investigation is contributed to the field of wind power forecasting by a comprehensive comparison of three advanced techniques – Scikit-opt, Optuna, and Hyperopt – for hyperparameter optimization of Convolutional Neural Network (CNN) and Long Short-Term Memory Network (LSTM) models, a facet that, to our knowledge, has not been systematically explored in existing literature. The impact of these optimization techniques on the accuracy and efficiency of the CNN and LSTM models are assessed by comparing the root mean square error (RMSE) of the predictions and the required time to tune the models. The results show that the Optuna algorithm, using a Tree-structured Parzen Estimator (TPE) search method and Expected Improvement (EI) acquisition function, has the best efficiency for both CNN and LSTM models. In terms of accuracy, it is demonstrated that while for the CNN model all the optimization methods achieve similar performances, the LSTM model optimized by the Hyperopt algorithm, based on the annealing search method, results in the highest accuracy. In addition, for the first time in this research, the impact of the random initialization features on the performance of the forecasting models with neural networks is investigated. The proposed structures for deep learning models were examined to determine the most robust structure with the minimal sensitivity to the randomness. What we have discovered from the comparison of advanced hyperparameter optimization methods can be used by researchers to tune the time series-based forecasting models.

Machine Learning Approaches for Predicting Song Popularity: A Case Study in Music Analytics

Comprehending the aspects that impact song popularity has become crucial in the always changing music industry. This study explores the field of music popularity predictive modeling using the cutting-edge algorithms XGBoost and LightGBM. Predictive models developed by the study using a large dataset that includes a variety of musical variables, such as song duration, tempo, lyrical content, and release year. To improve the models' predictive capacity, the study approach includes extensive work. To provide a thorough assessment of the algorithms' performance, the dataset is divided into training and testing sets. Additionally, the effectiveness of XGBoost and LightGBM in forecasting music popularity is evaluated by a comparison analysis. To increase the prediction models' accuracy, hyperparameter optimization methods—specifically, Optuna—are used to fine-tune them. In addition, the study looks at feature importance, illuminating elements of music that, in the eyes of each algorithm, greatly add to its appeal. Using a rigorous cross-validation approach, the models are validated, and their generalization capabilities are shown. The performance metrics, which provide a comprehensive picture of the models' predicted accuracy, include mean absolute error, mean squared error, median absolute error, and R-squared. By providing a comparative analysis of two well-known machine learning methods for forecasting music popularity, this paper advances the rapidly developing field of music analytics. The results offer significant perspectives for professionals in the field and data scientists who are looking for efficient approaches to forecast music popularity across various genres. Keywords — Music Popularity Prediction; Machine Learning; XGBoost; LightGBM; Predictive Modeling; Feature Engineering; Hyperparameter Optimization; Data Analytics; Comparative Analysis; Song Characteristics; Genre Classification; Ensemble Models; Cross-Validation; Optuna; Data Preprocessing; Feature Importance; Regression; Music Analytics.

Active learning methods for strength assessment of circular CFST under coupled long-term axial loading and random localized corrosion

In harsh marine environments, the complex combination of loading and corrosion severely affects the performances of concrete-filled steel tubular (CFST) structures. Therefore, accurately assessing their strengths becomes both complicated and vital for structural safety design and maintenance. Previously, the random localized corrosion along the steel surface was often simplified as uniform corrosion, which failed to reflect its inherent randomness and uncertainty. This study proposes an efficient modeling technique that simulates the coupled effects of long-term loading and random localized corrosion. The developed finite element models are validated against a series of test data. By combining numerical simulation and literature collection, databases of CFST with non-corrosion, uniform corrosion and random localized corrosion are constructed, comprising a total of 4038 samples. Seven machine learning methods are employed to predict the strength of CFST under the above three corrosion scenarios, where the active learning approach is adopted to select a few yet valuable data for training models. A hyperparameter optimization method, considering both structural mechanism and prediction accuracy, is proposed to ensure that the mechanical mechanisms are accounted for. The results show that CFST columns with random localized corrosion exhibit considerably lower strengths compared to counterparts with uniform corrosion. Among the established machine learning models, extreme gradient boosting (XGBoost) and Gaussian process regression (GPR) perform the best. The active learning method achieves similar model performance using only 63 % of the data as that using the entire data, demonstrating its capability in effectively addressing challenges in the field of structural engineering such as limited data and costly data acquisition.

Real-time temperature prediction of lunar regolith drilling based on ATT-Bi-LSTM network

Lunar drilling is conducted in extreme environments, where limited heat dissipation pathways may cause overheating during drilling, potentially leading to destructive effects on the drill tool. To address the challenge of rapid temperature change in lunar regolith drilling, this paper presents a spatiotemporal feature fusion method for the real-time prediction of drilling temperature inside a lunar regolith simulant. The method aims to accurately predict the temperature of eight positions on the drill tool in the next second based on seven state parameters during drilling from the previous 20 seconds. It utilizes deep learning techniques to combine the Bi-LSTM and self-attention mechanism to fully exploit past and future information to extract spatiotemporal features from time series data and focus on the crucial features for temperature prediction. Additionally, a proposed early stopping-Bayesian hyperparameter optimization method is incorporated to effectively adjust the hyperparameters of the model. Experimental results demonstrate the superior performance of this method on data from drilling experiments with lunar regolith simulant in a vacuum chamber, significantly outperforming other models in statistical terms and meeting real-time prediction requirements. Furthermore, it exhibits robustness and generalization capability in practical applications on two different robotic drills. This research provides reliable support for predicting drilling temperature in future lunar exploration missions and reveals the spatiotemporal relationship in the thermophysical field during drilling. It holds crucial significance in guiding mission planners and operators to proactively implement appropriate measures in advance, ensuring the smooth operation of the drilling apparatus and comprehending the generation, conduction, and dissipation mechanisms of heat during lunar regolith drilling.

A Transfer-Based Convolutional Neural Network Model with Multi-Signal Fusion and Hyperparameter Optimization for Pump Fault Diagnosis.

Pumps are one of the core components of drilling equipment, and their fault diagnosis is of great significance. The data-driven approach has made remarkable achievements in the field of pump fault diagnosis; however, most of them are easily affected by complex background conditions and usually suffer from data scarcity problems in real-industrial scenarios, which limit their application in practical engineering. To overcome the above shortcoming, a novel framework for a model named Hyperparameter Optimization Multiple-Signal Fusion Transfer Convolution Neural Network is proposed in this paper. A convolutional neural network model based on transfer learning is built to promote well-learned knowledge transfer over different background conditions, improve robustness, and generalize the model to cross-domain diagnosis tasks. The multi-signal fusion strategy is involved in capturing system state information for establishing the mapping relationship between the raw signal and fault pattern by integrating the multi-physical signal with the weight allocation protocol. The hyperparameter optimization method is explored in conjunction with the transfer-based model by integrating Grid Search with the Gradient Descent algorithm for further improvement of diagnosis performance. Results show that the proposed model can effectively realize the fault diagnosis of pumps under different background conditions, achieving 95% accuracy.

Hybrid quantum ResNet for car classification and its hyperparameter optimization

Image recognition is one of the primary applications of machine learning algorithms. Nevertheless, machine learning models used in modern image recognition systems consist of millions of parameters that usually require significant computational time to be adjusted. Moreover, adjustment of model hyperparameters leads to additional overhead. Because of this, new developments in machine learning models and hyperparameter optimization techniques are required. This paper presents a quantum-inspired hyperparameter optimization technique and a hybrid quantum-classical machine learning model for supervised learning. We benchmark our hyperparameter optimization method over standard black-box objective functions and observe performance improvements in the form of reduced expected run times and fitness in response to the growth in the size of the search space. We test our approaches in a car image classification task and demonstrate a full-scale implementation of the hybrid quantum ResNet model with the tensor train hyperparameter optimization. Our tests show a qualitative and quantitative advantage over the corresponding standard classical tabular grid search approach used with a deep neural network ResNet34. A classification accuracy of 0.97 was obtained by the hybrid model after 18 iterations, whereas the classical model achieved an accuracy of 0.92 after 75 iterations.

An Intrusion Detection Method Based on Hybrid Machine Learning and Neural Network in the Industrial Control Field

Aiming at the imbalance of industrial control system data and the poor detection effect of industrial control intrusion detection systems on network attack traffic problems, we propose an ETM-TBD model based on hybrid machine learning and neural network models. Aiming at the problem of high dimensionality and imbalance in the amount of sample data in the massive data of industrial control systems, this paper proposes an IG-based feature selection method and an oversampling method for SMOTE. In the ETM-TBD model, we propose a hyperparameter optimization method based on Bayesian optimization used to optimize the parameters of the four basic machine learners in the model. By introducing a multi-head-attention mechanism, the Transformer module increases the attention between local features and global features, enabling the discovery of the internal relationship between features. Additionally, the BiGRU is used to preserve the temporal features of the dataset, while the DNN is used to extract deeper features. Finally, the SoftMax classifier is used to classify the output. By analyzing the results of the comparison and ablation experiments, it can be concluded that the F1-score of the ETM-TBD model on a robotic arm dataset is 0.9665 and the model has very low FNR and FPR scores of 0.0263 and 0.0081, respectively. It can be seen that the model in this paper is better than the traditional single machine learning algorithm as well as the algorithm lacking any of the modules.

Flood susceptibility mapping using AutoML and a deep learning framework with evolutionary algorithms for hyperparameter optimization

Flooding is one of the most common natural hazards that have extremely detrimental consequences. Understanding which areas are vulnerable to flooding is crucial to addressing these effects. In this work, we use machine learning models and Automated machine learning (AutoML) systems for flood susceptibility mapping in Kerala, India. In particular, we used a three-dimensional convolutional neural network (CNN) architecture for this purpose. The CNN model was assisted with hyperparameter optimization techniques that combine Bayesian optimization with evolutionary algorithms like differential evolution and covariance matrix adaptation evolutionary strategies. The performances of all models are compared in terms of cross-entropy loss, accuracy, precision, recall, area under the curve (AUC) and kappa score. The CNN model shows better performance than the AutoML models. Evolutionary algorithm-assisted hyperparameter optimization methods improved the efficiency of the CNN model by 4 and 9 percent in terms of accuracy and by 0.0265 and 0.0497 with reference to the AUC score.

Supervised contrastive representation learning with tree-structured parzen estimator Bayesian optimization for imbalanced tabular data

Imbalanced tabular datasets adversely impact the predictive performance of most supervised learning algorithms as the imbalanced distribution can lead to a bias preferring the majority class. To address this problem, we propose utilizing supervised contrastive representation learning in conjunction with the tree-structured parzen estimator technique for imbalanced tabular data. Drawing on the success of contrastive representation learning in computer vision, we extend its application to the tabular domain. Through the introduction of supervised contrastive learning, we address the limitation of data augmentation methods for tabular data by incorporating label information. This approach enables us to extract hidden information from the tabular data and obtain discriminative representations, which enhances the performance of supervised learning algorithms. Additionally, the hyper-parameter temperature τ of supervised contrastive learning has a decisive influence on the performance and is difficult to tune. We introduce tree-structured parzen estimator, a Bayesian optimization technique, to automatically select the best τ. We evaluate our approach on fifteen real-world public tabular datasets from diverse domains. The results reveal the superiority of tree-structured parzen estimator over other hyper-parameter optimization methods in effectively searching for the optimal value of τ. More importantly, the proposed method outperforms baseline approaches for imbalanced learning, achieving average improvements of 5.1%, 6.0%, 9.0%, and 8.7% across four main evaluation metrics, which validates that the proposed method is well-suited for addressing imbalanced problems in real-world applications.