Hyperparameter Optimization Methods Research Articles

Systematic refinement of surrogate modelling procedure for useful application to building energy problems

Systematic procedures for applying surrogate model development to building energy problems have limited adoption and struggle to incorporate advancements in broader machine learning research. This work demonstrates an iterative approach that encompasses: Establishing clear, consistent baseline performance thresholds. Assembling a comprehensive set of domain-relevant evaluation metrics, including rigorous bounds on error and emphasis on regions of interest within the broader problem space. Characterizing and adapting the problem space using sub-sampling and pre-processing techniques. Accounting for variability, randomness and complexity in the building energy problem definition, data sampling and surrogate model training. Refining the metamodel decision space defining the neural network architecture and training algorithms, and explored by hyperparameter optimization methods. In addition to demonstrating improved performance predicting aggregate annual heating and cooling demands for an illustrative office case, accuracy, bias and bounds on error were all brought within domain-relevant thresholds for net zero regions of interest. Highlights Iterative refinement of surrogate modelling procedure for building energy problems Consideration of complexity and variability in energy data, problem and surrogate Addition of key dimensions to performance evaluation useful for domain application Greatly improved predictive value for net zero building energy regions of interest Systematic shift in hyperparameter tuning towards deep learning configurations

A Novel Approach to Diagnosis: Using Convolutional Neural Networks to Classify Non-Small Cell Lung Cancers on CT Scans

Background: Lung cancer is the leading cause of cancer-related deaths around the world, making early detection vital to treatment and survival. However, lung cancer has variable clinical presentations, so diagnosis, even by trained medical professionals, is challenging and time- consuming. Deep Learning (DL) has proven effective in training big data to generate accurate diagnoses through classification in a timely manner. Aim and Objective: This study uses data augmentation and hyperparameter optimization methods on convolutional neural networks (CNN) to understand the benefits of various architectures in addition to creating an accurate transfer-based tool for lung cancer diagnosis on CT scans. Materials and Methods: We found a dataset composed of chest CT images of patients with non-small cell lung cancers (NSCLC) categorized into adenocarcinoma, large cell carcinoma, and squamous cell carcinoma and a control group of normal chest CT scans. We tested the CNN architectures VGG16, InceptionV3, ResNet50, and EfficientNetB0 for feature extraction and observed each model on a variety of different metrics such as validation accuracy, statistical errors, and learning rate. Each model was trained on the prechosen data set 2 times at 6 and 12 epochs and several evaluation metrics were recorded. Observation and Results: EfficientNetB0, ResNet50, VGG16, InceptionV3 achieved test accuracies of 98.92%, 91.40%, 77.42%, 70.97% respectively when implemented with hyperparameter tuning and dropout layers. EfficientNetB0 also exceeded the other architectures in metrics such as precision, recall, and F1-score. Conclusion: Convolutional neural networks using EfficientNetB0 yielded a higher accuracy in diagnosis of non-small cell cancer on CT scan which encourages further research in developing robust diagnostic tools using DL to expedite diagnosis, especially in locations with inadequate healthcare resources.

Predicting place of delivery choice among childbearing women in East Africa: a comparative analysis of advanced machine learning techniques.

Sub-Saharan Africa faces high neonatal and maternal mortality rates due to limited access to skilled healthcare during delivery. This study aims to improve the classification of health facilities and home deliveries using advanced machine learning techniques and to explore factors influencing women's choices of delivery locations in East Africa. The study focused on 86,009 childbearing women in East Africa. A comparative analysis of 12 advanced machine learning algorithms was conducted, utilizing various data balancing techniques and hyperparameter optimization methods to enhance model performance. The prevalence of health facility delivery in East Africa was found to be 83.71%. The findings showed that the support vector machine (SVM) algorithm and CatBoost performed best in predicting the place of delivery, in which both of those algorithms scored an accuracy of 95% and an AUC of 0.98 after optimized with Bayesian optimization tuning and insignificant difference between them in all comprehensive analysis of metrics performance. Factors associated with facility-based deliveries were identified using association rule mining, including parental education levels, timing of initial antenatal care (ANC) check-ups, wealth status, marital status, mobile phone ownership, religious affiliation, media accessibility, and birth order. This study underscores the vital role of machine learning algorithms in predicting health facility deliveries. A slight decline in facility deliveries from previous reports highlights the urgent need for targeted interventions to meet Sustainable Development Goals (SDGs), particularly in maternal health. The study recommends promoting facility-based deliveries. These include raising awareness about skilled birth attendance, encouraging early ANC check-up, addressing financial barriers through targeted support programs, implementing culturally sensitive interventions, utilizing media campaigns, and mobile health initiatives. Design specific interventions tailored to the birth order of the child, recognizing that mothers may have different informational needs depending on whether it is their first or subsequent delivery. Furthermore, we recommended researchers to explore a variety of techniques and validate findings using more recent data.

Intelligent Fault Diagnosis of Inter-Turn Short Circuit Faults in PMSMs for Agricultural Machinery Based on Data Fusion and Bayesian Optimization

The permanent magnet synchronous motor (PMSM) plays an important role in the power system of agricultural machinery. Inter-turn short circuit (ITSC) faults are among the most common failures in PMSMs, and early diagnosis of these faults is crucial for enhancing the safety and reliability of motor operation. In this article, a multi-source data-fusion algorithm based on convolutional neural networks (CNNs) has been proposed for the early fault diagnosis of ITSCs. The contributions of this paper can be summarized in three main aspects. Firstly, synchronizing data from different signals extracted by different devices presents a significant challenge. To address this, a signal synchronization method based on maximum cross-correlation is proposed to construct a synchronized dataset of current and vibration signals. Secondly, applying a traditional CNN to the data fusion of different signals is challenging. To solve this problem, a multi-stream high-level feature fusion algorithm based on a channel attention mechanism is proposed. Thirdly, to tackle the issue of hyperparameter tuning in deep learning models, a hyperparameter optimization method based on Bayesian optimization is proposed. Experiments are conducted based on the derived early-stage ITSC fault-severity indicator, validating the effectiveness of the proposed fault-diagnosis algorithm.

Precise Lung Cancer Prediction using ResNet – 50 Deep Neural Network Architecture

The fact that lung cancer continues to be the leading cause of cancer-related death around the world emphasizes how important it is to improve diagnostic methods. Using computed tomography (CT) images and deep learning techniques, the goal of this study is to improve the classification of lung cancer. EfficientNetB1 and Inception V3 are two well-known convolutional neural network (CNN) architectures that we compare the performance of our modified ResNet50 architecture against in order to determine how well it performs in the classification of lung nodules. Analyzing the effects of various preprocessing and hyperparameter optimization methods on model performance is one of our research objectives. Another is to determine how well these models improve diagnostic accuracy. An extensive collection of CT images with annotated lung nodule classifications make up the utilized dataset. To ensure accurate model training and improve image quality, a rigorous preprocessing pipeline is used. Using the Keras Sequential framework, the models are trained with optimal dropout rates and L2 regularization to prevent overfitting. Metrics like accuracy, loss, and confusion matrices are used to evaluate model performance. A comprehensive evaluation of the model's sensitivity and specificity across various thresholds is also provided by means of the Free-Response Receiver Operating Characteristic (FROC) curve and Area Under the Curve (AUC) values. The adjusted ResNet50 model showed prevalent order exactness, accomplishing a precision of 98.1% and an AUC of 0.97, in this way beating different models in the review. EfficientNetB1 had an accuracy of 96.4 percent and an AUC of 0.94, while Inception V3 had an accuracy of 95.8 percent and an AUC of 0.93, as a comparison. Based on these findings, it appears that the accuracy of lung cancer detection from CT images can be significantly improved by combining specialized preprocessing and training methods with advanced CNN architectures. With potential implications for clinical practice and future research directions, this study offers a promising strategy for increasing lung cancer diagnostic accuracy.

Multi-head attention-based variational autoencoders ensemble for remaining useful life prediction of aero-engines

Abstract Accurate remaining useful life (RUL) prediction of aero-engines through condition monitoring (CM) data is of great significance for flight reliability and safety. Although deep learning (DL)-based approaches have been widely considered, individual DL models suffer from significant stochasticity and limited generalizability when predicting the RUL. To solve this issue, a novel multi-head attention-based variational autoencoders (MHAT-VAEs) ensemble model is proposed. Two distinct MHAT-VAEs are designed, employing linear and convolutional operations to capture global and temporal compressed representations of the CM data. Additionally, a dual-level ensemble strategy is introduced to adaptively fuse the outputs of the two base learners. A hyperparameter optimization method is also implemented to further enhance the efficiency and performance of the base learners. The effectiveness of the proposed method is validated using the C-MAPSS and N-CMAPSS datasets, with experimental results showing that it outperforms state-of-the-art approaches.

Deep Learning Enhanced CNN with Bio-Inspired Techniques and BCE For Effective Lung Nodules Detection & Classification For Accurate Diagnosis

Objectives: To propose a new hybrid deep learning model to boost lung nodule detection and classification in combination with a bioinspired method for efficient hyper-parameter optimization to maximize true positive and nodule discrimination rates. Methods: In order to extract the intrinsic and complex features from different lung nodules, deep learning-based enhanced CNN (ECNN) is used. Segmentation masks are generated using Mask-RCNN to isolate and capture the ROIs, which serve as input to CNN. Prior to segmentation and extraction, data cleaning, transformation, and augmentation is done. Hyper-parameter optimization is done using a bio-inspired differential evolution method, which helps to identify the learning rate and number of layers to achieve optimal performance. We utilize BCE to quantify the performance of classification tasks. A large-scale CT-DICOM image dataset with 25,1135 images is used for this research work, collected from the cancer imaging archive, which has a clinically proven record of 355 instances with detailed image analysis, tumor location, and bounding boxes with a resolution of 512x512 pixels. 175794 images are used for training, and 75341 images are used for testing and validation. The performance of the new system is assessed using MATLAB, where results are compared with existing models such as SVM-WSS, GCPSO-PNN, and 3D-DLCNN models. Findings: The newly suggested ECNN with DE Bio-inspired model boosts the performance of lung nodule detection and classification with 94.7% accuracy, 93.8% sensitivity, 94.5% specificity, 93.4% F1 score, 92.4% dice coefficient, 0.1 Log Loss and AUC-ROC with 0.93 TPR and 0.07 FPR. Novelty: The novel method presents an advanced computational model using deep learning and bio-inspired algorithms for robust classification of lung nodules, which significantly improves the early diagnosis. This CAD model overcomes the limitations of the existing approaches SVM-WSS, GCPSO-PNN, and 3D-DLCNN in terms of segmentation, classification, error detection, and hyper-parameter tuning. Keywords: Lung Nodules, Deep Learning, Disease Classification, Image Processing, ECNN- DE Classifier, CT-DICOM Dataset

Deep neural network based distribution system state estimation using hyperparameter optimization

In the past decade, distribution system state estimation has become a crucial topic in power system research due to the increasing importance of distribution networks amidst the decline of centralized energy production. This paper addresses a gap in the literature regarding the application of modern hyperparameter optimization techniques in low-voltage distribution system state estimation using deep neural networks. In particular, it demonstrates the use of the Tree-structured Parzen Estimator algorithm, which is a Bayesian hyperparameter optimization method, for distribution system state estimation on real Hungarian low-voltage networks. The study uses data from four real-life low-voltage supply areas in Hungary, which were modeled to address the challenges in obtaining network information. Compared to traditional methods like the weighted least squares method, the Tree-structured Parzen Estimator algorithm significantly improves the accuracy of the voltage amplitude and angle estimations, reducing the relative error by 14–73%. Additionally, it is shown that TPE outperforms simpler methods like Random Search in hyperparameter optimization. The results also reveal connections between the distribution system size and optimal hyperparameters, such as batch size, learning rate, and hidden layer configuration. The proposed non-iterative algorithm, combined with the parallel computation capabilities of deep neural networks utilizing GPU, resulted in four orders of magnitude improvement in runtime. These advancements make the proposed approach a valuable tool for renewable energy integration planning and real-time monitoring, highlighting its potential for practical applications in the power industry.

A novel systematically optimized tabular neural network (TabNet) algorithm for predicting the tensile modulus of additively manufactured PLA parts

Assessing the elastic modulus of 3D-printed polylactic acid (PLA) components is essential for understanding their stiffness and load capacity, which are crucial for predicting product performance and durability. In this study, the predictive accuracy of a Tabular Neural Network (TabNet) algorithm for determining the elastic modulus of 3D-printed PLA components via fused deposition modeling (FDM) was investigated. Utilizing a comprehensive dataset of 128 literature-sourced data points, divided into 80 % for training and 20 % for validation, the study proposed a new Taguchi-based method for efficient hyperparameter optimization of the TabNet algorithm. This optimization revealed that a configuration of 8 decision blocks, 16 attention blocks, and 5 decision steps, along with the "Adam" optimizer, a gamma of 1, learning rate of 0.1, and lambda-sparse of 0.01, yielded the highest prediction accuracy for the elastic modulus of PLA parts. The performance of the optimized TabNet model was evaluated using R-squared (R²), Mean Absolute Error (MAE), Mean Squared Error (MSE), and Root Mean Squared Error (RMSE) measures. The findings highlighted an R² of 96.855 %, an MAE of 0.158, an MSE of 0.037, and an RMSE of 0.193 in the validation dataset, demonstrating substantial predictive reliability. To further test the model’s robustness, fourteen unseen data points were analyzed. The observed discrepancies between predicted and actual values were under 10 %, affirming the Taguchi-optimized TabNet algorithm’s effectiveness in forecasting the elastic modulus of FDM 3D-printed PLA components. This investigation provides a significant advancement in additive manufacturing, introducing a precise and reliable method for predicting the mechanical properties of 3D-printed materials.

Hyper-Parameter Optimization-based multi-source fusion for remote sensing inversion of non-photosensitive water quality parameters

ABSTRACT The constraints of spatiotemporal heterogeneity and spatial resolution constitute two crucial challenges in the establishment of remote sensing inversion models. Spatiotemporal heterogeneity gives rise to an inadequate generalization capacity of remote sensing models, demanding extensive manual parameter adjustment for each model construction. This not only escalates the task’s work intensity but also leads to unstable performance. The limited spatial resolution of remote sensing images leads to suboptimal inversion accuracy for sampling points influenced by mixed pixel effects. To tackle these problems, we take the case of non-photosensitive water quality parameter inversion in the narrow rivers of Longnan area. By integrating advanced Hyper-Parameter Optimization (HPO) techniques, such as Optuna from machine learning, an inversion model was developed, incorporating the bands of Sentinel-2 and Sentinel-3 as model features. Among these features, bands with lower spatial resolution are employed to furnish surrounding information, thereby enhancing the inversion accuracy. The research outcomes demonstrate that: 1) The model constructed based on the HPO method, Optuna, attained favourable inversion results, with R2 values of 0.68, 0.77, 0.35, and 0.60 for Total Nitrogen (TN), Total Phosphorus (TP), Ammonia Nitrogen (NH3-N), and Chemical Oxygen Demand (COD), respectively. 2) The fusion of Sentinel-2 and Sentinel-3 data enhanced the inversion accuracy compared to using them separately, highlighting the considerable significance of multi-source data fusion methods in improving inversion accuracy. This research fills a void in the remote sensing inversion domain and lays the groundwork for future endeavours.

A fast smoothing newton method for bilevel hyperparameter optimization for SVC with logistic loss

Support vector classification (SVC) with logistic loss has excellent theoretical properties in classification problems where the label values are not continuous. In this paper, we reformulate the hyperparameter selection for SVC with logistic loss as a bilevel optimization problem in which the upper-level problem and the lower-level problem are both based on logistic loss. The resulting bilevel optimization model is converted to a single-level nonlinear programming (NLP) based on the KKT conditions of the lower-level problem. Such NLP contains a set of nonlinear equality constraints and a simple lower-bound constraint. The second-order sufficient condition is characterized, which guarantees that the strict local optimizers are obtained. To solve such NLP, we apply the smoothing Newton method proposed in [Liang L, Sun D., Toh KC. A squared smoothing Newton method for semidefinite programming, 2023] to solve the KKT conditions, which contain one pair of complementarity constraints. We show that the smoothing Newton method has a superlinear convergence rate. Extensive numerical results verify the efficiency of the proposed approach and strict local minimizers can be achieved both numerically and theoretically. In particular, compared with other methods, our algorithm can achieve competitive results while consuming less time than other methods.

Constrained Density-Based Spatial Clustering of Applications with Noise (DBSCAN) using hyperparameter optimization

This article proposes a hyperparameter optimization method for density-based spatial clustering of applications with noise (DBSCAN) with constraints, termed HC-DBSCAN. While DBSCAN is effective at creating non-convex clusters, it cannot limit the number of clusters. This limitation is difficult to address with simple adjustments or heuristic methods. We approach constrained DBSCAN as an optimization problem and solve it using a customized alternating direction method of multipliers Bayesian optimization (ADMMBO). Our custom ADMMBO enables HC-DBSCAN to reuse clustering results for enhanced computational efficiency, handle integer-valued parameters, and incorporate constraint functions that account for the degree of violations to improve clustering performance. Furthermore, we propose an evaluation metric, penalized Davies–Bouldin score, with a computational cost of O(N). This metric aims to mitigate the high computational cost associated with existing metrics and efficiently manage noise instances in DBSCAN. Numerical experiments demonstrate that HC-DBSCAN, equipped with the proposed metric, generates high-quality non-convex clusters and outperforms benchmark methods on both simulated and real datasets.

An improved chaotic GWO-LGBM hybrid algorithm for emotion recognition

To address the challenges associated with the unbalanced search abilities and low search accuracy of the Gray Wolf Optimization (GWO) algorithm, we propose an enhanced version termed the Improved Chaotic Gray Wolf Optimization Algorithm (ICGWO) based on cubic mapping. Initially, the population positions of gray wolves were perturbed to expedite the convergence speed. Subsequently, a nonlinear convergence factor was introduced to enhance both search ability and efficiency. Finally, within the algorithm, a dynamic inertia-weighting strategy was implemented to improve convergence speed and solution accuracy. Experimental simulations encompassing 23 classical test functions demonstrate that the ICGWO algorithm exhibits superior convergence speed, solution accuracy, and stability compared to other swarm intelligence optimization algorithms and certain enhanced GWO algorithms. In comparison to GWO, utilizing the ICGWO method for hyperparameter optimization within the Light Gradient Boosting Machine (LGBM) for emotional recognition led to notable enhancements in average accuracy and F1 score by 7.29 % and 3.92 % respectively for valence, and 4.33 % and 3.23 % respectively for arousal. Additionally, the standard deviation of the ICGWO-LGBM was less than one-tenth of the standard deviation of GWO-LGBM in both conditions. These results underscore the strong optimization capabilities and high stability of the ICGWO algorithm.

Towards Optimising the Derivation of Phenological Phases of Different Crop Types over Germany Using Satellite Image Time Series

Operational crop monitoring applications, including crop type mapping, condition monitoring, and yield estimation, would benefit from the ability to robustly detect and map crop phenology measures related to the crop calendar and management activities like emergence, stem elongation, and harvest timing. However, this has proven to be challenging due to two main issues: first, the lack of optimised approaches for accurate crop phenology retrievals, and second, the cloud cover during the crop growth period, which hampers the use of optical data. Hence, in the current study, we outline a novel calibration procedure that optimises the settings to produce high-quality NDVI time series as well as the thresholds for retrieving the start of the season (SOS) and end of the season (EOS) of different crops, making them more comparable and related to ground crop phenological measures. As a first step, we introduce a new method, termed UE-WS, to reconstruct high-quality NDVI time series data by integrating a robust upper envelope detection technique with the Whittaker smoothing filter. The experimental results demonstrate that the new method can achieve satisfactory performance in reducing noise in the original NDVI time series and producing high-quality NDVI profiles. As a second step, a threshold optimisation approach was carried out for each phenophase of three crops (winter wheat, corn, and sugarbeet) using an optimisation framework, primarily leveraging the state-of-the-art hyperparameter optimization method (Optuna) by first narrowing down the search space for the threshold parameter and then applying a grid search to pinpoint the optimal value within this refined range. This process focused on minimising the error between the satellite-derived and observed days of the year (DOY) based on data from the German Meteorological Service (DWD) covering two years (2019–2020) and three federal states in Germany. The results of the calculation of the median of the temporal difference between the DOY observations of DWD phenology held out from a separate year (2021) and those derived from satellite data reveal that it typically ranged within ±10 days for almost all phenological phases. The validation results of the detection of dates of phenological phases against separate field-based phenological observations resulted in an RMSE of less than 10 days and an R-squared value of approximately 0.9 or greater. The findings demonstrate how optimising the thresholds required for deriving crop-specific phenophases using high-quality NDVI time series data could produce timely and spatially explicit phenological information at the field and crop levels.

Ada-XG-CatBoost: A Combined Forecasting Model for Gross Ecosystem Product (GEP) Prediction

The degradation of the ecosystem and the loss of natural capital have seriously threatened the sustainable development of human society and economy. Currently, most research on Gross Ecosystem Product (GEP) is based on statistical modeling methods, which face challenges such as high modeling difficulty, high costs, and inaccurate quantitative methods. However, machine learning models are characterized by high efficiency, fewer parameters, and higher accuracy. Despite these advantages, their application in GEP research is not widespread, particularly in the area of combined machine learning models. This paper includes both a GEP combination model and an explanatory analysis model. This paper is the first to propose a combined GEP prediction model called Ada-XGBoost-CatBoost (Ada-XG-CatBoost), which integrates the Extreme Gradient Boosting (XGBoost), Categorical Boosting (CatBoost) algorithms, and SHapley Additive exPlanations (SHAP) model. This approach overcomes the limitations of single-model evaluations and aims to address the current issues of inaccurate and incomplete GEP assessments. It provides new guidance and methods for enhancing the value of ecosystem services and achieving regional sustainable development. Based on the actual ecological data of a national city, data preprocessing and feature correlation analysis are carried out using XGBoost and CatBoost algorithms, AdaGrad optimization algorithm, and the Bayesian hyperparameter optimization method. By selecting the 11 factors that predominantly influence GEP, training the model using these selected feature datasets, and optimizing the Bayesian parameters, the error gradient is then updated to adjust the weights, achieving a combination model that minimizes errors. This approach reduces the risk of overfitting in individual models and enhances the predictive accuracy and interpretability of the model. The results indicate that the mean squared error (MSE) of the Ada-XG-CatBoost model is reduced by 65% and 70% compared to the XGBoost and CatBoost, respectively. Additionally, the mean absolute error (MAE) is reduced by 4.1% and 42.6%, respectively. Overall, the Ada-XG-CatBoost combination model has a more accurate and stable predictive performance, providing a more accurate, efficient, and reliable reference for the sustainable development of the ecological industry.

Physics-based natural gradient boosting probabilistic prediction model for seismic failure modes of reinforced concrete columns

A novel physics-based natural gradient boosting (PNGB) probabilistic prediction model for flexure failure (FF), flexure-shear failure (FSF) and shear failure (SF) of reinforced concrete (RC) columns under strong earthquakes has been developed to address the limitations of empirical and machine learning (ML) models, which often fail to learn physical laws, insufficiently consider uncertainties, and exhibit poor generalization performance. Firstly, an improved natural gradient boosting (NGB) probabilistic model of seismic failure modes was created by using natural gradient boosting and gradient boosting regressor algorithms. Then a new hyperparameter optimization method for the improved NGB probabilistic model was proposed based on the ML algorithms and the physical laws. Finally, the predictive performance and engineering practicability of the PNGB probabilistic model were verified compared to empirical and ML models. According to the analysis results, the PNGB model can effectively improve the predictive recall of FF, FSF and SF by about 19–65 %, 20–67 % and 20–56 % respectively compared with empirical models, and by about 1–7 %, 7–43 % and 6–32 % respectively compared with ML models. Moreover, the PNGB model exhibits satisfactory generalization performance and minimal dispersion, which can be used to calibrate the ML models based on the confidence intervals. Furthermore, the result predicted by the PNGB model satisfies the physical laws relating to the seismic failure modes and design parameters, which can further represent the competitive relationships of various seismic failure modes.

Hyperparameter optimization of two-branch neural networks in multi-target prediction

As a result of the ever increasing complexity of configuring and fine-tuning machine learning models, the field of automated machine learning (AutoML) has emerged over the past decade. However, software implementations like Auto-WEKA and Auto-sklearn typically focus on classical machine learning (ML) tasks such as classification and regression. Our work can be seen as the first attempt at offering a single AutoML framework for most problem settings that fall under the umbrella of multi-target prediction, which includes popular ML settings such as multi-label classification, multivariate regression, multi-task learning, dyadic prediction, matrix completion, and zero-shot learning. Automated problem selection and model configuration are achieved by extending DeepMTP, a general deep learning framework for MTP problem settings, with popular hyperparameter optimization (HPO) methods. Our extensive benchmarking across different datasets and MTP problem settings identifies cases where specific HPO methods outperform others.

A Parkinson's Auxiliary Diagnosis Algorithm Based on a Hyperparameter Optimization Method of Deep Learning.

Parkinson's disease is a common mental disease in the world, especially in the middle-aged and elderly groups. Today, clinical diagnosis is the main diagnostic method of Parkinson's disease, but the diagnosis results are not ideal, especially in the early stage of the disease. In this paper, a Parkinson's auxiliary diagnosis algorithm based on a hyperparameter optimization method of deep learning is proposed for the Parkinson's diagnosis. The diagnosis system uses ResNet50 to achieve feature extraction and Parkinson's classification, mainly including speech signal processing part, algorithm improvement part based on Artificial Bee Colony algorithm (ABC) and optimizing the hyperparameters of ResNet50 part. The improved algorithm is called Gbest Dimension Artificial Bee Colony algorithm (GDABC), proposing "Range pruning strategy" which aims at narrowing the scope of search and "Dimension adjustment strategy" which is to adjust gbest dimension by dimension. The accuracy of the diagnosis system in the verification set of Mobile Device Voice Recordings at King's College London (MDVR-CKL) dataset can reach more than 96%. Compared with current Parkinson's sound diagnosis methods and other optimization algorithms, our auxiliary diagnosis system shows better classification performance on the dataset within limited time and resources.

Enhancing the Performance and Accuracy in Real-Time Football and Player Detection Using Upgraded YOLOv5 Architecture

The study presents a significantly improved version of the YOLOv5 real-time object detection model for football player recognition. The proposed technique includes feature-tuning and hyper-parameter optimization methods that have been carefully selected to enhance both speed and accuracy, resulting in a superior real-time performance of the YOLOv5 architecture. Furthermore, the YOLOv5 model incorporates a SimSPPF module that enables multi-scale feature extraction with less computational power, making it a highly efficient and effective solution. We selected the GhostNet module to reduce complexity and the Slim scale detection layer for precise bounding box prediction. Our tests, conducted with recordings of multiple football matches, demonstrate that our model accurately detects both the football and players even in complex scenarios with occlusions and dynamic illumination. The suggested method outperforms the original YOLOv5n model in terms of precision, recall, and mean average precision at 0.5 IoU. It is also more computationally efficient. This method has potential applications in live broadcasting, player monitoring, and sports analytics. The upgraded YOLOv5 model demonstrates superior accuracy and efficiency compared to previous methods that rely on traditional image processing techniques or two-stage detectors. This makes it highly suitable for practical, real-world deployments.

Predicting and analyzing the cementing quality of oil well reservoirs based on Bayesian-random forest model

Cementing quality crucially influences the safety, efficiency, and costs of oil and gas wells. Traditional evaluations rely on empirical and qualitative methods, introducing uncertainty. This study analyzed nearly 2000 wells casing and cementing data from 2016 to 2021 in an eastern China oil field. The original data was analyzed using multiple data cleaning techniques, and effectively dealing with missing values and anomalies. Additionally, the application of feature selection techniques such as random forest successfully identified key elements within the data. Three machine learning models and hyperparameter optimization methods were compared. A Bayesian random forest (BS-RF) model was established with hyperparameter optimization for cementing-quality evaluation. Finally, the relationship between the influencing factors and well-cementing quality were determined by the SHapley Additive exPlanations (SHAP) method. This enabled the evaluation and prediction of well cementing quality. The results indicate that the model achieved an accuracy of 93.21%. When the validation set was used for verification, the predictive accuracy was 95%. This indicates that the model is relatively stable and exhibits strong generalization capabilities across diverse datasets. Performance numeric of cement slurry (SPN), drilling fluid density before cementing, deviation angle, and flushing time are the four key features that influence cementing quality. Each has a SHAP value above 0.12. This article provides guidance for future well-cemented construction schemes.