Hyperparameter Optimization Methods Research Articles

Deep Learning Enhanced CNN with Bio-Inspired Techniques and BCE For Effective Lung Nodules Detection & Classification For Accurate Diagnosis

Objectives: To propose a new hybrid deep learning model to boost lung nodule detection and classification in combination with a bioinspired method for efficient hyper-parameter optimization to maximize true positive and nodule discrimination rates. Methods: In order to extract the intrinsic and complex features from different lung nodules, deep learning-based enhanced CNN (ECNN) is used. Segmentation masks are generated using Mask-RCNN to isolate and capture the ROIs, which serve as input to CNN. Prior to segmentation and extraction, data cleaning, transformation, and augmentation is done. Hyper-parameter optimization is done using a bio-inspired differential evolution method, which helps to identify the learning rate and number of layers to achieve optimal performance. We utilize BCE to quantify the performance of classification tasks. A large-scale CT-DICOM image dataset with 25,1135 images is used for this research work, collected from the cancer imaging archive, which has a clinically proven record of 355 instances with detailed image analysis, tumor location, and bounding boxes with a resolution of 512x512 pixels. 175794 images are used for training, and 75341 images are used for testing and validation. The performance of the new system is assessed using MATLAB, where results are compared with existing models such as SVM-WSS, GCPSO-PNN, and 3D-DLCNN models. Findings: The newly suggested ECNN with DE Bio-inspired model boosts the performance of lung nodule detection and classification with 94.7% accuracy, 93.8% sensitivity, 94.5% specificity, 93.4% F1 score, 92.4% dice coefficient, 0.1 Log Loss and AUC-ROC with 0.93 TPR and 0.07 FPR. Novelty: The novel method presents an advanced computational model using deep learning and bio-inspired algorithms for robust classification of lung nodules, which significantly improves the early diagnosis. This CAD model overcomes the limitations of the existing approaches SVM-WSS, GCPSO-PNN, and 3D-DLCNN in terms of segmentation, classification, error detection, and hyper-parameter tuning. Keywords: Lung Nodules, Deep Learning, Disease Classification, Image Processing, ECNN- DE Classifier, CT-DICOM Dataset

Deep neural network based distribution system state estimation using hyperparameter optimization

In the past decade, distribution system state estimation has become a crucial topic in power system research due to the increasing importance of distribution networks amidst the decline of centralized energy production. This paper addresses a gap in the literature regarding the application of modern hyperparameter optimization techniques in low-voltage distribution system state estimation using deep neural networks. In particular, it demonstrates the use of the Tree-structured Parzen Estimator algorithm, which is a Bayesian hyperparameter optimization method, for distribution system state estimation on real Hungarian low-voltage networks. The study uses data from four real-life low-voltage supply areas in Hungary, which were modeled to address the challenges in obtaining network information. Compared to traditional methods like the weighted least squares method, the Tree-structured Parzen Estimator algorithm significantly improves the accuracy of the voltage amplitude and angle estimations, reducing the relative error by 14–73%. Additionally, it is shown that TPE outperforms simpler methods like Random Search in hyperparameter optimization. The results also reveal connections between the distribution system size and optimal hyperparameters, such as batch size, learning rate, and hidden layer configuration. The proposed non-iterative algorithm, combined with the parallel computation capabilities of deep neural networks utilizing GPU, resulted in four orders of magnitude improvement in runtime. These advancements make the proposed approach a valuable tool for renewable energy integration planning and real-time monitoring, highlighting its potential for practical applications in the power industry.

A novel systematically optimized tabular neural network (TabNet) algorithm for predicting the tensile modulus of additively manufactured PLA parts

Assessing the elastic modulus of 3D-printed polylactic acid (PLA) components is essential for understanding their stiffness and load capacity, which are crucial for predicting product performance and durability. In this study, the predictive accuracy of a Tabular Neural Network (TabNet) algorithm for determining the elastic modulus of 3D-printed PLA components via fused deposition modeling (FDM) was investigated. Utilizing a comprehensive dataset of 128 literature-sourced data points, divided into 80 % for training and 20 % for validation, the study proposed a new Taguchi-based method for efficient hyperparameter optimization of the TabNet algorithm. This optimization revealed that a configuration of 8 decision blocks, 16 attention blocks, and 5 decision steps, along with the "Adam" optimizer, a gamma of 1, learning rate of 0.1, and lambda-sparse of 0.01, yielded the highest prediction accuracy for the elastic modulus of PLA parts. The performance of the optimized TabNet model was evaluated using R-squared (R²), Mean Absolute Error (MAE), Mean Squared Error (MSE), and Root Mean Squared Error (RMSE) measures. The findings highlighted an R² of 96.855 %, an MAE of 0.158, an MSE of 0.037, and an RMSE of 0.193 in the validation dataset, demonstrating substantial predictive reliability. To further test the model’s robustness, fourteen unseen data points were analyzed. The observed discrepancies between predicted and actual values were under 10 %, affirming the Taguchi-optimized TabNet algorithm’s effectiveness in forecasting the elastic modulus of FDM 3D-printed PLA components. This investigation provides a significant advancement in additive manufacturing, introducing a precise and reliable method for predicting the mechanical properties of 3D-printed materials.

Hyper-Parameter Optimization-based multi-source fusion for remote sensing inversion of non-photosensitive water quality parameters

ABSTRACT The constraints of spatiotemporal heterogeneity and spatial resolution constitute two crucial challenges in the establishment of remote sensing inversion models. Spatiotemporal heterogeneity gives rise to an inadequate generalization capacity of remote sensing models, demanding extensive manual parameter adjustment for each model construction. This not only escalates the task’s work intensity but also leads to unstable performance. The limited spatial resolution of remote sensing images leads to suboptimal inversion accuracy for sampling points influenced by mixed pixel effects. To tackle these problems, we take the case of non-photosensitive water quality parameter inversion in the narrow rivers of Longnan area. By integrating advanced Hyper-Parameter Optimization (HPO) techniques, such as Optuna from machine learning, an inversion model was developed, incorporating the bands of Sentinel-2 and Sentinel-3 as model features. Among these features, bands with lower spatial resolution are employed to furnish surrounding information, thereby enhancing the inversion accuracy. The research outcomes demonstrate that: 1) The model constructed based on the HPO method, Optuna, attained favourable inversion results, with R2 values of 0.68, 0.77, 0.35, and 0.60 for Total Nitrogen (TN), Total Phosphorus (TP), Ammonia Nitrogen (NH3-N), and Chemical Oxygen Demand (COD), respectively. 2) The fusion of Sentinel-2 and Sentinel-3 data enhanced the inversion accuracy compared to using them separately, highlighting the considerable significance of multi-source data fusion methods in improving inversion accuracy. This research fills a void in the remote sensing inversion domain and lays the groundwork for future endeavours.

A fast smoothing newton method for bilevel hyperparameter optimization for SVC with logistic loss

Support vector classification (SVC) with logistic loss has excellent theoretical properties in classification problems where the label values are not continuous. In this paper, we reformulate the hyperparameter selection for SVC with logistic loss as a bilevel optimization problem in which the upper-level problem and the lower-level problem are both based on logistic loss. The resulting bilevel optimization model is converted to a single-level nonlinear programming (NLP) based on the KKT conditions of the lower-level problem. Such NLP contains a set of nonlinear equality constraints and a simple lower-bound constraint. The second-order sufficient condition is characterized, which guarantees that the strict local optimizers are obtained. To solve such NLP, we apply the smoothing Newton method proposed in [Liang L, Sun D., Toh KC. A squared smoothing Newton method for semidefinite programming, 2023] to solve the KKT conditions, which contain one pair of complementarity constraints. We show that the smoothing Newton method has a superlinear convergence rate. Extensive numerical results verify the efficiency of the proposed approach and strict local minimizers can be achieved both numerically and theoretically. In particular, compared with other methods, our algorithm can achieve competitive results while consuming less time than other methods.

Constrained Density-Based Spatial Clustering of Applications with Noise (DBSCAN) using hyperparameter optimization

This article proposes a hyperparameter optimization method for density-based spatial clustering of applications with noise (DBSCAN) with constraints, termed HC-DBSCAN. While DBSCAN is effective at creating non-convex clusters, it cannot limit the number of clusters. This limitation is difficult to address with simple adjustments or heuristic methods. We approach constrained DBSCAN as an optimization problem and solve it using a customized alternating direction method of multipliers Bayesian optimization (ADMMBO). Our custom ADMMBO enables HC-DBSCAN to reuse clustering results for enhanced computational efficiency, handle integer-valued parameters, and incorporate constraint functions that account for the degree of violations to improve clustering performance. Furthermore, we propose an evaluation metric, penalized Davies–Bouldin score, with a computational cost of O(N). This metric aims to mitigate the high computational cost associated with existing metrics and efficiently manage noise instances in DBSCAN. Numerical experiments demonstrate that HC-DBSCAN, equipped with the proposed metric, generates high-quality non-convex clusters and outperforms benchmark methods on both simulated and real datasets.

An improved chaotic GWO-LGBM hybrid algorithm for emotion recognition

To address the challenges associated with the unbalanced search abilities and low search accuracy of the Gray Wolf Optimization (GWO) algorithm, we propose an enhanced version termed the Improved Chaotic Gray Wolf Optimization Algorithm (ICGWO) based on cubic mapping. Initially, the population positions of gray wolves were perturbed to expedite the convergence speed. Subsequently, a nonlinear convergence factor was introduced to enhance both search ability and efficiency. Finally, within the algorithm, a dynamic inertia-weighting strategy was implemented to improve convergence speed and solution accuracy. Experimental simulations encompassing 23 classical test functions demonstrate that the ICGWO algorithm exhibits superior convergence speed, solution accuracy, and stability compared to other swarm intelligence optimization algorithms and certain enhanced GWO algorithms. In comparison to GWO, utilizing the ICGWO method for hyperparameter optimization within the Light Gradient Boosting Machine (LGBM) for emotional recognition led to notable enhancements in average accuracy and F1 score by 7.29 % and 3.92 % respectively for valence, and 4.33 % and 3.23 % respectively for arousal. Additionally, the standard deviation of the ICGWO-LGBM was less than one-tenth of the standard deviation of GWO-LGBM in both conditions. These results underscore the strong optimization capabilities and high stability of the ICGWO algorithm.

Towards Optimising the Derivation of Phenological Phases of Different Crop Types over Germany Using Satellite Image Time Series

Operational crop monitoring applications, including crop type mapping, condition monitoring, and yield estimation, would benefit from the ability to robustly detect and map crop phenology measures related to the crop calendar and management activities like emergence, stem elongation, and harvest timing. However, this has proven to be challenging due to two main issues: first, the lack of optimised approaches for accurate crop phenology retrievals, and second, the cloud cover during the crop growth period, which hampers the use of optical data. Hence, in the current study, we outline a novel calibration procedure that optimises the settings to produce high-quality NDVI time series as well as the thresholds for retrieving the start of the season (SOS) and end of the season (EOS) of different crops, making them more comparable and related to ground crop phenological measures. As a first step, we introduce a new method, termed UE-WS, to reconstruct high-quality NDVI time series data by integrating a robust upper envelope detection technique with the Whittaker smoothing filter. The experimental results demonstrate that the new method can achieve satisfactory performance in reducing noise in the original NDVI time series and producing high-quality NDVI profiles. As a second step, a threshold optimisation approach was carried out for each phenophase of three crops (winter wheat, corn, and sugarbeet) using an optimisation framework, primarily leveraging the state-of-the-art hyperparameter optimization method (Optuna) by first narrowing down the search space for the threshold parameter and then applying a grid search to pinpoint the optimal value within this refined range. This process focused on minimising the error between the satellite-derived and observed days of the year (DOY) based on data from the German Meteorological Service (DWD) covering two years (2019–2020) and three federal states in Germany. The results of the calculation of the median of the temporal difference between the DOY observations of DWD phenology held out from a separate year (2021) and those derived from satellite data reveal that it typically ranged within ±10 days for almost all phenological phases. The validation results of the detection of dates of phenological phases against separate field-based phenological observations resulted in an RMSE of less than 10 days and an R-squared value of approximately 0.9 or greater. The findings demonstrate how optimising the thresholds required for deriving crop-specific phenophases using high-quality NDVI time series data could produce timely and spatially explicit phenological information at the field and crop levels.

Ada-XG-CatBoost: A Combined Forecasting Model for Gross Ecosystem Product (GEP) Prediction

The degradation of the ecosystem and the loss of natural capital have seriously threatened the sustainable development of human society and economy. Currently, most research on Gross Ecosystem Product (GEP) is based on statistical modeling methods, which face challenges such as high modeling difficulty, high costs, and inaccurate quantitative methods. However, machine learning models are characterized by high efficiency, fewer parameters, and higher accuracy. Despite these advantages, their application in GEP research is not widespread, particularly in the area of combined machine learning models. This paper includes both a GEP combination model and an explanatory analysis model. This paper is the first to propose a combined GEP prediction model called Ada-XGBoost-CatBoost (Ada-XG-CatBoost), which integrates the Extreme Gradient Boosting (XGBoost), Categorical Boosting (CatBoost) algorithms, and SHapley Additive exPlanations (SHAP) model. This approach overcomes the limitations of single-model evaluations and aims to address the current issues of inaccurate and incomplete GEP assessments. It provides new guidance and methods for enhancing the value of ecosystem services and achieving regional sustainable development. Based on the actual ecological data of a national city, data preprocessing and feature correlation analysis are carried out using XGBoost and CatBoost algorithms, AdaGrad optimization algorithm, and the Bayesian hyperparameter optimization method. By selecting the 11 factors that predominantly influence GEP, training the model using these selected feature datasets, and optimizing the Bayesian parameters, the error gradient is then updated to adjust the weights, achieving a combination model that minimizes errors. This approach reduces the risk of overfitting in individual models and enhances the predictive accuracy and interpretability of the model. The results indicate that the mean squared error (MSE) of the Ada-XG-CatBoost model is reduced by 65% and 70% compared to the XGBoost and CatBoost, respectively. Additionally, the mean absolute error (MAE) is reduced by 4.1% and 42.6%, respectively. Overall, the Ada-XG-CatBoost combination model has a more accurate and stable predictive performance, providing a more accurate, efficient, and reliable reference for the sustainable development of the ecological industry.

Physics-based natural gradient boosting probabilistic prediction model for seismic failure modes of reinforced concrete columns

A novel physics-based natural gradient boosting (PNGB) probabilistic prediction model for flexure failure (FF), flexure-shear failure (FSF) and shear failure (SF) of reinforced concrete (RC) columns under strong earthquakes has been developed to address the limitations of empirical and machine learning (ML) models, which often fail to learn physical laws, insufficiently consider uncertainties, and exhibit poor generalization performance. Firstly, an improved natural gradient boosting (NGB) probabilistic model of seismic failure modes was created by using natural gradient boosting and gradient boosting regressor algorithms. Then a new hyperparameter optimization method for the improved NGB probabilistic model was proposed based on the ML algorithms and the physical laws. Finally, the predictive performance and engineering practicability of the PNGB probabilistic model were verified compared to empirical and ML models. According to the analysis results, the PNGB model can effectively improve the predictive recall of FF, FSF and SF by about 19–65 %, 20–67 % and 20–56 % respectively compared with empirical models, and by about 1–7 %, 7–43 % and 6–32 % respectively compared with ML models. Moreover, the PNGB model exhibits satisfactory generalization performance and minimal dispersion, which can be used to calibrate the ML models based on the confidence intervals. Furthermore, the result predicted by the PNGB model satisfies the physical laws relating to the seismic failure modes and design parameters, which can further represent the competitive relationships of various seismic failure modes.

Hyperparameter optimization of two-branch neural networks in multi-target prediction

As a result of the ever increasing complexity of configuring and fine-tuning machine learning models, the field of automated machine learning (AutoML) has emerged over the past decade. However, software implementations like Auto-WEKA and Auto-sklearn typically focus on classical machine learning (ML) tasks such as classification and regression. Our work can be seen as the first attempt at offering a single AutoML framework for most problem settings that fall under the umbrella of multi-target prediction, which includes popular ML settings such as multi-label classification, multivariate regression, multi-task learning, dyadic prediction, matrix completion, and zero-shot learning. Automated problem selection and model configuration are achieved by extending DeepMTP, a general deep learning framework for MTP problem settings, with popular hyperparameter optimization (HPO) methods. Our extensive benchmarking across different datasets and MTP problem settings identifies cases where specific HPO methods outperform others.

A Parkinson's Auxiliary Diagnosis Algorithm Based on a Hyperparameter Optimization Method of Deep Learning.

Parkinson's disease is a common mental disease in the world, especially in the middle-aged and elderly groups. Today, clinical diagnosis is the main diagnostic method of Parkinson's disease, but the diagnosis results are not ideal, especially in the early stage of the disease. In this paper, a Parkinson's auxiliary diagnosis algorithm based on a hyperparameter optimization method of deep learning is proposed for the Parkinson's diagnosis. The diagnosis system uses ResNet50 to achieve feature extraction and Parkinson's classification, mainly including speech signal processing part, algorithm improvement part based on Artificial Bee Colony algorithm (ABC) and optimizing the hyperparameters of ResNet50 part. The improved algorithm is called Gbest Dimension Artificial Bee Colony algorithm (GDABC), proposing "Range pruning strategy" which aims at narrowing the scope of search and "Dimension adjustment strategy" which is to adjust gbest dimension by dimension. The accuracy of the diagnosis system in the verification set of Mobile Device Voice Recordings at King's College London (MDVR-CKL) dataset can reach more than 96%. Compared with current Parkinson's sound diagnosis methods and other optimization algorithms, our auxiliary diagnosis system shows better classification performance on the dataset within limited time and resources.

Enhancing the Performance and Accuracy in Real-Time Football and Player Detection Using Upgraded YOLOv5 Architecture

The study presents a significantly improved version of the YOLOv5 real-time object detection model for football player recognition. The proposed technique includes feature-tuning and hyper-parameter optimization methods that have been carefully selected to enhance both speed and accuracy, resulting in a superior real-time performance of the YOLOv5 architecture. Furthermore, the YOLOv5 model incorporates a SimSPPF module that enables multi-scale feature extraction with less computational power, making it a highly efficient and effective solution. We selected the GhostNet module to reduce complexity and the Slim scale detection layer for precise bounding box prediction. Our tests, conducted with recordings of multiple football matches, demonstrate that our model accurately detects both the football and players even in complex scenarios with occlusions and dynamic illumination. The suggested method outperforms the original YOLOv5n model in terms of precision, recall, and mean average precision at 0.5 IoU. It is also more computationally efficient. This method has potential applications in live broadcasting, player monitoring, and sports analytics. The upgraded YOLOv5 model demonstrates superior accuracy and efficiency compared to previous methods that rely on traditional image processing techniques or two-stage detectors. This makes it highly suitable for practical, real-world deployments.

Predicting and analyzing the cementing quality of oil well reservoirs based on Bayesian-random forest model

Cementing quality crucially influences the safety, efficiency, and costs of oil and gas wells. Traditional evaluations rely on empirical and qualitative methods, introducing uncertainty. This study analyzed nearly 2000 wells casing and cementing data from 2016 to 2021 in an eastern China oil field. The original data was analyzed using multiple data cleaning techniques, and effectively dealing with missing values and anomalies. Additionally, the application of feature selection techniques such as random forest successfully identified key elements within the data. Three machine learning models and hyperparameter optimization methods were compared. A Bayesian random forest (BS-RF) model was established with hyperparameter optimization for cementing-quality evaluation. Finally, the relationship between the influencing factors and well-cementing quality were determined by the SHapley Additive exPlanations (SHAP) method. This enabled the evaluation and prediction of well cementing quality. The results indicate that the model achieved an accuracy of 93.21%. When the validation set was used for verification, the predictive accuracy was 95%. This indicates that the model is relatively stable and exhibits strong generalization capabilities across diverse datasets. Performance numeric of cement slurry (SPN), drilling fluid density before cementing, deviation angle, and flushing time are the four key features that influence cementing quality. Each has a SHAP value above 0.12. This article provides guidance for future well-cemented construction schemes.

Optimizing PGRs for in vitro shoot proliferation of pomegranate with bayesian-tuned ensemble stacking regression and NSGA-II: a comparative evaluation of machine learning models

BackgroundThe process of optimizing in vitro shoot proliferation is a complicated task, as it is influenced by interactions of many factors as well as genotype. This study investigated the role of various concentrations of plant growth regulators (zeatin and gibberellic acid) in the successful in vitro shoot proliferation of three Punica granatum cultivars (‘Faroogh’, ‘Atabaki’ and ‘Shirineshahvar’). Also, the utility of five Machine Learning (ML) algorithms—Support Vector Regression (SVR), Random Forest (RF), Extreme Gradient Boosting (XGB), Ensemble Stacking Regression (ESR) and Elastic Net Multivariate Linear Regression (ENMLR)—as modeling tools were evaluated on in vitro multiplication of pomegranate. A new automatic hyperparameter optimization method named Adaptive Tree Pazen Estimator (ATPE) was developed to tune the hyperparameters. The performance of the models was evaluated and compared using statistical indicators (MAE, RMSE, RRMSE, MAPE, R and R2), while a specific Global Performance Indicator (GPI) was introduced to rank the models based on a single parameter. Moreover, Non‑dominated Sorting Genetic Algorithm‑II (NSGA‑II) was employed to optimize the selected prediction model.ResultsThe results demonstrated that the ESR algorithm exhibited higher predictive accuracy in comparison to other ML algorithms. The ESR model was subsequently introduced for optimization by NSGA‑II. ESR-NSGA‑II revealed that the highest proliferation rate (3.47, 3.84, and 3.22), shoot length (2.74, 3.32, and 1.86 cm), leave number (18.18, 19.76, and 18.77), and explant survival (84.21%, 85.49%, and 56.39%) could be achieved with a medium containing 0.750, 0.654, and 0.705 mg/L zeatin, and 0.50, 0.329, and 0.347 mg/L gibberellic acid in the ‘Atabaki’, ‘Faroogh’, and ‘Shirineshahvar’ cultivars, respectively.ConclusionsThis study demonstrates that the 'Shirineshahvar' cultivar exhibited lower shoot proliferation success compared to the other cultivars. The results indicated the good performance of ESR-NSGA-II in modeling and optimizing in vitro propagation. ESR-NSGA-II can be applied as an up-to-date and reliable computational tool for future studies in plant in vitro culture.

A hybrid quantum ensemble learning model for malicious code detection

Quantum computing as a new computing model with parallel computing capability and high information carrying capacity, has attracted a lot of attention from researchers. Ensemble learning is an effective strategy often used in machine learning to improve the performance of weak classifiers. Currently, the classification performance of quantum classifiers is not satisfactory enough due to factors such as the depth of quantum circuit, quantum noise, and quantum coding method, etc. For this reason, this paper combined the ensemble learning idea and quantum classifiers to design a novel hybrid quantum machine learning model. Firstly, we run the Stacking method in classical machine learning to realize the dimensionality reduction of high-latitude data while ensuring the validity of data features. Secondly, we used the Bagging method and Bayesian hyperparameter optimization method applied to quantum support vector machine (QSVM), quantum K nearest neighbors (QKNN), variational quantum classifier (VQC). Thirdly, the voting method is used to ensemble the predict results of QSVM, QKNN, VQC as the final result. We applied the hybrid quantum ensemble machine learning model to malicious code detection. The experimental results show that the classification precision (accuracy, F1-score) of this model has been improved to 98.9% (94.5%, 94.24%). Combined with the acceleration of quantum computing and the higher precision rate, it can effectively deal with the growing trend of malicious codes, which is of great significance to cyberspace security.

Ensemble effort estimation with metaheuristic hyperparameters and weight optimization for achieving accuracy.

Software development effort estimation (SDEE) is recognized as vital activity for effective project management since under or over estimating can lead to unsuccessful utilization of project resources. Machine learning (ML) algorithms are largely contributing in SDEE domain, particularly ensemble effort estimation (EEE) works well in rectifying bias and subjectivity to solo ML learners. Performance of EEE significantly depends on hyperparameter composition as well as weight assignment mechanism of solo learners. However, in EEE domain, impact of optimization in terms of hyperparameter tunning as well as weight assignment is explored by few researchers. This study aims in improving SDEE performance by incorporating metaheuristic hyperparameter and weight optimization in EEE, which enables accuracy and diversity to the ensemble model. The study proposed Metaheuristic-optimized Multi-dimensional bagging scheme and Weighted Ensemble (MoMdbWE) approach. This is achieved by proposed search space division and hyperparameter optimization method named as Multi-dimensional bagging (Mdb). Metaheuristic algorithm considered for this work is Firefly algorithm (FFA), to get best hyperparameters of three base ML algorithms (Random Forest, Support vector machine and Deep Neural network) since FFA has shown promising results of fitness in terms of MAE. Further enhancement in performance is achieved by incorporating FFA-based weight optimization to construct Metaheuristic-optimized weighted ensemble (MoWE) of individual multi-dimensional bagging schemes. Proposed scheme is implemented on eight frequently utilized effort estimation datasets and results are evaluated by 5 error metrices (MAE, RMSE, MMRE, MdMRE, Pred), standard accuracy and effect size along with Wilcox statistical test. Findings confirmed that the use of FFA optimization for hyperparameter (with search space sub-division) and for ensemble weights, has significantly enhanced performance in comparison with individual base algorithms as well as other homogeneous and heterogenous EEE techniques.

Identification of multiclass tympanic membranes by using deep feature transfer learning and hyperparameter optimization

Middle ear health is a process that generally depends on eardrum health. Middle ear disorders are more common during childhood. Permanent damage may occur in bacterial or viral infections in this region if an early diagnosis is not made. Infectious ear disease, especially known as Otitis Media, is one of the diseases. In this study, morphological features of images are obtained by using various feature extraction methods. Deep feature-based transfer learning and hyperparameter optimization methods were used to detect the presence and type of otitis media. While the EfficientNet convolutional neural network (CNN) model was used to extract deep features, KNN, SVM, and Ensemble classifiers were used as classifiers. Bayesian, Grid Search, and Random Search were used for hyperparameter optimization. As a result of the experiments carried out, it was observed that the classification performance was 99.1%.

A Novel Iterative Hyperparameter Optimization Method for Enhanced Diabetic Retinopathy Screening

A major contributor to blindness in people of working age worldwide is diabetic retinopathy (DR), which is mostly caused by vascular damage brought on by high blood sugar in those who have diabetes mellitus. Since early detection greatly reduces the risk of vision loss, it is essential for the effective management of DR. This work presents a novel method that uses artificial intelligence (AI) to improve fundus image analysis for early diagnosis of diabetes mellitus (DR). Understanding how important high-quality images are to precise diagnosis, our approach starts with a sophisticated preprocessing stage. This stage focuses on improving the pictures by removing erroneous dark patches and other distortions, readying the dataset for additional examination. After preprocessing, we utilize an innovative hybrid hyperparameter optimization model that creatively combines the best features of grid and random search techniques. In order to identify early indicators of DR, such as microaneurysms and vessel changes, this model seeks to optimize the parameters of machine learning algorithms. Through a methodical analysis of the preprocessed fundus images, our model more effectively and accurately detects patterns suggestive of early-stage DR. The results of this study not only show that using AI in conjunction with careful preprocessing and hyperparameter optimization is effective in diagnosing DR, but they also open up new avenues for future investigation into the use of AI in early disease detection. In the end, this study makes a substantial contribution to the field of medical imaging and presents a viable treatment option for preventing vision loss in diabetic patients through early diagnostic intervention.

Improving classification accuracy of fine-tuned CNN models: Impact of hyperparameter optimization

The immense popularity of convolutional neural network (CNN) models has sparked a growing interest in optimizing their hyperparameters. Discovering the ideal values for hyperparameters to achieve optimal CNN training is a complex and time-consuming task, often requiring repetitive numerical experiments. As a result, significant attention is currently being devoted to developing methods aimed at tailoring hyperparameters for specific CNN models and classification tasks. While existing optimization methods often yield favorable image classification results, they do not provide guidance on which hyperparameters are worth optimizing, the appropriate value ranges for those hyperparameters, or whether it is reasonable to use a subset of training data for the optimization process. This work is focused on the optimization of hyperparameters during transfer learning, with the goal of investigating how different optimization methods and hyperparameter selections impact the performance of fine-tuned models. In our experiments, we assessed the importance of various hyperparameters and identified the ranges within which optimal CNN training can be achieved. Additionally, we compared four hyperparameter optimization methods—grid search, random search, Bayesian optimization, and the Asynchronous Successive Halving Algorithm (ASHA). We also explored the feasibility of fine-tuning hyperparameters using a subset of the training data. By optimizing the hyperparameters, we observed an improvement in CNN classification accuracy of up to 6%. Furthermore, we found that achieving a balance in class distribution within the subset of data used for parameter optimization is crucial in establishing the optimal set of hyperparameters for CNN training. The results we obtained demonstrate that hyperparameter optimization is highly dependent on the specific task and dataset at hand.