Parzen Estimator Research Articles

Advanced Hybrid Techniques for Predicting Discharge Coefficients in Ogee-Crested Spillways: Integrating Physical, Numerical, and Machine Learning Models

Abstract The primary objective of this work was to examine the flow characteristics over an ogee spillway using both a numerical model and the Machine Learning (ML) approach. A 3D computational fluid dynamics (CFD) model was employed to simulate the flow over an ogee spillway, utilizing the Reynolds averaged Navier–Stokes equations. The simulation encompassed a wide variety of head ratios, ranging from 0.1 to 6.0, to extend the rating curve of discharge coefficient (C) and head ratio (He/H0). The formation of the negative pressure zone rapidly occurred, and the maximum velocity area developed from toe to top of the spillway surface as the head ratio increased. Then, four ML models—RF, FNN, ADB, and KNN—were utilized to estimate the discharge coefficient of the spillway. Hyperparameter tuning using the Tree-Structured Parzen Estimator (TPE) and five-fold cross-validation ensured robust model performance. The ML model's efficacy was assessed by conducting 200 random seed simulations. The RF and ADB models exhibited the highest predictive accuracy and consistency, with mean correlation coefficient (CC) values of 0.979 and 0.975, respectively. FNN and KNN also performed well but showed greater variability in their prediction. The results demonstrated reasonably good agreement between the physical, numerical, and ML models. Both numerical simulation methods and ML models, particularly RF, proved to be cost-efficient and reliable tools for designing and analyzing flow over an ogee spillway. These findings highlight the potential of integrating numerical simulations and advanced ML techniques to enhance the prediction and analysis of hydraulic structures, providing valuable insights for the design and management of spillway systems.

Bayesian optimization of one-dimensional convolutional neural networks (1D CNN) for early diagnosis of Autistic Spectrum Disorder

Autistic Spectrum Disorder (ASD) is a challenging neurological development disorder, which involves poor social interaction, communication, and repetitive behaviours. If autism is identified early enough it can be treated with better outcomes but present diagnostic tests are dependent on subjective opinion, consume a lot of time, and are vague. This study is aimed at optimizing one-dimensional convolutional neural networks (1D CNN) to improve the precision and speed of early ASD diagnosis. Four ASD datasets representing different age groups — toddlers, children, adolescents, and adults were modelled using one-dimensional convolutional neural networks (1D CNN). These datasets are accessible to the public on the UCI Machine Learning Repository and Kaggle, they consist of behavioural features relevant to ASD diagnosis. Each dataset underwent feature selection, categorical encoding, and missing value handling. Then, baseline 1D CNN with predefined hyperparameters was modelled on each of the datasets. Subsequently, the baseline models were optimized using the Tree-structured Parzen Estimator (TPE). An interactive web-based ASD diagnostic tool was developed, where user inputs are processed through age-specific pre-trained optimized models to determine ASD probability. The optimized 1D CNN models significantly outperformed the baseline models across all age groups and achieved scores of 100% in accuracy, precision, recall, F1-score, MCC, and AUC ROC. This implies that the optimized models can reliably identify people in various age groups who have and do not have ASD. The development of an interactive web-based diagnostic tool extends the practical utility of the models, making them accessible for clinical and at-home use.

ITSC fault diagnosis for PMSM by using adaptive filtering and tree-structured parzen estimator optimized-automated random forest

ITSC fault diagnosis for PMSM by using adaptive filtering and tree-structured parzen estimator optimized-automated random forest

Modeling the bond–slip behavior of the interface between a stiffened core and cemented soil based on machine learning approaches

The bond–slip behavior of stiffened deep cement mixing (SDCM) piles—which is crucial for their bearing capacity—evolves continuously with curing age. In the study reported here, 20 element tests were conducted on the interface between cemented soil and a stiffened core, analyzing the bond–slip behavior affected by curing temperature and age, and then ensemble learning methods (XGBoost, random forest) were used to establish models for the evolution of the bond–slip behavior considering thermal effects. The constructed models can predict the peak shear strength (τmax), the residual shear strength (τres), and the interfacial shear modulus (G). The test results show that the shear strength of the stiffened-core–cemented-soil interface grows with the increasing curing temperature and age, with faster growth at 0–14 days compared to 60–90 days. To lessen the reliance on ineffective brute-force searching, Bayesian optimization with a tree-structured Parzen estimator is used to select the hyperparameters of the established models. The results demonstrate the superior performance of the chosen approach, with R2 > 0.93 for the training set and R2 > 0.81 for the test set. The results of the XGBoost model are best for τmax, with a mean absolute percentage error of less than 5 %, thereby enabling accurate predictions of the mechanical parameters of the stiffened-core–cemented-soil. This research enhances the understanding of the mechanical properties of SDCM piles and provides valuable guidance for projects involving such piles.

Comparative Study of LSTM and ANN Models for Power Consumption Prediction of Variable Refrigerant Flow (VRF) Systems in Buildings

The optimized control of variable refrigerant flow (VRF) system requires an accurate time series forecast model for power consumption. Currently, physics-based and black-box models widely used for forecasting power consumption may not be able to capture the dynamic and non-linear behavior of such complex systems. This study presents a long-short-term memory (LSTM), deep learning-based model to accurately predict the power consumption of a VRF system with heat recovery units. The model training used one year of VRF system field test data. The feature selection through the Pearson correlation coefficient was implemented to improve the model's accuracy and computational efficiency. The sensitivity analysis of feature selection was performed by preparing three feature sets, including different levels of relationship with the predicted target. Additionally, the hyperparameters of the models were optimized by Bayesian optimization with the Tree-structured Parzen Estimator algorithm. The deep learning model, LSTM model, was compared to the baseline machine learning model, Artificial Neural Network (ANN) and decision tree. The results show that LSTM-30feat with input time step 4 has the best testing performance of Coefficient of the Variation of the Root Mean Square Error (CvRMSE) 23.3%. The best ANN model is ANN-10feat with input time step 8, which has a CvRMSE of 27.8% in testing and 13,569 trainable parameters. However, LSTM-10feat with input time step 4 has the CvRMSE of 24.8% in testing, and the trainable parameters are 1,809. A higher number of trainable parameters in models might result in increased memory usage on the computer and be computationally expensive.

An optimized multi-layer ensemble model for airborne networks intrusion detection

In view of the characteristics of airborne networks, such as the traffic data of 1553B data bus and public networks, numerous redundant and irrelevant data, and fewer but more types of attack behaviors, a highly representative balanced data subset is generated by combining K-means with synthetic minority oversampling technique (SMOTE). A feature selection (FS) method of fast correlation-based filter combined with information gain (IG-FCBF) is proposed to filter the key characteristics with 90 % cumulative importance. Aiming at the problem that it is not easy for a single classifier to accurately classify various types of attacks, as well as the difficulty in obtaining the best prediction results by adjusting default and manual parameters, a multi-layer ensemble model based on Bayesian optimization tree-structured Parzen estimator (BO-TPE) is proposed for airborne networks intrusion detection, combining the advantages of supervised learning, stacking ensemble method and hyperparameter optimization (HPO). The experimental results show the superiority and effectiveness of the model, as well as its universality in Integrated Avionics Systems (IAS) and onboard public networks, which provides a new approach for airborne networks intrusion identification and protection.

Deep neural network based distribution system state estimation using hyperparameter optimization

In the past decade, distribution system state estimation has become a crucial topic in power system research due to the increasing importance of distribution networks amidst the decline of centralized energy production. This paper addresses a gap in the literature regarding the application of modern hyperparameter optimization techniques in low-voltage distribution system state estimation using deep neural networks. In particular, it demonstrates the use of the Tree-structured Parzen Estimator algorithm, which is a Bayesian hyperparameter optimization method, for distribution system state estimation on real Hungarian low-voltage networks. The study uses data from four real-life low-voltage supply areas in Hungary, which were modeled to address the challenges in obtaining network information. Compared to traditional methods like the weighted least squares method, the Tree-structured Parzen Estimator algorithm significantly improves the accuracy of the voltage amplitude and angle estimations, reducing the relative error by 14–73%. Additionally, it is shown that TPE outperforms simpler methods like Random Search in hyperparameter optimization. The results also reveal connections between the distribution system size and optimal hyperparameters, such as batch size, learning rate, and hidden layer configuration. The proposed non-iterative algorithm, combined with the parallel computation capabilities of deep neural networks utilizing GPU, resulted in four orders of magnitude improvement in runtime. These advancements make the proposed approach a valuable tool for renewable energy integration planning and real-time monitoring, highlighting its potential for practical applications in the power industry.

Hybrid XGB model for predicting unconfined compressive strength of solid waste-cement-stabilized cohesive soil

The utilization of cement has been found to have negative environmental impacts. In order to reduce the quantity of cement used and improve the mechanical properties of solid waste-cement-stabilized cohesive soil, the incorporation of solid waste as additives has been investigated. Unconfined compressive strength is a crucial parameter in geotechnical engineering. However, existing empirical formulas have limited accuracy and applicability when it comes to the unconfined compressive strength of solid waste-cement-stabilized cohesive soil. The machine learning model can be used to provide accurate and comprehensive predictions by considering the nonlinear relationships between independent and dependent variables. This study aims to propose a machine learning model tuned by optimization algorithms with high generalization performance in accurately predicting the unconfined compressive strength. Firstly, a database containing 474 specimens was developed. Secondly, eight machine learning models were established, composed five single models and three hybrid models, to train and test the database. Six performance indicators were employed to evaluate the generalization ability of these models. Finally, the optimal model was selected for analysis of the importance of the feature variables using shapley additive explanations, which were compared with those of the existing empirical model. The research findings indicated that, the extreme gradient boosting model tuned with tree-structured parzen estimators exhibited the highest predictive accuracy and generalization ability. The curing age, cement content, plastic limit, and water content were identified as the most critical factors influencing the unconfined compressive strength. Among the chemical components in solid waste, the aluminum oxide content and silicon dioxide content were found to significantly influence the unconfined compressive strength, while the impact of calcium oxide content was relatively minor. Furthermore, the optimal solid waste content was found to be around 10 %. This study made a significant contribution to the effective utilization of waste resources in the context of sustainable construction practices.

A machine learning method based on TPE-XGBoost model for TRIP/TWIP near-β titanium alloy design

The traditional design method for TRIP/TWIP near-β titanium alloys is time-consuming and expensive. This paper proposes a machine-learning design method for a near-β titanium alloy that uses the Tree-structured Parzen Estimator (TPE) algorithm to optimize the hyperparameter space of XGBoost for improved model prediction accuracy in TRIP/TWIP. The model predicts the mechanical properties and plasticity mechanisms of the alloy based on its composition, key empirical parameters derived from the composition, and solution treatment temperature. The test results of the TPE-XGBoost model showed that when predicting the ultimate tensile strength (UTS), the adjusted determination coefficient (Adjusted R2), root mean square error (RMSE), and mean absolute error (MAE) values were 0.9264, 39.63 MPa and 44.83 MPa, respectively. When predicting elongation (El), the Adjusted R2, RMSE, and MAE values are 0.9134, 1.97 %, and 1.89 %, respectively. When predicting the plastic mechanisms of the alloy, the F1-score, Recall, Precision, Accuracy, and Area Under the ROC Curve (AUC) were 0.74, 0.75, 0.80, 0.78, and 0.83, respectively. A near-β titanium alloy, Ti-8V-4Mo-3.5Al-3Cr-2Zr (wt%), along with an associated heat treatment process, has been developed using this model. The experimental results have demonstrated that the model accurately predicts the strength, elongation, and plasticity mechanisms of the alloys. The TRIP/TWIP effects occur due to {332}<113> twinning and secondary α" martensite formation during tensile deformation. This study offers an alternative approach for designing TRIP/TWIP near-β titanium alloys.

Ensemble Learning for Nuclear Power Generation Forecasting Based on Deep Neural Networks and Support Vector Regression

Forecasting nuclear energy production is essential for market operations such as security, economic efficiency, resource optimization, grid stability, and the integration of renewable energy sources. Forecasting approaches allow nuclear power plants to operate consistently, contributing to the overall reliability and long-term viability of the energy system. It is noted that energy systems researchers are increasingly interested in machine learning models used to face the challenge of time series forecasting. This study evaluates a hybrid ensemble learning of three time series forecasting models including least-squares support vector regression, gated recurrent unit, and long short-term memory models applied to nuclear power time series forecasting on the dataset of French power plants from 2009 to 2020. Furthermore, this research evaluates forecasting results in which approaches are directed towards the optimized RreliefF (Robust relief Feature) selection algorithm using a hyperparameter optimization based on tree-structured Parzen estimator and following an ensemble learning approach, showing promising results in terms of performance metrics. The suggested ensemble learning model, which combines deep learning and the RreliefF algorithm using a hold-out, outperforms the other nine forecasting models in this study according to performance criteria such as 75% for the coefficient of determination, a root squared error average of 0.108, and an average absolute error of 0.080.

Carbon Price Forecasting Using Optimized Sliding Window Empirical Wavelet Transform and Gated Recurrent Unit Network to Mitigate Data Leakage

Precise forecasts of carbon prices are crucial for reducing greenhouse gas emissions and promoting sustainable, low-carbon development. To mitigate noise interference in carbon price data, hybrid models integrating data decomposition techniques are commonly utilized. However, it has been observed that the improper utilization of data decomposition techniques can lead to data leakage, thereby invalidating the model’s practical applicability. This study introduces a leakage-free hybrid model for carbon price forecasting based on the sliding window empirical wavelet transform (SWEWT) algorithm and the gated recurrent unit (GRU) network. First, the carbon price data are sampled using a sliding window approach and then decomposed into more stable and regular subcomponents through the EWT algorithm. By exclusively employing the data from the end of the window as input, the proposed method can effectively mitigate the risk of data leakage. Subsequently, the input data are passed into a multi-layer GRU model to extract patterns and features from the carbon price data. Finally, the optimized hybrid model is obtained by iteratively optimizing the hyperparameters of the model using the tree-structured Parzen estimator (TPE) algorithm, and the final prediction results are generated by the model. When used to forecast the closing price of the Guangdong Carbon Emission Allowance (GDEA) for the last nine years, the proposed hybrid model achieves outstanding performance with an R2 value of 0.969, significantly outperforming other structural variants. Furthermore, comparative experiments from various perspectives have validated the model’s structural rationality, practical applicability, and generalization capability, confirming that the proposed framework is a reliable choice for carbon price forecasting.

Machine-learning approach for operating electron beam at KEK electron/positron injector linac

In current accelerators, numerous parameters and monitored values are to be adjusted and evaluated, respectively. In addition, fine adjustments are required to achieve the target performance. Therefore, the conventional accelerator-operation method, in which experts manually adjust the parameters, is reaching its limits. We are currently investigating the use of machine learning for accelerator tuning as an alternative to expert-based tuning. In recent years, machine-learning algorithms have progressed significantly in terms of speed, sensitivity, and application range. In addition, various libraries are available from different vendors and are relatively easy to use. Herein, we report the results of electron-beam tuning experiments using Bayesian optimization, a tree-structured Parzen estimator, and a covariance matrix-adaptation evolution strategy. Beam-tuning experiments are performed at the KEK e−/e+ injector Linac to maximize the electron-beam charge and reduce the energy-dispersion function. In each case, the performance achieved is comparable to that of a skilled expert. Published by the American Physical Society 2024

Spatial Mapping of Soil CO2 Flux in the Yellow River Delta Farmland of China Using Multi-Source OpticalRemote Sensing Data

The spatial prediction of soil CO2 flux is of great significance for assessing regional climate change and high-quality agricultural development. Using a single satellite to predict soil CO2 flux is limited by climatic conditions and land cover, resulting in low prediction accuracy. To this end, this study proposed a strategy of multi-source spectral satellite coordination and selected seven optical satellite remote sensing data sources (i.e., GF1-WFV, GF6-WFV, GF4-PMI, CB04-MUX, HJ2A-CCD, Sentinel 2-L2A, and Landsat 8-OLI) to extract auxiliary variables (i.e., vegetation indices and soil texture features). We developed a tree-structured Parzen estimator (TPE)-optimized extreme gradient boosting (XGBoost) model for the prediction and spatial mapping of soil CO2 flux. SHapley additive explanation (SHAP) was used to analyze the driving effects of auxiliary variables on soil CO2 flux. A scatter matrix correlation analysis showed that the distributions of auxiliary variables and soil CO2 flux were skewed, and the linear correlations between them (r &lt; 0.2) were generally weak. Compared with single-satellite variables, the TPE-XGBoost model based on multiple-satellite variables significantly improved the prediction accuracy (RMSE = 3.23 kg C ha−1 d−1, R2 = 0.73), showing a stronger fitting ability for the spatial variability of soil CO2 flux. The spatial mapping results of soil CO2 flux based on the TPE-XGBoost model revealed that the high-flux areas were mainly concentrated in eastern and northern farmlands. The SHAP analysis revealed that PC2 and the TCARI of Sentinel 2-L2A and the TVI of HJ2A-CCD had significant positive driving effects on the prediction accuracy of soil CO2 flux. The above results indicate that the integration of multiple-satellite data can enhance the reliability and accuracy of spatial predictions of soil CO2 flux, thereby supporting regional agricultural sustainable development and climate change response strategies.

Improving the Interpretability of Data-Driven Models for Additive Manufacturing Processes Using Clusterwise Regression

Wire Arc Additive Manufacturing (WAAM) represents a disruptive technology in the field of metal additive manufacturing. Understanding the relationship between input factors and layer geometry is crucial for studying the process comprehensively and developing various industrial applications such as slicing software and feedforward controllers. Statistical tools such as clustering and multivariate polynomial regression provide methods for exploring the influence of input factors on the final product. These tools facilitate application development by helping to establish interpretable models that engineers can use to grasp the underlying physical phenomena without resorting to complex physical models. In this study, an experimental campaign was conducted to print steel components using WAAM technology. Advanced statistical methods were employed for mathematical modeling of the process. The results obtained using linear regression, polynomial regression, and a neural network optimized using the Tree-structured Parzen Estimator (TPE) were compared. To enhance performance while maintaining the interpretability of regression models, clusterwise regression was introduced as an alternative modeling technique along with multivariate polynomial regression. The results showed that the proposed approach achieved results comparable to neural network modeling, with a Mean Absolute Error (MAE) of 0.25 mm for layer height and 0.68 mm for layer width compared to 0.23 mm and 0.69 mm with the neural network. Notably, this approach preserves the interpretability of the models; a further discussion on this topic is presented as well.

A comparative analysis of machine learning algorithms with tree-structured parzen estimator for liver disease prediction

The liver is one of the most essential organs in the body, which helps with metabolism and keeping the body healthy. Successful treatments and better patient outcomes depend on early and correct Liver Disease (LD) diagnosis and identification. This study proposes a system for predicting the LD by combining the techniques of Machine Learning (ML) algorithms that include the Decision Tree, Random Forest, Extra Tree Classifier (ETC), LightGBM, and Adaboost, with the Tree-Structured Parzen Estimator (TPE) method for hyperparameter tuning. No previous literature research has utilized ML algorithms with TPE to predict LD. For this research, the Indian Liver Patients’ Dataset with 583 instances and 11 attributes was used. In the pre-processing of the data, techniques such as upsampling have been utilized to address the class imbalance problem. Normalization has been employed to scale the dataset, and feature selection has been applied to choose important features. The proposed model has been analyzed and compared using a 10-fold cross-validation process, with various evaluation metrics including accuracy, precision, recall, and F1-score. The model proposed in this study achieved the best level of accuracy while employing the ETC with the TPE approach, with a recorded accuracy of 95.8%.

Optimizing Mean Fragment Size Prediction in Rock Blasting: A Synergistic Approach Combining Clustering, Hyperparameter Tuning, and Data Augmentation

Accurate estimation of the mean fragment size is crucial for optimizing open-pit mining operations. This study presents an approach that combines clustering, hyperparameter optimization, and data augmentation to enhance prediction accuracy using the Xtreme Gradient Boosting (XGBoost) regression model. A dataset of 110 blasts was divided into 97 blasts for training and testing, whereas a separate set of 13 new, unseen blasts was used to evaluate the robustness and generalization of the model. Hierarchical Agglomerative (HA) and K-means clustering algorithms were used, with HA clustering providing a higher cluster quality. To address class imbalance and improve model generalization, a synthetic minority oversampling technique for regression with Gaussian noise (SMOGN) was employed. Hyperparameter tuning was conducted using HyperOpt by comparing Random Search (RS) with the Advanced Tree-structured Parzen Estimator (ATPE). The combination of ATPE with HA clustering and SMOGN in an expanded search space produced the best results, achieving superior prediction accuracy and reliability. The proposed HAC1-SMOGN model, which integrates HA clustering, ATPE tuning, and SMOGN augmentation, achieved a mean squared error (MSE) of 0.0002 and an R2 of 0.98 on the test set. This study highlights the synergistic benefits of clustering, hyperparameter optimization, and data augmentation in enhancing machine learning models for regression tasks, particularly in scenarios with class imbalance or limited data.

Image-based intrusion detection system for GPS spoofing cyberattacks in unmanned aerial vehicles

The operations of unmanned aerial vehicles (UAVs) are susceptible to cybersecurity risks, mainly because of their firm reliance on the Global Positioning System (GPS) and radio frequency (RF) sensors. GPS and RF sensors are vulnerable to potential threats, such as spoofing attacks that can cause the UAVs to behave erratically. Since these threats are widespread and potent, it is imperative to develop effective intrusion detection systems. In this paper, we propose an image-based intrusion detection system for detecting GPS spoofing cyberattacks based on a deep learning methodology. We combine convolutional neural networks with Principal Component Analysis (PCA) to reduce the dimensionality of the dataset features, data augmentation to increase the size and diversity of the training dataset, and transfer learning to improve the proposed model’s performance with limited data to design a fast, accurate, and general method. Extensive numerical experiments demonstrate the effectiveness of the proposed solution carried out using benchmark datasets. We achieved an accuracy of 100% within a running time of 120.64 s at 0.3529 ms latency and a detection time of 2.035 s in the case of the training dataset. Further, using this trained model, we achieved an accuracy of 99.25% within a detection time of 2.721 s on an unseen dataset that was unrelated to the one used for training the model. In contrast, other models, such as Inception-v3, showed lower accuracy on unseen datasets. However, Inception-v3 performance improved significantly after Bayesian optimization, with the Tree-structured Parzen Estimator reaching 99.06% accuracy. Our results demonstrate that the proposed image-based intrusion detection method outperforms the existing solutions while providing a general model for detecting cyberattacks included in unseen datasets.

A Method for Predicting Indoor CO2 Concentration in University Classrooms: An RF-TPE-LSTM Approach

Classrooms play a pivotal role in students’ learning, and maintaining optimal indoor air quality is crucial for their well-being and academic performance. Elevated CO2 levels can impair cognitive abilities, underscoring the importance of accurate predictions of CO2 concentrations. To address the issue of inadequate analysis of factors affecting classroom CO2 levels in existing models, leading to suboptimal feature selection and limited prediction accuracy, we introduce the RF-TPE-LSTM model in this study. Our model integrates factors that affect classroom CO2 levels to enhance predictions, including occupancy, temperature, humidity, and other relevant factors. It combines three key components: random forest (RF), tree-structured Parzen estimator (TPE), and long short-term memory (LSTM). By leveraging these techniques, our model enhances the predictive capabilities and refines itself through Bayesian optimization using TPE. Experiments conducted on a self-collected dataset of classroom CO2 concentrations and influencing factors demonstrated significant improvements in the MAE, RMSE, MAPE, and R2. Specifically, the MAE, RMSE, and MAPE were reduced to 2.96, 5.54, and 0.60%, respectively, with the R2 exceeding 98%, highlighting the model’s effectiveness in assessing indoor air quality.

Dam Deformation Prediction Considering the Seasonal Fluctuations Using Ensemble Learning Algorithm

Dam deformation is the most visual and relevant monitoring quantity that reflects the operational condition of a concrete dam. The seasonal variations in the external environment can induce seasonal fluctuations in the deformation of concrete dams. Hence, preprocessing the deformation monitoring series to identify seasonal fluctuations within the series can effectively enhance the accuracy of the predictive model. Firstly, the dam deformation time series are decomposed into the seasonal and non-seasonal components based on the seasonal decomposition technique. The advanced ensemble learning algorithm (Extreme Gradient Boosting model) is used to forecast the seasonal and non-seasonal components independently, as well as employing the Tree-structured Parzen Estimator (TPE) optimization algorithm to tune the model parameters, ensuring the optimal performance of the prediction model. The results of the case study indicate that the predictive performance of the proposed model is intuitively superior to the benchmark models, demonstrated by a higher fitting accuracy and smaller prediction residuals. In the comparison of the objective evaluation metrics RMSE, MAE, and R2, the proposed model outperforms the benchmark models. Additionally, using feature importance measures, it is found that in predicting the seasonal component, the importance of the temperature component increases, while the importance of the water pressure component decreases compared to the prediction of the non-seasonal component. The proposed model, with its elevated predictive accuracy and interpretability, enhances the practicality of the model, offering an effective approach for predicting concrete dam deformation.

Optimizing the light gradient-boosting machine algorithm for an efficient early detection of coronary heart disease

BackgroundCoronary heart disease (CHD) remains a prominent cause of mortality globally, necessitating early and accurate detection methods. Traditional diagnostic approaches can be invasive, costly, and time-consuming, necessitating the need for more efficient alternatives. This aimed to optimize the Light Gradient-Boosting Machine (LightGBM) algorithm to enhance its performance and accuracy in the early detection of CHD, providing a reliable, cost-effective, and non-invasive diagnostic tool. MethodsThe Framingham Heart Study (FHS) dataset publicly available on Kaggle was used in this study. Multiple Imputations by Chained Equations (MICE) were applied separately to the training and testing sets to handle missing data. Borderline-SMOTE (Synthetic Minority Over-sampling Technique) was used on the training set to balance the dataset. The LightGBM algorithm was selected for its efficiency in classification tasks, and Bayesian Optimization with Tree-structured Parzen Estimator (TPE) was employed to fine-tune its hyperparameters. The optimized LightGBM model was trained and evaluated using metrics such as accuracy, precision, and AUC-ROC on the test set, with cross-validation to ensure robustness and generalizability. FindingsThe optimized LightGBM model showed significant improvement in early CHD detection. The baseline LightGBM model with dropped missing values had an accuracy of 0.8333, sensitivity of 0.1081, precision of 0.3429, F1 score of 0.1644, and AUC of 0.6875. With MICE imputation, performance improved to an accuracy of 0.9399, sensitivity of 0.6693, precision of 0.9043, F1 score of 0.7692, and AUC of 0.9457. The combined approach of Borderline-SMOTE, MICE imputation, and TPE for LightGBM achieved an accuracy of 0.9882, sensitivity of 0.9370, precision of 0.9835, F1 score of 0.9597, and AUC of 0.9963, indicating a highly effective and robust model. InterpretationThe optimized model demonstrated outstanding performance in early CHD detection. The study's strengths include its comprehensive approach to addressing missing data and class imbalance and the fine-tuning of hyperparameters through Bayesian Optimization. However, there is a need to test with other datasets for its generalizability to be well-established. This study provides a strong framework for early CHD detection, improving clinical practice by allowing for more precise and dependable diagnostics and effective interventions.