Hyperparameter Values Research Articles

CART-ANOVA-Based Transfer Learning Approach for Seven Distinct Tumor Classification Schemes with Generalization Capability

Background/Objectives: Deep transfer learning, leveraging convolutional neural networks (CNNs), has become a pivotal tool for brain tumor detection. However, key challenges include optimizing hyperparameter selection and enhancing the generalization capabilities of models. This study introduces a novel CART-ANOVA (Cartesian-ANOVA) hyperparameter tuning framework, which differs from traditional optimization methods by systematically integrating statistical significance testing (ANOVA) with the Cartesian product of hyperparameter values. This approach ensures robust and precise parameter tuning by evaluating the interaction effects between hyperparameters, such as batch size and learning rate, rather than relying solely on grid or random search. Additionally, it implements seven distinct classification schemes for brain tumors, aimed at improving diagnostic accuracy and robustness. Methods: The proposed framework employs a ResNet18-based knowledge transfer learning (KTL) model trained on a primary dataset, with 20% allocated for testing. Hyperparameters were optimized using CART-ANOVA analysis, and statistical validation ensured robust parameter selection. The model’s generalization and robustness were evaluated on an independent second dataset. Performance metrics, including precision, accuracy, sensitivity, and F1 score, were compared against other pre-trained CNN models. Results: The framework achieved exceptional testing accuracy of 99.65% for four-class classification and 98.05% for seven-class classification on the source 1 dataset. It also maintained high generalization capabilities, achieving accuracies of 98.77% and 96.77% on the source 2 datasets for the same tasks. The incorporation of seven distinct classification schemes further enhanced variability and diagnostic capability, surpassing the performance of other pre-trained models. Conclusions: The CART-ANOVA hyperparameter tuning framework, combined with a ResNet18-based KTL approach, significantly improves brain tumor classification accuracy, robustness, and generalization. These advancements demonstrate strong potential for enhancing diagnostic precision and informing effective treatment strategies, contributing to advancements in medical imaging and AI-driven healthcare solutions.

Exploring the Relationship Between the Geometry of a Fixed Embedding of Image Data and Its Underlying Cluster Structure

Standard self-supervised clustering algorithms transform input data via embedding models that are trained to fit the data and then cluster the embedded vectors. Despite its flexibility, a data driven embedding model may not be applicable when the raw data are unavailable due to privacy or security concerns, and it cannot be adapted to the increasingly common framework of transfer learning. We previously proposed a Merge & Expand (ME) framework for clustering images using a fixed embedding model. In this study, we have substantially modified our ME framework and conducted a series of experimental analysis to explore the relationship between the geometry of a fixed embedding space and the underlying cluster structure. We demonstrate that the clustering outcomes are robust against varying hyperparameter values. We assessed the heterogeneity of predicted labels in each region, revealing that it is a strong indicator of the quality of clustering outcomes. We further exploited the heterogeneity information to modify the ME framework, improving clustering accuracy by introducing a second embedding. Moreover, we provide intuitive explanations for sources of confusion in merged seed regions. Comparisons with numerous other clustering methods on five datasets indicate that our ME framework performs competitively despite employing a fixed embedding, a simple CNN architecture, and a common loss function. Thus, our ME framework enables users to better understand the relationship between the geometry of the embedding space and the underlying cluster structure. Supplementary materials for this article are available online.

Development and experimental validation of a machine learning model for the prediction of new antimalarials

A large set of antimalarial molecules (N ~ 15k) was employed from ChEMBL to build a robust random forest (RF) model for the prediction of antiplasmodial activity. Rather than depending on high throughput screening (HTS) data, molecules tested at multiple doses against blood stages of Plasmodium falciparum were used for model development. The open-access and code-free KNIME platform was used to develop a workflow to train the model on 80% of data (N ~ 12k). The hyperparameter values were optimized to achieve the highest predictive accuracy with nine different molecular fingerprints (MFPs), among which Avalon MFPs (referred to as RF-1) provided the best results. RF-1 displayed 91.7% accuracy, 93.5% precision, 88.4% sensitivity and 97.3% area under the Receiver operating characteristic (AUROC) for the remaining 20% test set. The predictive performance of RF-1 was comparable to that of the malaria inhibitor prediction platform (MAIP), a recently reported consensus model based on a large proprietary dataset. However, hits obtained from RF-1 and MAIP from a commercial library did not overlap, suggesting that these two models are complementary. Finally, RF-1 was used to screen small molecules under clinical investigations for repurposing. Six molecules were purchased, out of which two human kinase inhibitors were identified to have single-digit micromolar antiplasmodial activity. One of the hits (compound 1) was a potent inhibitor of β-hematin, suggesting the involvement of parasite hemozoin (Hz) synthesis in the parasiticidal effect. The training and test sets are provided as supplementary information, allowing others to reproduce this work.

Application of Symbolic Classifiers and Multi-Ensemble Threshold Techniques for Android Malware Detection

Android malware detection using artificial intelligence today is a mandatory tool to prevent cyber attacks. To address this problem in this paper the proposed methodology consists of the application of genetic programming symbolic classifier (GPSC) to obtain symbolic expressions (SEs) that can detect if the android is malware or not. To find the optimal combination of GPSC hyperparameter values the random hyperparameter values search method (RHVS) method and the GPSC were trained using 5-fold cross-validation (5FCV). It should be noted that the initial dataset is highly imbalanced (publicly available dataset). This problem was addressed by applying various preprocessing and oversampling techniques thus creating a huge number of balanced dataset variations and on each dataset variation the GPSC was trained. Since the dataset has many input variables three different approaches were considered: the initial investigation with all input variables, input variables with high feature importance, application of principal component analysis. After the SEs with the highest classification performance were obtained they were used in threshold-based voting ensembles and the threshold values were adjusted to improve classification performance. Multi-TBVE has been developed and using them the robust system for Android malware detection was achieved with the highest accuracy of 0.98 was obtained.

AUTOMATING METER CLASSIFICATION OF ARABIC POEMS: A HARRIS HAWKS OPTIMIZATION WITH DEEP LEARNING PERSPECTIVE

Meter classification in Arabic poetry is a crucial factor that describes the rhythmic structure of poems. Classical Arabic poetry relies on explicit meters, referred to as “Arud”, to create a structured and harmonious flow. Arabic meter is based on the pattern of short and long syllables, and each meter has a particular combination of feet (taf’ilah) that defines its unique rhythmic structure. Poets use diverse Arabic meters to evoke aesthetic or emotional qualities in their poetry. The mastery of meter is considered a sophisticated and skillful aspect of traditional Arabic poetry, which reflects the rich heritage of Arabic literature. The meter provides poets with unique opportunities and constraints, influencing the style and tone of their verses. Using deep learning (DL) for the meter classification of Arabic poems includes leveraging a neural network to automatically learn the features and patterns that discriminate between various meters. This paper presents a Fractal Harris Hawks Optimization with DL-based Meter Classification of Arabic Poems (HHODL-MCAP) technique. The HHODL-MCAP technique exploits the optimal DL model for the identification of distinct classes of meters of Arabic poems. The HHODL-MCAP technique involves a three-layered process. Primarily, the HHODL-MCAP technique performs data preprocessing to transform the data into a beneficial format. Second, the HHODL-MCAP technique applies long short-term memory (LSTM) with a Bidirectional Temporal Convolutional Networks (BiTCNs) model for the automated identification of various Arabic meter classes. At last, the HHO algorithm can be exploited to choose the hyperparameter values of the LSTM-BiTCN model optimally. A series of experiments were conducted to ensure the improved detection outcomes of the HHODL-MCAP technique. The extensive simulation results underline the supremacy of the HHODL-MCAP technique in the meter classification process.

Applying MLP-Mixer and gMLP to Human Activity Recognition.

The development of deep learning has led to the proposal of various models for human activity recognition (HAR). Convolutional neural networks (CNNs), initially proposed for computer vision tasks, are examples of models applied to sensor data. Recently, high-performing models based on Transformers and multi-layer perceptrons (MLPs) have also been proposed. When applying these methods to sensor data, we often initialize hyperparameters with values optimized for image processing tasks as a starting point. We suggest that comparable accuracy could be achieved with fewer parameters for sensor data, which typically have lower dimensionality than image data. Reducing the number of parameters would decrease memory requirements and computational complexity by reducing the model size. We evaluated the performance of two MLP-based models, MLP-Mixer and gMLP, by reducing the values of hyperparameters in their MLP layers from those proposed in the respective original papers. The results of this study suggest that the performance of MLP-based models is positively correlated with the number of parameters. Furthermore, these MLP-based models demonstrate improved computational efficiency for specific HAR tasks compared to representative CNNs.

Can a Cochlear Implant Be Used as an Electrical Impedance Tomography Device?

The imaging of the live cochlea is a challenging task. Regardless of the quality of images obtained from modern clinical imaging techniques, the internal structures of the cochlea mainly remain obscured. Electrical impedance tomography (EIT) is a safe, low-cost alternative medical imaging technique with applications in various clinical scenarios. In this article, EIT is investigated as an alternative method to image and extract the centre of gravity of the modiolus invivo. This information can be used to augment present postoperative medical imaging techniques to investigate the cochlea. The cochlear implant EIT system was simulated by modelling user-specific electrode array trajectories within a simple conductive medium containing an inhomogeneity representing the modiolus. The method included an adapted adjacent stimulation protocol for data collection. For the image reconstruction, NOSER and Tikhonov priors were considered. A parameter analysis was conducted to find the most robust combination of image priors and hyperparameters for this application. The cochlear implant EIT methodology was validated at different noise levels for four electrode array trajectories. Comparing the NOSER and Tikhonov priors, it was observed that the NOSER prior exhibits superior centre of gravity localisation performance in cochlear implant EIT image reconstruction for different noise levels and user-dependent variability in electrode array trajectories. Image reconstruction, using a NOSER prior at a hyperparameter value of approximately 0.001, resulted in an average centre of gravity localisation error of less than 4% for all electrode array trajectories using difference imaging and less than 5.5% using absolute imaging.

Multi-Criteria Decision Support System for Predicting Financial Futures Using Ensemble of Deep Learning Algorithms with Heuristic Search Mechanisms

Financial markets are an intricate dynamic system. The difficulty comes from the contact among a market and its applicants, which means, the integrated consequence of the activities of whole applicants decides the market trend, while the market trend disturbs the actions of applicants. These linked interactions make financial markets keep developing. Financial markets are interchange financial instruments like savings certificates, bonds, stocks, and much more. Particularly in stocks, because variations in stock prices are inclined by numerous factors, with economic cycles, financial trends, financial structure, and other macro issues, as well as industry growth, listed businesses’ financial quality. In the last few years, deep learning (DL) and machine learning (ML) techniques have been very effective in predicting financial futures. This study develops a Multi-Criteria Decision Support System for Predicting Financial Futures Using Ensemble of Deep Learning Algorithms with Heuristic Search Mechanisms (MDSSPFF-EDLAHS) model. The main intention of the MDSSPFF-EDLAHS method is to predict future of finances using advanced ensemble models. At first, the data normalization stage applies min-max normalization for transforming input data into a beneficial format. Besides, the ensemble of deep learning models namely variational auto encoder (VAE), bidirectional long short-term memory (Bi-LSTM) technique, and dueling double deep Q-network (DDQN) system have been executed for the prediction of financial futures. At last, the spider wasp optimization (SWO) algorithm adjusts the hyperparameter values of the ensemble models optimally and outcomes in greater prediction performance. The experimental evaluation of the MDSSPFF-EDLAHS is examined on a benchmark dataset. The extensive outcomes highlight the significant solution of the MDSSPFF-EDLAHS approach to the financial future predicting process

Optimizing Financial Fraud Detection: Understandings from Variable Selection with Neutrosophic Vague Soft Set

Neutrosophy is the neutralities study and prolongs the discussion of the truth of opinions. Neutrosophic logic might be used in all sectors, to provide the solution for the indeterminate challenges. Some real-time data experience issues like inconsistency, incompleteness, and indeterminacy. A fuzzy set (FS) offers an uncertain solution, and an intuitionistic fuzzy set (IFS) processes partial data, but both fail to handle uncertain data. Financial fraud, believed as a deceptive strategy to gain financial assistance, has recently become a common threat in organizations and companies. Traditional methods namely manual inspections and verifications are costly, time-consuming, and imprecise to identify such fraudulent actions. With the development of artificial intelligence (AI), machine learning (ML)-based algorithms are applied logically to identify fraud transactions by investigating a larger amount of financial data. Therefore, the study offers an Optimizing Financial Fraud Detection using Bayesian Optimization and Variable Selection with Neutrosophic Vague Soft Set (OFFDBO-VSNVS) Algorithm. The OFFDBO-VSNVS model presents an optimized framework for fraud detection by integrating advanced variable selection techniques and classification models. Initially, the OFFDBO-VSNVS technique applies the Z-score data normalization technique to transform input data into a compatible layout. Next, the grey wolf optimizer (GWO)--based feature selection to effectively reduce dimensionality and highlight the most relevant features. For the classification and detection of financial fraud, the neutrosophic vague soft set (NVS) model can be employed. Eventually, the Bayesian optimization (BO) model adjusts the hyperparameter values of the NVS algorithm optimally and outcomes in greater classification performance. The stimulated outcome study of the OFFDBO-VSNVS model occurs and the outcomes are examined in terms of changing features. The experimental study represented the superiority of the OFFDBO-VSNVS method across the existing state-of-the-art methods

Boosting Financial Risk Prediction Model Using Attention Mechanism Based Recurrent Neural Networks with Red‐Tailed Hawks Algorithm

The systemic prediction of financial risk issues has become a main attention in the area of finance. Financial risk is the main likelihood that stockholders will lose currency after they finance a business that has debt if the business flow of cash demonstrates insufficient to see its economic requirements. The incorporation of deep learning (DL) methods into financial risk forecast and investigation has altered conventional techniques. While traditional quantitative systems often trust basic metrics such as the highest reduction, the arrival of DL requires a more nuanced assessment, highlighting the model's generalization capability, particularly in market crises like stock market crashes. DL techniques are efficient in removing intricate patterns from massive data collections and become an effective model for forecasting financial trends. In this paper, we offer Boosting Financial Risk Prediction Model Using Attention Mechanism with Red‐Tailed Hawk (BFRPM-AMRTH) Algorithm. The presented BFRPM-AMRTH model aims to address the challenges of identifying and mitigating potential financial threats in a dynamic environment. Initially, the BFRPM-AMRTH technique applies the linear scaling normalization (LSN) data normalization technique to standardize the input features and ensure consistency across the dataset. In addition, the long short-term memory auto encoder with attention mechanism (LSTMA-AE) technique can be employed for classifying financial risks. Eventually, the red‐tailed hawk (RTH) algorithm adjusts the hyperparameter values of the LSTMA-AE algorithm optimally and outcomes in greater classification performance. To ensure the improved performance of BFRPM-AMRTH system, a huge range of simulation studies has been achieved and the obtained outcomes establish the advancement of the BFRPM-AMRTH system over the existing techniques

Enhancing Cybersecurity Attack Detection Using Multiplayer Battle Game Optimizer with Hybridization of Deep Learning Models

Cybersecurity is advancing and the rate of cybercrime, which is always rising. Advanced attacks are measured as the novel normal as they are one of the more normal and extensive. Cybersecurity threats have risen promptly in many areas like healthcare, smart homes, energy, automation, agriculture, and industrial processes. An intrusion detection system (IDS) discovers intrusions by analyzing attack designs or mining signatures from system packets. To assess an IDS model, use Machine Learning (ML) and deep learning (DL) approaches for recognizing data traffic into malicious and healthy. ML and DL techniques has earned an extensive interest on countless applications and domains of study, mostly in Cybersecurity. With computing power and hardware becoming more available, ML and DL systems can be employed in order to classify and analyze corrupt actors from a massive group of accessible data. This manuscript presents an Enhancing Detection of Cybersecurity Attack Using Multiplayer Battle Game Optimizer with Hybrid Deep Learning (EDCA-MBGOHDL) technique. The main intention of the EDCA-MBGOHDL technique is to provide a robust framework for cyberattack detection using deep learning integrated with a hyperparameter tuning approach. At first, the feature selection process is implemented by applying improved Harris hawk optimization (IHHO) algorithm for ensuring that only the most relevant features are fed into the model. Furthermore, the hybrid of convolutional neural network, bidirectional long short-term memory and attention mechanism (CNN-BiLSTM-AM) model is employed for the classification of cybersecurity threats. Eventually, the multiplayer battle game optimizer (MBGO) algorithm adjusts the hyperparameter values of the CNN-BiLSTM-AM classifier optimally and outcomes in greater classification performance. The wide range of analysis of the EDCA-MBGOHDL technique takes place using a benchmark dataset. The outcomes pointed out the superior performance of the EDCA-MBGOHDL system across existing models

БАЙЕСОВСКАЯ ОПТИМИЗАЦИЯ МЕТОДА ИДЕНТИФИКАЦИИ МЕЗОМАСШТАБНЫХ ОКЕАНСКИХ ВИХРЕЙ В МОРЕ ЛАБРАДОР В ДАННЫХ ВИХРЕРАЗРЕШАЮЩЕГО МОДЕЛИРОВАНИЯ

Deep convection in the Labrador Sea has a significant role in the formation of the Northern Hemisphere climate. The eddy activity in the Labrador Sea, represented by Irminger rings, influences the spatial and temporal heterogeneity of the mixed layer depth. Automated eddy identification methods are widely used as a tool to study eddy activity in statistically significant samples. However, the most commonly used method for finding local extrema is highly dependent on a variety of parameters chosen by the author or the user of the method. In this paper, a new algorithm for the identification of mesoscale anticyclonic eddies in numerical simulation data is developed. Using Bayesian optimisation method, the optimal values of hyperparameters of the developed eddy identification algorithm were selected. The identification quality in the F1-score measure is improved to 0.232 compared to 0.352 in the baseline configuration.

Predicting age at first calving of dairy breed calves using whale optimization-based ensemble learning framework

Dairy product requirement and the demand–supply gap of milk in Ethiopia have been increasing at an alarming rate due to various factors such as shortage of animal’s feeds, feed staffs, feed costs, and poor genetic merits of the local breeds of the country. This problem can be lessened by selecting best breed and modern animal breeding facilities, which require technologies like big data analysis and machine learning. In this study, a prediction model that can predict age at first calving of weaned calves based on their pre-weaning and weaning parameters, including dam’s parity number, season of calving, birth weight, pre-weaning health status, pre-weaning average daily weight gain (ADG), weaning age and weaning weight is developed. Primary data collected by Ardayta Dairy Research Centre; Ethiopia is used for this research. First, different pre-trained models developed using support vector regression (SVR), Linear support vector regression (LSVR) and Nu support vector regression (NuSVR) techniques with their default hyperparameter values in which SVR performed best. Second, a model was developed by tuning hyperparameters of SVR including kernel function, regularization (C-parameter) and gamma parameters, and it resulted in an accuracy of 96.46%. Next, Whale optimization technique is used to select the optimized features of the dataset. Furthermore, an ensemble of SVR, LSVR, NuSVR is designed, and the framework is trained by optimized features of data. The designed model achieved an accuracy of 98.3% superseding the other combinations.

Exploring the Versatility of Spiking Neural Networks: Applications Across Diverse Scenarios.

In the last few decades, Artificial Neural Networks have become more and more important, evolving into a powerful tool to implement learning algorithms. Spiking neural networks represent the third generation of Artificial Neural Networks; they have earned growing significance due to their remarkable achievements in pattern recognition, finding extensive utility across diverse domains such as e.g. diagnostic medicine. Usually, Spiking Neural Networks are slightly less accurate than other Artificial Neural Networks, but they require a reduced amount of energy to perform calculations; this amount of energy further reduces in a very significant manner if they are implemented on hardware specifically designed for them, like neuromorphic hardware. In this work, we focus on exploring the versatility of Spiking Neural Networks and their potential applications across a range of scenarios by exploiting their adaptability and dynamic processing capabilities, which make them suitable for various tasks. A first rough network is designed based on the dataset's general attributes; the network is then refined through an extensive grid search algorithm to identify the optimal values for hyperparameters. This dual-step process ensures that the Spiking Neural Network can be tailored to diverse and potentially very different situations in a direct and intuitive manner. We test this by considering three different scenarios: epileptic seizure detection, both considering binary and multi-classification tasks, as well as wine classification. The proposed methodology turned out to be highly effective in binary class scenarios: the Spiking Neural Networks models achieved significantly lower energy consumption compared to Artificial Neural Networks while approaching nearly 100% accuracy. In the case of multi-class classification, the model achieved an accuracy of approximately 90%, thus indicating that it can still be further improved.

Comparison Between LSTM, GRU and VARIMA in Forecasting of Air Quality Time Series Data

Air quality forecast is essential in alerting the public, especially those who have respiratory diseases, to take necessary precautions beforehand. The public can be forewarned of any worsening of air quality and be aware of the importance of reducing air pollution. In recent years, forecasting techniques based on deep learning algorithms such as recurrent neural network (RNN) have seen improvements in both accuracy and execution speed. Long short-term memory (LSTM) network and gated recurrent unit (GRU) are among the most popular variants of RNN. In this study, the hourly PM2.5 concentrations at five selected air quality monitoring stations, provided by the Department of Environment Malaysia, are forecasted using LSTM, GRU and vector autoregressive integrated moving average (VARIMA) models respectively. Data containing missing, negative and zero values are pre-processed using an interpolation technique before being split into training and test sets on an 80:20 ratio basis. Optimal combinations of hyperparameter values are selected via manual tuning based on the 10-fold growing window cross-validation results. The model performance is evaluated based on RMSE, MAE and MAPE. The results demonstrate that neural network models significantly outperform the multivariate time series model in which the LSTM and GRU models have comparable performance in forecasting the hourly PM2.5 concentration, with a slightly better prediction in the west coast region for LSTM and the east coast region for GRU. However, due to the complex architecture of neural networks, the computational time to train both LSTM and GRU models is three times longer than that for VARIMA. Additionally, it is observed that a higher percentage of interpolated values leads to lower prediction errors.

The effect of different approaches to determining the regularization parameter of bayesian LASSO on the accuracy of genomic prediction.

Using dense genomic markers opens up new opportunities and challenges for breeding programs. The need to penalize marker-specific regression coefficients becomes particularly important when dense markers are available. Therefore, fitting the marker effects to observations using a regularization technique, such as Bayesian LASSO (BL) regression, is of greatinteresting. When the Laplace prior distribution is applied to the regression coefficients, BL can be interpreted as a regularization of the norm based on the Bayesian approach. A critical issue is the appropriate selection of hyperparameters values in the prior distributions of regularization techniques, asthese values essentiallycontrol the sparsity in the estimated model. The purpose of this study was toevaluate different approaches for selecting the regularization parameter in BL, based on fully Bayesian approaches-such as gamma prior (BL_Gamma), beta prior (BL_Beta) and fixed prior (BL_Fixed) as well as data-driven approaches like cross-validation based on mean square error (BL_CV_MSE) and prediction accuracy (BL_CV_PA). Additionally, information-criteria-based methods including Akaike's information criterion (BL_AIC), Bayesian information criterion (BL_BIC) and Deviance information criterion (BL_DIC), were explored. For this purpose, a genome containing eight chromosomes (each 1 Morgan in length) with 100 randomly distributed quantitative trait loci was simulated. The studied scenarios were as follows: Scenario 1involved4000 markers and heritability of 0.2, scenario 2involved 4000 markers and heritability of 0.6, scenario 3involved 16,000 markers and heritability of 0.2; and scenario 4involved16,000 markers and heritability of 0.6. The results showed that among thefully Bayesian and cross-validation approaches, BL_Gamma, BL_Beta, and BL_CV_MSE provided the highest prediction accuracy (PA) in scenario 1 and 3. With increased marker density and heritability (scenario 4), the cross-validation approaches performed slightly better. The information-criteria-based methods demonstrated the lowest PA.Increasing heritability and marker density led to a decrease and an increase in the model penalty on the regression coefficients, respectively. The PA obtained in the target population ranged from 0.210 to 0.413in Scenario 1, 0.402to0.600in Scenario 2, 0.256to0.442in Scenario 3, and 0.478to 0.653 in Scenario 4.In generally, fully Bayesian approaches based on random priors for the regularization parameter are recommended for BL, as they provide acceptable PA with lower computational loads.

Short-Term Energy Generation Forecasts at a Wind Farm—A Multi-Variant Comparison of the Effectiveness and Performance of Various Gradient-Boosted Decision Tree Models

High-quality short-term forecasts of wind farm generation are crucial for the dynamically developing renewable energy generation sector. This article addresses the selection of appropriate gradient-boosted decision tree models (GBDT) for forecasting wind farm energy generation with a 10-min time horizon. In most forecasting studies, authors utilize a single gradient-boosted decision tree model and compare its performance with other machine learning (ML) techniques and sometimes with a naive baseline model. This paper proposes a comprehensive comparison of all gradient-boosted decision tree models (GBDTs, eXtreme Gradient Boosting (XGBoost), Light Gradient-Boosting Machine (LightGBM), and Categorical Boosting (CatBoost)) used for forecasting. The objective is to evaluate each model in terms of forecasting accuracy for wind farm energy generation (forecasting error) and computational time during model training. Computational time is a critical factor due to the necessity of testing numerous models with varying hyperparameters to identify the optimal settings that minimize forecasting error. Forecast quality using default hyperparameters is used here as a reference. The research also seeks to determine the most effective sets of input variables for the predictive models. The article concludes with findings and recommendations regarding the preferred GBDT models. Among the four tested models, the oldest GBDT model demonstrated a significantly longer training time, which should be considered a major drawback of this implementation of gradient-boosted decision trees. In terms of model quality testing, the lowest nRMSE error was achieved by the oldest model—GBDT in its tuned version (with the best hyperparameter values obtained from exploring 40,000 combinations).

Research on the Influence of Genetic Algorithm Parameters on XGBoost in Load Forecasting

Electric load forecasting is crucial in a power system comprising electricity generation, transmission, distribution, and retail. Due to its high accuracy, the ensemble learning method XGBoost has been widely applied in load forecasting. XGBoost's performance depends on its hyperparameters and the Genetic Algorithm (GA) is a commonly used algorithm in determining the optimal hyperparameters for this model. In this study, we propose a flowchart algorithm to investigate the impact of GA parameters on the accuracy of XGBoost models over the hyperparameter grid for load forecasting. The maximum load data of Queensland, Australia, are used for the research. The analysis of the results indicates that the accuracy of the XGBoost model significantly depends on the values of its hyperparameters. Using default hyperparameter values may lead to substantial errors in load forecasts, while selecting appropriate values for the GA to determine the optimal hyperparameters for the XGBoost model can significantly improve its accuracy.

Implication of ICWFPSO as Optimization Neural Network Algorithm on Sales Forecasting System

Predictive systems play a crucial role in a company's operations and strategy by aiding in more informed and data-driven decision-making and more effective planning and budgeting. It is possible to develop an intelligent system to perform forecasting. Neural networks offer significant advantages in forecasting systems due to their flexible modeling capabilities. However, this algorithm's fundamental weakness is the slow convergence rate and being trapped in a local minimum. To overcome it, this research is conducted to optimize the NN algorithm using the ICWFPSO to produce a forecasting algorithm with high accuracy and faster execution time using real e-commerce sales data for the past 7 years. Algorithm performance testing tests the Mean Absolute Error (MAE) value of the forecasting system using three scenarios: the NN forecasting algorithm, the NN optimized with ICWFPSO on the weight value, and the same scheme. Still, the optimized value is the hyperparameter value. ICWPSO has been shown to enhance the performance of PSO by tuning the inertia weight dynamically, which helps balance exploration and exploitation during the optimization process. The best prediction result is obtained when optimizing the hyperparameters using the ICWFPSO optimization technique compared to using traditional NN or optimizing weight value with ICWFPSO with the MAE value of 245.32958984375, and the best performance is obtained at iterations below 100. Further, gradient-based optimization methods might be generally preferred for their efficiency and effectiveness in handling large-scale neural network training.

Handwritten Hiragana Letter Detection Using CNN

Hiragana is one of the primary alphabets used in Japanese. Hiragana is a phonetic symbol; each letter represents one syllable. Hiragana letters are formed from curved lines and strokes. However, detecting Hiragana letters causes many errors because people still rely on their vision to detect the letters, especially people familiar with them for the first time. It will be difficult and not very clear to read the letters. Therefore, a Convolutional Neural Network (CNN) method is used to detect handwritten Hiragana letters and help people who first get to know Hiragana letters when the letters are too complicated for human eyes to detect. This research uses the YOLOv8 model as a handwritten Hiragana letter detection algorithm. The Hiragana letters to be detected are basic letters with 46 characters. This research uses the YOLOv8 model run on Google Collaboratory with the Ultralytics library version 8.0.20 using the Python programming language. The dataset is collected from the internet and annotated using the Roboflow framework and dataset 4600 Hiragana letters. From the test results, the best model is YOLOv8l using SGD optimizer and learning rate 0.01 with a precision value of 98.5%, recall value of 95.7%, f1-score value of 97.1%, and mAP value of 95.5%. In the future, we aim to expand the number of datasets and employ a broader range of hyperparameter values to optimize the classification precision and accuracy of the Hiragana Letter Detection system.