Hyperparameter Tuning Research Articles

Leveraging machine learning for preoperative prediction of supramaximal ablation in laser interstitial thermal therapy for brain tumors.

Maximizing safe resection in neuro-oncology has become paramount to improving patient survival and outcomes. Laser interstitial thermal therapy (LITT) offers similar survival benefits to traditional resection, alongside shorter hospital stays and faster recovery times. The extent of ablation (EOA) achieved using LITT is linked to patient outcomes, with greater EOA correlating with improved outcomes. However, the preoperative predictors for achieving supramaximal ablation (EOA ≥ 100%) are not well understood. By leveraging machine learning (ML) techniques, this study aimed to identify these predictors to enhance patient selection and therefore outcomes. The objective was to explore preoperative predictors for supramaximal EOA using ML in patients with glioblastoma. A retrospective study was conducted on the medical records of 254 patients undergoing LITT from 2013 to 2023 at a single tertiary center. Cohort criteria included age ≥ 18 years, diagnosis of glioblastoma, single-trajectory ablation, and a complete dataset. The study assessed preoperative clinical and radiographic factors, using EOA ≥ 100% as the endpoint. Five ML models were used: logistic regression, random forest (RF), gradient boosting, Gaussian naive Bayes, and support vector machine. Training and testing cohorts were subsequently assessed across ML models with fivefold cross-validation. Models were optimized using hyperparameter tuning. Performance was primarily quantified using the area under the curve (AUC) of the receiver operating characteristic curve. The final cohort consisted of 72 patients. Among the ML models, RF achieved the highest AUC (mean ± SD 0.94 ± 0.06). The leading models identified that lower preoperative volume, history of prior radiation therapy, history of prior craniotomy, preoperative neurological deficits, history of preoperative seizures, and distance from intracranial heat sinks were predictive of successful ablations in patients. Additionally, RF had the best mean metrics: accuracy 0.88, precision 0.87, specificity 0.87, and sensitivity 0.89. This is the first study to investigate the role of ML for optimizing ablation volumes in LITT. These ML models suggest that low preoperative volumes, previous craniotomy, previous radiation therapy, no previous neurological deficits, larger catheter-heat sink distance, and the presence of preoperative seizures are important prognostic factors for predicting successful supramaximal ablations with LITT.

Optimizing SMS Spam Detection Using Machine Learning: A Comparative Analysis of Ensemble and Traditional Classifiers

With the rapid rise of mobile communication, Short Message Service (SMS) has become an essential platform for transmitting information. However, the growing volume of unsolicited and harmful spam messages presents significant challenges for both users and mobile network operators. This study explores the effectiveness of various machine learning models, including Random Forest, Gradient Boosting, AdaBoost, Support Vector Machine (SVM), Logistic Regression, and an Ensemble Voting Classifier, in detecting SMS spam. A dataset containing 5,572 SMS messages, labeled as either spam or ham (legitimate), was used to evaluate these models. Hyperparameter tuning was performed on each model to optimize accuracy, and the models were assessed using metrics such as precision, recall, F1-score, and accuracy. The results indicated that the SVM and Ensemble Voting Classifier achieved the highest performance, with accuracies of 0.9857 and 0.9848, respectively. Both models demonstrated superior recall for spam messages, making them highly effective for real-world spam detection systems. While Random Forest, Gradient Boosting, and AdaBoost also performed well, their slightly lower recall for spam suggests that they may misclassify some spam as legitimate messages. The study highlights the effectiveness of machine learning models in addressing the SMS spam problem, particularly when using ensemble methods. Future research should focus on addressing class imbalance and exploring deep learning approaches to further enhance model performance. These findings offer valuable insights for developing more accurate and scalable SMS spam detection systems.

A large scale multi institutional study for radiomics driven machine learning for meningioma grading

This study aims to develop and evaluate radiomics-based machine learning (ML) models for predicting meningioma grades using multiparametric magnetic resonance imaging (MRI). The study utilized the BraTS-MEN dataset’s training split, including 698 patients (524 with grade 1 and 174 with grade 2–3 meningiomas). We extracted 4872 radiomic features from T1, T1 with contrast, T2, and FLAIR MRI sequences using PyRadiomics. LASSO regression reduced features to 176. The data was split into training (60%), validation (20%), and test (20%) sets. Five ML algorithms (TabPFN, XGBoost, LightGBM, CatBoost, and Random Forest) were employed to build models differentiating low-grade (grade 1) from high-grade (grade 2–3) meningiomas. Hyperparameter tuning was performed using Optuna, optimizing model-specific parameters and feature selection. The CatBoost model demonstrated the best performance, achieving an area under the receiver operating characteristic curve (AUROC) of 0.838 [95% confidence interval (CI): 0.689–0.935], precision of 0.492 (95% CI: 0.371–0.623), recall of 0.838 (95% CI: 0.689–0.935), F1 score of 0.620 (95% CI: 0.495–0.722), accuracy of 0.729 (95% CI: 0.650–0.800), an area under the precision-recall curve (AUPRC) of 0.620 (95% CI: 0.433–0.753), and Brier score of 0.156 (95% CI: 0.122-0.200). Other models showed comparable performance, with mean AUROCs ranging from 0.752 to 0.784. The radiomics-based ML approach presented in this study showcases the potential for non-invasive and pre-operative grading of meningiomas using multiparametric MRI. Further validation on larger and independent datasets is necessary to establish the robustness and generalizability of these findings.

AutoML and Automated Data Science by Democratizing AI through End-to-End Automation

The rise of data-driven decision-making has led to a significant demand for data science and machine learning (ML) solutions across industries. However, developing these solutions requires extensive expertise in data preprocessing, feature engineering, model selection, hyperparameter tuning and evaluation. AutoML (Automated Machine Learning) and Automated Data Science (AutoDS) have emerged as transformative approaches that aim to democratize data science by automating the endto-end ML pipeline. This paper explores the foundational concepts of AutoML, highlighting key techniques and algorithms, such as neural architecture search (NAS), hyperparameter optimization and meta-learning. We delve into AutoDS's broader scope, which seeks to fully automate tasks from data acquisition to deployment. Real-world applications, such as predictive modeling, anomaly detection and time series forecasting, are examined to demonstrate the impact of these technologies. Additionally, the paper analyzes the current frameworks and platforms facilitating automation, including Auto-sklearn, Google AutoML and H2O.ai and evaluates their performance across different tasks. While the potential to accelerate data science workflows and make AI accessible to non-experts is evident, challenges remain, particularly regarding transparency, interpretability and ethical considerations in fully automated systems. This research provides insights into current trends, future opportunities and the transformative role of AutoML and AutoDS in driving innovation in the data science landscape.

Circular rubber aggregate CFST stub columns under axial compression: prediction and reliability analysis

Extensive studies support using steel tubes to enhance the structural integrity of rubber aggregate concrete (RBAC), namely RBAC-filled steel tubes (RCFST). However, current design codes for assessing the axial compressive behaviour of circular stub RCFST (CS-RCFST) columns are limited. Furthermore, there is a scarcity of studies focused on ensuring the structural safety of these columns. Based on an extensive experimental database comprising 145 columns, this study explores machine learning (ML) capabilities for predicting the axial strength of CS-RCFST columns, using six typical machine-learning models, i.e., symbolic regression (SR), XGBoost, CatBoost, random forest, LightGBM, and Gaussian process regression models. The hyperparameter tuning of the introduced ML models is performed using the Bayesian Optimization technique. The comparison results show that the CatBoost model is the most reliable and accurate ML model (R2 = 0.999 and 0.993 for the training and testing sets, respectively). In addition, a simple and practical design expression for CS-RCFST columns has been developed with acceptable accuracy based on the SR model (an average test-to-prediction ratio of 0.99 and CoV of 0.132). Meanwhile, the axial strength predicted by ML models was compared with two prominent practice codes (i.e., AISC360 and EC4). The comparison results indicated that the ML models could introduce a highly reliable and accurate approach over current design standards for strength prediction. Furthermore, a reliability analysis is conducted on two different ML models to evaluate the reliability of utilising ML models in practical design applications. This assessment involves identifying the statistical properties associated with the compressive strength of RBAC, as well as introducing the required resistance design factors aligned with the target reliability recommended by code standards.

Sine Cosine Algorithm Based on Optimal Convolutional Autoencoder for Intrusion Detection and Classification Models

Network security comprises a multifaceted method that aims to protect computer networks from malicious activities, unauthorized access, and data breaches. The security mechanism is Intrusion Detection which is an important constituent that is employed to monitor and analyse the network traffic for recognizing and responding to intrusive or suspicious behavior. Innovative methods such as deep learning (DL) are employed to enhance the effectiveness of Intrusion Detection Systems (IDSs). DL is extremely implemented for IDS owing to its proficiency for automatically learning and extracting complex patterns and features from massive and multifaceted network datasets. Neural network (NN) models, permit the system to distinguish between anomalous patterns and normal network behaviors, increasing the accuracy of intrusion detection. The flexibility of DL methods to emerging cyberattacks with their adeptness to handle large - scale and various data, positions them as a strong and efficient tool for proactive and intelligent intrusion detection in existing cybersecurity settings. This article presents a Sine Cosine Algorithm with Optimal Convolutional Autoencoder for Intrusion Detection and Classification (SCAOCAE - IDC) method. The developed SCAOCAE - IDC system presents a wide - ranging strategy to improve the precision and effectiveness of IDSs. The method combines diverse advanced mechanisms like Min - Max scalar normalization for data preprocessing, Sine Cosine Algorithm (SCA) for feature selection (FS), Convolutional Autoencoder (CAE) for better feature extraction and classification, and Heap - Based Optimization (HBO) for hyperparameter tuning. The Min - Max scalar makes sure of robust data normalization, SCA increasingly chooses main features, CAE capably captures complex patterns in the data, and HBO fine - tunes hyperparameters for improved system performance. By employing the synergistic combination of such modules, the presented SCAOCAE - IDC algorithm indicates considerable outcomes for increasing the reliability and accuracy of IDSs and classification systems.

The Investigation of Hyperparameters Influence on Fruit and Vegetable Image Recognition Performance Using SVM

This study aims to improve the efficiency of automatic classification and quality control of fruits and vegetables through image recognition technology, to achieve efficient and accurate intelligent sorting in agricultural production, reduce labor costs and improve market competitiveness. A high-quality image dataset of 36 fruit and vegetable categories from Kaggle is used in this study. The images in the dataset have been preprocessed to ensure that the data is suitable for the classification task and sets the stage for efficient training and evaluation of the model. Logistic regression was first used as the baseline model in order to compare the performance with the Support Vector Machine (SVM) model. Subsequently, hyperparameter tuning is performed to optimize the model to achieve the best cross-validation accuracy. Next, the SVM model is trained with the selected hyperparameters, and the training time is recorded. The performance of the model was evaluated in detail by confusion matrix and classification reports, and the test set was used for final validation to ensure that the model would also perform well on unseen data. The SVM model achieves an accuracy of 96% on both validation and test sets, which is a very good performance. The hyperparameters optimized by GridSearchCV (C = 10, gamma = 0.1, kernel = rbf) effectively improve the performance of the model, verifying that reasonable hyperparameter selection is crucial to the SVM model. These results show that the model has good generalization ability and potential to be applied to real-time classification tasks.

Enhancing Heart Disease Prediction through Machine Learning: A Comparative Analysis of Algorithm Generalization across Datasets with Various Distributions

Heart disease, due to its high prevalence and mortality, remains a key area of global research. Although traditional diagnostic methods are effective, they are often invasive and time-consuming, highlighting the need for non-invasive, AI-based approaches. A significant challenge in real-world applications is ensuring model generalization across different datasets, particularly when the datasets are small. In this study, the performance of machine learning models, including Decision Tree, Random Forest, Support Vector Machine (SVM), and Multi-Layer Perceptron (MLP), was evaluated on two distinct heart disease datasets with different distributions and relatively small sizes. Two datasets with varying distributions were used for training and testing, with the primary focus on assessing model generalization in crossdataset applications. It is shown in the results that, while the Decision Tree model performed best after hyperparameter tuning, the improvements in Random Forest and MLP were limited, and SVM exhibited a decline in performance after tuning in the cross-dataset task. It was found that grid search tuning has limitations in cross-dataset scenarios, especially with small datasets, where complex models are prone to overfitting. The study demonstrates that, with smaller datasets, simpler models like Decision Trees often adapt better to different datasets. Furthermore, transfer learning and domain adaptation techniques are suggested as crucial for improving model generalization. Future research should focus on employing these techniques to enhance the robustness and accuracy of heart disease prediction models across diverse datasets.

Artificial intelligence (AI)-enhanced electrocardiography: a machine-learning model for differential diagnosis between hypertrophic cardiomyopathy, cardiac amyloidosis and Anderson-Fabry disease

Abstract Background The differential diagnosis between Hypertrophic Cardiomyopathy (HCM) and phenocopies is challenging and relies on a comprehensive approach, which includes clinical and instrumental data, as well as laboratory exams and genetic testing. In this setting, 12-lead electrocardiogram (ECG) represents a valuable tool, but although typical patterns have been described, their diagnostic ability to discriminate between different etiologies is limited and requires considerable clinical experience. However, a correct diagnosis is of pivotal importance, allowing the prompt initiation of phenotype-specific therapies and consequently improving patient survival. Purpose Aim of this study was to develop a machine-learning model based on human-based ECG features to support the differential diagnosis between HCM, Cardiac Amyloidosis (CA) and Anderson-Fabry Disease (AFD). Methods We retrospectively examined ECGs of 200 patients, 50 affected by HCM, 100 affected by CA (50 AL CA and 50 transthyretin CA) and 50 with a diagnosis of AFD cardiomyopathy. The patients were then divided in sinus rhythm (SR) subgroup (158 patients) and atrial fibrillation (AF) subgroup (42 patients). For each subgroup, patients were randomly split into a training (80%) and a testing cohort (20%), and a random forest (RF) model was trained on the training cohort, with hyperparameter tuning via 10-fold cross validation. The resulting model was then tested on the testing cohort to assess its predictive accuracy. Results Both RF models (SR and AF) showed good discriminative performance in this multiclass classification task on the testing cohort [SR: accuracy 67%; multiclass area under the curve (AUC) 0.797, 95% confidence interval (CI) 0.472 -0.827; AF: accuracy 75%; multiclass AUC 0.792, 95% CI 0.349 – 0.968]. Figure 1 and figure 2 report for each model the rank of the most important variables for class prediction. Conclusions An AI-based ECG interpretation based on human-derived ECG features can predict the etiology underlying an hypertrophic phenotype, helping to perform a correct diagnosis and consequently to set a specific therapeutic process.

Enhancing drug-target interaction predictions in context of neurodegenerative diseases using bidirectional long short-term memory in male Swiss albino mice pharmaco-EEG analysis

Background and ObjectiveEmerging diseases like Parkinson or Alzheimer's, which are not curable, endanger human mental health and are challenging to research. Drug target interactions (DTI) are pivotal in the screening of candidate drugs and focus on a small pool of drug targets. Electroencephalogram shows the responses to psychotropic medicines in the brain bioelectric activity. Synaptic activity can be analyzed by using Local Field Potential recordings obtained from micro-electrodes implanted in the brain. The aim is to evaluate the effects of drug on brain bioelectric activity and increase the drug classification accuracy. The ultimate goal is to advance our understanding of how drugs affect synaptic activity and open the door to more focused treatment for neurodegenerative diseases. MethodsIn this study, Pharmaco-EEG recordings are processed using Advanced neural network models, particularly Convolutional Neural Networks, to assess the effects of medications. The five different medicines used in this study are Ephedrine, Fluoxetine, Kratom, Morphine, and Saline. The signals observed are local field potential signals. To overcome some limits of DTI prediction, we propose Bidirectional Long Short-Term Memory (LSTM) for the categorization of intracranial EEG (i-EEG) data, departing from standard approaches. Similar EEG patterns are presumably caused by drugs that work by homologous pharmacological pathways, producing similar psychotropic effects. To improve accuracy and reduce training loss, our study introduces a bidirectional LSTM model for classification along with Bayesian optimization ResultsHigh recall, precision, and F1-Scores, particularly a 95% F1-Score for morphine, ephedrine, fluoxetine, and saline, suggest good performance in predicting these drug classes. Kratom produces a somewhat lower recall of 94%, but a high F1-Score of 97% and perfect precision of 1.00. The weighted average F1-Score, macro average, and overall accuracy are all consistently high (around 97%), indicating that the model works well throughout the spectrum of drugs. ConclusionsImproved model performance was demonstrated by using a diversified dataset with five drug categories and bidirectional LSTM boosted with Bayesian optimization for hyperparameter tuning. From earlier limited-category models, it represents a substantial advancement.

The Development Mask R-CNN Model for Identification of Melon Plant Leaves and Branches

The quality of melons can be enhanced and optimized by pruning melon plants. Pruning is a removal process carried out on specific parts of the plant. Currently, melon plants are still pruned manually by farmers, but there are many drawbacks to this method. In this research, pruning is conducted on the branches and leaves of melon plants. Pruning can be facilitated with the assistance of a robot capable of recognizing leaves and branches. In this study, the method used to detect branches and leaves is the Mask Region-based Convolutional Neural Network (Mask R-CNN). Hyperparameter tuning technique is employed to obtain the best parameter values, including learning rate, weight decay, and learning momentum. Two scenarios are considered in this research, one with 10 epochs and the other with 30 epochs. The obtained Average Precision (AP) values at 10 epochs are 32.2% for leaf objects and 0% for branches. At 30 epochs, the AP values are 56.8% for leaf objects and 4.1% for branches. The mean Average Precision (mAP) is 16.1% for 10 epochs and 28.4% for 30 epochs.

Deep learning modeling to assess surface temperature and greenery impact on emergency room access

Abstract The escalating land surface temperature is a key environmental indicator with potential repercussions on human health. An artificial NN model was built to predict the number of ER accesses as a dependent variable, to establish the predictive power and interactions of land surface temperature and normalized differential vegetation index(NDVI), which is used to measure urban greenery.A map of Taranto was built by vector files on the Apulia region website and processed using R Studio 2023.06.2.The vector file containing the mapping of the city of Taranto was processed to obtain a 200mx200m grid of the entire surface.The Local Health Authority of Taranto provided indications of the residences of ER accesses, and coordinates were merged to gridded vector file to obtain the average of ER accesses and the socio demographic features per single grid unit. NDVI and surface temperature were averaged for July 2023 using images from Landsat 8(30m) and merged to the vector providing an average value of temperature and NDVI index per single quadrant. The dataset was divided into two db according to an 80/20 ratio respectively for the training and test dataset.A NN model was built on dichotomous value (above/below median) for each quadrant (6513 units) with hyperparameter tuning technique using the accuracy to select the best model.A MOC model was built to reduce the probability of NN detection by at least 40% in a randomly selected subject.A MOC model was built considering sociodemographic variables as fixed features and an epsilon of 0 for 175 generations and an ICE curves population initialization strategy. NNET accuracy value was 0.849 and 0.841 in training and testing, suggesting a good fitting of the model.Relative change plots were adopted to show the amount and the direction of generated counterfactuals, to reduce grid unit prediction to be over median.LST decreasing and NDVI score increasing were associated with a reduction of classification probability of grid unit. Key messages • Rising land surface temperature is a critical environmental indicator with potential impacts on human health. • Artificial neural network models perform well in predicting the number of ER accesses with respect to land surface temperature and NDVI, which is used to measure urban greenery.

A Comparative Study of Sensor Quantity and Data Granularity in AAL Systems for Activity Recognition

Abstract As populations age, the demand for non-intrusive elderly care solutions increases, highlighting the need for efficient AAL systems. Current research predominantly focuses on wearable sensors at high data frequencies. There is a critical gap in research in understanding the effectiveness of model performance using ambient sensors operating at varied granularities and with varied numbers of sensors. This study addresses this gap by investigating how sensor quantity and data frequency affect an artificial intelligence model’s ability to detect various activities of daily living. The methodology employs a quantitative, experimental design to systematically assess the performance of artificial intelligence models across sensor quantity and data frequencies. This assessment will be conducted through a multi-study approach involving different populations to ensure robust and generalizable findings. Each model’s efficacy will be evaluated using 5-fold cross-validation and GridSearchCV for rigorous hyperparameter tuning, employing diverse data aggregation and imputations techniques to maintain comprehensive analysis integrity. The primary goal of this research is to determine the optimal data granularity and number of sensors that maximize AI models’ ability to detect daily living activities while minimizing resource demands, thereby enhancing the sustainability and scalability of AAL systems. This work aims to advance the field of ambient sensing in elderly care, offering significant implications for designing and implementing future AAL technologies and potentially improving the quality of life for the elderly population. Key messages • The study explores the optimal configuration of data granularity and sensor quantity to maximize AI model efficiency in detecting activities of daily living among older adults. • The study investigates the impact of data frequency and sensor count on AI model performance in detecting daily activities, aiming to optimize Ambient Assisted Living systems for the elderly.

Research on match fluctuation prediction and strategy based on OFS algorithm

The aim of this paper is to construct a model for predicting on-field fluctuations in matches by using OFS algorithm, combined with random forest and lattice point search algorithms, and to analyse the key factors in order to improve the coach's decision-making ability in matches. Cross-validation and hyper-parameter tuning techniques were used during the model building process to find the best hyper-parameter combinations through lattice point search to optimise the model performance and generalisation ability. The study provides a detailed description of the model implementation process, including data processing, feature selection, parameter setting, model building, training and evaluation. Meanwhile, a series of targeted recommendations are proposed to address the variability of match fluctuations, including the analysis of past matches, the focus on recent performance, the assessment of psychological quality, the study of opponent's weaknesses, the flexible response to changes in matches and the maintenance of focus, with a view to helping athletes and coaches to formulate a more effective match strategy and to achieve better match results.

On the interpretability of the SVM model for predicting infant mortality in Bangladesh

BackgroundAlthough machine learning (ML) models are well-liked for their outperformance in prediction, greatly avoided due to the lack of intuition and explanation of their predictions. Interpretable ML is, therefore, an emerging research field that combines the performance and interpretability of ML models to create comprehensive solutions for complex decision-making analysis. Conversely, infant mortality is a global public health concern affecting health, social well-being, socio-economic development, and healthcare services. The study employs advanced interpretable ML techniques to anticipate and understand the factors affecting infant mortality in Bangladesh, overcoming the shortcomings of the conventional logistic regression (LR) model.MethodsBy utilizing the global surrogate model and local individual conditional expectation (ICE) interpretability technique, the interpretable support vector machine (SVM) has been used in this study to reveal significant characteristics of infant mortality using data from the Bangladesh Demographic and Health Survey (BDHS) 2017–18. To investigate intricate decision-making analysis of infant mortality, we adapted SVM and LR techniques with the hyperparameter tuning parameters. These models’ performances were initially assessed using the receiver operating characteristics (ROC) curve, run-time, and confusion matrix parameters with 100 permutations. Afterward, the SVM model’s model-agnostic explanation and the LR model’s interpretation were compared to enhance advanced comprehension for further insights.ResultsThe results of the 100 permutations demonstrated that the LR model (Average: accuracy = 0.9105, precision = NaN, sensitivity = 0, specificity = 1, F1-score = 0, area under the ROC curve (AUC) = 0.6780, run-time = 0.0832) outperformed the SVM model (Average: accuracy = 0.8470, precision = 0.1062, sensitivity = 0.0949, specificity = 0.9209, F1-score = 0.1000, AUC = 0.5632, run-time = 0.0254) in predicting infant mortality, but the LR model had a slower run-time and it was unable to predict any positive cases. The interpretation of LR analysis revealed that infant mortality rates decrease when mothers give birth after over two years, with higher educational attainment, overweight or obese mothers, working mothers, and families with polluted cooking fuel having lower rates. The local ICE interpretability technique, which depicts individual influences on the average likelihood of dying within the first birthday, explores the interpretable SVM model that mothers with normal BMIs, giving birth within two years, using less polluted cooking fuel, working mothers, and having male infant were more likely to experience infant death. The interpretable SVM model based on the global surrogate model also reveals that working mothers who used polluted cooking fuel at home and working women who used less polluted cooking fuel but had a longer period between pregnancies than two years would have higher infant death rates. Even among non-working mothers who used polluted cooking fuel and gave birth within two years of the preceding one, infant death rates were higher.ConclusionsThe interpretable SVM model reveals global interpretations help clinicians understand the entire conditional distribution, while local interpretations focus on specific instances, providing different insights into model behavior. Interpretable ML models aid policymakers, stakeholders, and families in understanding and preventing infant deaths by improving policy-making strategies and establishing effective family counseling services.

Optimizing crop monitoring: mapping cultivation stages and types with sentinel-1/2 and random forest algorithm

ABSTRACT Crop-type mapping is essential for agricultural monitoring, improving crop yield models, and supporting sustainable land management. Studying crop phenology, which includes the timing from germination to maturity, is equally important for precise agricultural practices. However, maps detailing crop phenology are scarce. This study addresses this gap by mapping both crop types and cultivation time, offering valuable insights for agriculture applications. The research integrates Sentinel-1/2 data, NDVI phenology curves, and machine learning (ML) algorithms through Google Earth Engine to map crops in El-Behera Governorate, Egypt. By stratifying the region into different soil types, it accounts for various agro-environmental conditions. Field visits in July 2022 provided crop-type information, and Sentinel-2 NDVI curves were used to determine cultivation time. The random forest (RF) classifier, known for its accuracy, was employed, achieving an overall accuracy (OA) of 97.4% for crop type and 95.6% for cultivation time, with high Kappa coefficients. Feature importance analysis highlighted the significance of specific spectral bands, particularly in the Red Edge region, for classification accuracy. Hyperparameter tuning across different zones demonstrated the need for customized tuning. This study enhances understanding of agro-environmental variability’s impact on crop classification and shows the potential of the RF algorithm in advancing crop mapping, suggesting further studies on how shifts in cultivation time could affect crop yield.

Enhancing Machine Learning Models Through PCA, SMOTE-ENN, and Stochastic Weighted Averaging

Predicting survival outcomes in critical accidents has been a focal point in machine learning research. This study addresses several limitations of existing methods, including insufficient management of data imbalance, lack of emphasis on hyperparameter tuning, and proneness to overfitting. Many existing models struggle to generalize effectively on imbalanced datasets or depend on default hyperparameter settings, resulting in biased predictions. By integrating Principal Component Analysis (PCA), hyperparameter optimization, and resampling methods, as well as combining Edited Nearest Neighbors (ENN) with the Synthetic Minority Oversampling Technique (SMOTE), the model significantly improves predictive accuracy and model generalization. An ensemble model combining seven machine learning algorithms—Logistic Regression, Support Vector Machine, KNN, Random Forest, XGBoost, LightGBM, and CatBoost—was applied to predict survival outcomes. Stochastic Weighted Averaging (SWA) was applied to mitigate overfitting and enhance generalization. The accuracy increased from 91.97% to 94.89% after SWA was applied in this specific scenario. The combination of PCA-based dimensionality reduction, hyperparameter tuning, and resampling techniques (ENN + SMOTE) ensured the model handled data imbalance and optimized predictive accuracy. The final model demonstrated excellent performance, with Area Under the Curve (AUC) and Average Precision (AP) values both reaching 0.98, indicating high accuracy and precision. These improvements were validated using the Titanic dataset in a binary classification problem of predicting passenger survival. The results emphasize that ensemble learning, enhanced by SWA, offers a powerful framework for handling imbalanced and complex datasets, providing significant advancements in predictive modeling accuracy. This study provides insights into how machine learning techniques can be effectively combined to solve classification challenges in real-world scenarios.

Hyperparameter Tuning of Load-Forecasting Models Using Metaheuristic Optimization Algorithms—A Systematic Review

Load forecasting is an integral part of the power industries. Load-forecasting techniques should minimize the percentage error while prediction future demand. This will inherently help utilities have an uninterrupted power supply. In addition to that, accurate load forecasting can result in saving large amounts of money. This article provides a systematic review based on the Preferred Reporting Items for Systematic Review and Meta-Analyses (PRISMA) framework. This article presents a complete framework for short-term load forecasting using metaheuristic algorithms. This framework consists of three sub-layers: the data-decomposition layer, the forecasting layer, and the optimization layer. The data-decomposition layer decomposes the input data series to extract important features. The forecasting layer is used to predict the result, which involves different statistical and machine-learning models. The optimization layer optimizes the parameters of forecasting methods to improve the accuracy and stability of the forecasting model using different metaheuristic algorithms. Single models from the forecasting layer can predict the results. However, they come with their limitations, such as low accuracy, high computational burden, stuck to local minima, etc. To improve the prediction accuracy, the hyperparameters of these models need to be tuned properly. Metaheuristic algorithms cab be used to tune these hyperparameters considering their interdependencies. Hybrid models combining the three-layer methods can perform better by overcoming the issues of premature convergence and trapping into a local minimum solution. A quantitative analysis of different metaheuristic algorithms and deep-learning forecasting methods is presented. Some of the most common evaluation indices that are used to evaluate the performance of the forecasting models are discussed. Furthermore, a taxonomy of different state-of-the-art articles is provided, discussing their advantages, limitations, contributions, and evaluation indices. A future direction is provided for researchers to deal with hyperparameter tuning.

ENHANCING NATURAL LANGUAGE PROCESSING USING WALRUS OPTIMIZER WITH SELF-ATTENTION DEEP LEARNING MODEL IN APPLIED LINGUISTICS

Sentiment analysis (SA) is the popular natural language processing (NLP) problem utilized for analyzing texts, including uploaded reviews and user posts on e-commerce portals, forums, and social media platforms, regarding their opinions about the event or person, product, and service. The SA task comprises analyzing text to determine whether the sentiment expressed is negative, positive, or neutral, aiming for precise subjective data analysis. The deep learning (DL) technique enables various architectures to model SA tasks and has surpassed other machine learning (ML) techniques as the first method to perform SA tasks. Recent advancements in the DL model depart from the growing dominance of transformer language algorithms and convolutional neural network (CNN) and recurrent neural network (RNN) models. Utilizing pre-trained transformer language models to transfer knowledge to downstream tasks has emerged as a cutting-edge model in NLP. Therefore, the study designs a fractal walrus optimizer with self-attention DL-based SA in applied linguistics (WOSADL-SAAL) method. The WOSADL-SAAL method aims to recognize and classify the presence of sentiments in social media content. In the WOSADL-SAAL technique, data preprocessing is initially performed to transform the input dataset into a meaningful format. In addition, the WOSADL-SAAL technique uses the bag of words (BoW) model for extracting features. The self-attention bidirectional long short-term memory (SA-BiLSTM) network is applied to classify sentiment. The walrus optimizer (WO) model performs the hyperparameter tuning model to boost the detection outcomes of the SA-BiLSTM network. The performance evaluation of the WOSADL-SAAL method takes place under the benchmark SA dataset. The experimental analysis of the WOSADL-SAAL method exhibited a superior accuracy value of 99.07% and 99.24% under TUSA and IMDB datasets over existing approaches.

Feature Extraction and Identification of Rheumatoid Nodules Using Advanced Image Processing Techniques

Background/Objectives: Accurate detection and classification of nodules in medical images, particularly rheumatoid nodules, are critical due to the varying nature of these nodules, where their specific type is often unknown before analysis. This study addresses the challenges of multi-class prediction in nodule detection, with a specific focus on rheumatoid nodules, by employing a comprehensive approach to feature extraction and classification. We utilized a diverse dataset of nodules, including rheumatoid nodules sourced from the DermNet dataset and local rheumatologists. Method: This study integrates 62 features, combining traditional image characteristics with advanced graph-based features derived from a superpixel graph constructed through Delaunay triangulation. The key steps include image preprocessing with anisotropic diffusion and Retinex enhancement, superpixel segmentation using SLIC, and graph-based feature extraction. Texture analysis was performed using Gray-Level Co-occurrence Matrix (GLCM) metrics, while shape analysis was conducted with Fourier descriptors. Vascular pattern recognition, crucial for identifying rheumatoid nodules, was enhanced using the Frangi filter. A Hybrid CNN–Transformer model was employed for feature fusion, and feature selection and hyperparameter tuning were optimized using Gray Wolf Optimization (GWO) and Particle Swarm Optimization (PSO). Feature importance was assessed using SHAP values. Results: The proposed methodology achieved an accuracy of 85%, with a precision of 0.85, a recall of 0.89, and an F1 measure of 0.87, demonstrating the effectiveness of the approach in detecting and classifying rheumatoid nodules in both binary and multi-class classification scenarios. Conclusions: This study presents a robust tool for the detection and classification of nodules, particularly rheumatoid nodules, in medical imaging, offering significant potential for improving diagnostic accuracy and aiding in the early identification of rheumatoid conditions.