Fine-tuning Hyperparameters Research Articles

Optimizing pulmonary chest x-ray classification with stacked feature ensemble and swin transformer integration

This research presents an integrated framework designed to automate the classification of pulmonary chest x-ray images. Leveraging convolutional neural networks (CNNs) with a focus on transformer architectures, the aim is to improve both the accuracy and efficiency of pulmonary chest x-ray image analysis. A central aspect of this approach involves utilizing pre-trained networks such as VGG16, ResNet50, and MobileNetV2 to create a feature ensemble. A notable innovation is the adoption of a stacked ensemble technique, which combines outputs from multiple pre-trained models to generate a comprehensive feature representation. In the feature ensemble approach, each image undergoes individual processing through the three pre-trained networks, and pooled images are extracted just before the flatten layer of each model. Consequently, three pooled images in 2D grayscale format are obtained for each original image. These pooled images serve as samples for creating 3D images resembling RGB images through stacking, intended for classifier input in subsequent analysis stages. By incorporating stacked pooling layers to facilitate feature ensemble, a broader range of features is utilized while effectively managing complexities associated with processing the augmented feature pool. Moreover, the study incorporates the Swin Transformer architecture, known for effectively capturing both local and global features. The Swin Transformer architecture is further optimized using the artificial hummingbird algorithm (AHA). By fine-tuning hyperparameters such as patch size, multi-layer perceptron (MLP) ratio, and channel numbers, the AHA optimization technique aims to maximize classification accuracy. The proposed integrated framework, featuring the AHA-optimized Swin Transformer classifier utilizing stacked features, is evaluated using three diverse chest x-ray datasets-VinDr-CXR, PediCXR, and MIMIC-CXR. The observed accuracies of 98.874%, 98.528%, and 98.958% respectively, underscore the robustness and generalizability of the developed model across various clinical scenarios and imaging conditions.

Modified deep inductive transfer learning diagnostic systems for diabetic retinopathy severity levels classification

Diabetic Retinopathy (DR), a retinal illness that degenerates the retina and causes blindness, can be effectively treated with early detection and examination. Although expensive and unpleasant, manual retinography is the gold standard for DR diagnosis. Many Deep Learning (DL)-based algorithms have shown promise as deep learning (DR) diagnostic tools, performing similarly to human picture evaluation. These strategies work best with fine-tuned hyperparameters and huge databases. To develop Deep Inductive Transfer Learning-based Diagnostic Systems (DITL-DS) for Multiclass DR Severity Classification. The benchmark Indian Diabetic Retinopathy Image Dataset (IDRiD) is chosen first. Next, the dataset is pre-processed by proportionally scaling the image to 128*128 and using Convolutional Local Area Histogram Equalization (CLAHE). Third, the uneven class label dataset is balanced for efficiency. Fourth, a global average pooling layer and bottlenecking are added to five DITL models—Inception V3, ResNet34, EfficientNet B0, VGG16, and Xception—to compensate for information loss. Finally, we compare our improved approaches to five basic DITL classifiers. Our modified Xception model separates severity stages best, with an accuracy of 0.9840, precision of 0.9845, recall of 0.9650, AUC of 0.9979, and AUC-ROC of 0.9902. Precision, recall, and F1 scores are 0.97, 0.99, and 0.98 for stage0, 0.99, 1, and 1 for stage1, 0.98, 0.94, and 0.96 for stage2, 0.99, 0.99, and 0.99 for stage3, and 1, 1, and 1 for stage4. This may help clinicians objectively diagnose DR early. To ensure robustness, the model’s generalizability was validated using APTOS (Asia Pacific Tele-Ophthalmology Society).

An Improve Method for Plant Leaf Disease Detection and Classification using Deep Learning

In countries like India, whose important occupation is agriculture, face a huge loss when the crops get affected by any type of disease. These diseases attack the crops in various stages and can destroy the entire production. Since most diseases are transmitted from one crop to another there is an essential requirement to detect the type of disease in the early stage so that farmers can take the required action to “save the crops” and production. Early disease detection is one of the essential activities for enhancing agricultural productivity. Diseases spread very quickly in the parts of the leaves that affect the growth of the plants. Early detection is a challenging task as the symptoms are mild for accurate identification. This research paper presents an enhanced CNN based MCC-ECNN model with fine-tuned hyper-parameters and various batch sizes for accurate plant leaf disease classification.

Variational Autoencoders for Generative Drug-Gene Interactions in Periodontal Bone Resorption.

Introduction Periodontal bone resorption is a significant dental problem causing tooth loss and impaired oral function. It is influenced by factors such as bacterial plaque, genetic predisposition, smoking, systemic diseases, medications, hormonal changes, and poor oral hygiene. This condition disrupts bone remodeling, favoring resorptive processes. Variational autoencoders (VAEs) can learn the distribution of drug-gene interactions from existing data, identify potential drug targets, and predict therapeutic effects. This study investigates the generation of drug-gene interactions in periodontal bone resorption using VAEs. Methods A bone resorptive drugs dataset was retrieved from Probes and Drugs and analyzed using Cytoscape (https://cytoscape.org/) and CytoHubba (https://apps.cytoscape.org/apps/cytohubba), powerful tools for studying drug-gene interactions in bone resorption. The dataset was then prepared for matrix representation, with normalized input data. It was subsequently divided into training, validation, and testing sets. We then built an encoder-decoder network, defined a loss function, optimized parameters, and fine-tuned hyperparameters. Using VAEs, we generated new drug-gene interactions, assessed model performance, and visualized the latent space with reconstructed drug-gene interactions for further insights. Results The analysis revealed the top hub genes in drug-gene interactions, including Matrix Metalloproteinase (MMP) 14, MMP 9, HIF1A, STAT1, MAPT, CAS9, MMP2, CASP3, MMP1, and MAK1. The VAE's reconstruction accuracy was measured using mean squared error (MSE), with an average squared difference of 0.077. Additionally, the KL divergence value was 2.349, and the average reconstruction log-likelihood was -246. Conclusion The generative variational encoder model for drug-gene interactions in bone resorption demonstrates high accuracy and reliability in representing complex drug-gene relationships within this context.

A tree-based automated machine learning approach of the obstructed view factor of thermal radiation in nuclear pebble beds

For the nuclear pebble bed of the high temperature gas-cooled reactor (HTGR), a tree-based automated machine learning approach is developed to discuss the complicated thermal radiation behaviors. The AutoML model for calculating the obstructed view factor between any two particles in the pebble bed includes the gradient boosting regression tree model with the fine-tuned hyperparameters and the analytical base model. Using the discrete packing of the HTR-PM nuclear reactor and the leaped Halton sequence, a large dataset of the view factor under different conditions is generated to train the AutoML model. On the same platform, numerical results show that AutoML model is approximately 5 × 105 times faster than the traditional approach. The Pareto front of the AutoML model indicates that the mean squared error decreases with the model complexity until it reaches the optimal solution. Then, the trained AutoML model is applied to the engineering pebble bed. It is shown that the average radiation exchange factor near the wall is lower than that in the bulk region. In addition, the packing height is also an important parameter for evaluating the radiative behaviors. In particular, when the height of the nuclear pebble bed is less than 20 times particle diameter, it is necessary to consider the contribution of the packing height to the radiation effective thermal conductivity.

KasNAT: Non-autoregressive machine translation for Kashmiri to English using knowledge distillation

Non-Autoregressive Machine Translation (NAT) represents a groundbreaking advancement in Machine Translation, enabling the simultaneous prediction of output tokens and significantly boosting translation speeds compared to traditional auto-regressive (AR) models. Recent NAT models have adeptly balanced translation quality and speed, surpassing their AR counterparts. The widely employed Knowledge Distillation (KD) technique in NAT involves generating training data from pre-trained AR models, enhancing NAT model performance. While KD has consistently proven its empirical effectiveness and substantial accuracy gains in NAT models, its potential within Indic languages has yet to be explored. This study pioneers the evaluation of NAT model performance for Indic languages, focusing mainly on Kashmiri to English translation. Our exploration encompasses varying encoder and decoder layers and fine-tuning hyper-parameters, shedding light on the vital role KD plays in facilitating NAT models to capture variations in output data effectively. Our NAT models, enhanced with KD, exhibit sacreBLEU scores ranging from 16.20 to 22.20. The Insertion Transformer reaches a SacreBLEU of 22.93, approaching AR model performance.

Optimizing echo state networks for continuous gesture recognition in mobile devices: A comparative study

Continuous gesture recognition can be used to enhance human-computer interaction. This can be accomplished by capturing human movement with the use of the Inertial Measurement Units in smartphones and using machine learning algorithms to predict the intended gestures. Echo State Networks (ESNs) consist of a fixed internal reservoir that is able to generate rich and diverse nonlinear dynamics in response to input signals that capture temporal dependencies within the signal. This makes ESNs well-suited for time series prediction tasks, such as continuous gesture recognition. However, their application has not been rigorously explored, with regard to gesture recognition. In this study, we sought to enhance the efficacy of ESN models in continuous gesture recognition by exploring diverse model structures, fine-tuning hyperparameters, and experimenting with various training approaches. We used three different training schemes that used the Leave-one-out Cross-validation (LOOCV) protocol to investigate the performance in real-world scenarios with different levels of data availability: Leaving out data from one user to use for testing (F1-score: 0.89), leaving out a fraction of data from all users to use in testing (F1-score: 0.96), and training and testing using LOOCV on a single user (F1-score: 0.99). The obtained results outperformed the Long Short-Term Memory (LSTM) performance from past research (F1-score: 0.87) while maintaining a low training time of approximately 13 seconds compared to 63 seconds for the LSTM model. Additionally, we further explored the performance of the ESN models through behaviour space analysis using memory capacity, Kernel Rank, and Generalization Rank. Our results demonstrate that ESNs can be optimized to achieve high performance on gesture recognition in mobile devices on multiple levels of data availability. These findings highlight the practical ability of ESNs to enhance human-computer interaction.

A Deep Learning Framework for Timely Bone Fracture Detection and Prevention

Accurate bone fracture identification remains critical in medical diagnostics, motivating researchers to investigate deep-learning systems to address this difficulty. The performance of convolutional neural networks (CNN) and region-based convolutional neural networks (RCNN) in fracture diagnosis is investigated in this paper. Algorithmic efficacy was investigated using a complete set of experiments involving various epochs, batch sizes, and optimization techniques (Adam and SGD). As discussed in the discussion, the results continuously highlight the superiority of the RCNN algorithm, which demonstrated amazing performance across many experimental settings. Algorithms trained using the Adam optimizer regularly demonstrated high levels of accuracy, precision, recall, and F1 score. Nonetheless, the effect of epoch counts and batch size on performance variability was seen, necessitating careful consideration to avoid overfitting and ensure generalization. These findings support careful algorithm selection guided by optimization techniques and fine-tuned hyperparameters. The RCNN algorithm's impressive results, proven by its constant superiority, underline its potential to revolutionize bone fracture diagnosis. Further research should focus on hyperparameter tuning and comprehensive validation across several datasets, fostering accurate and efficient solutions for bone fracture diagnoses in medical practice.

Optimizing the performance of convolutional neural network for enhanced gesture recognition using sEMG

Deep neural networks (DNNs) have demonstrated higher performance results when compared to traditional approaches for implementing robust myoelectric control (MEC) systems. However, the delay induced by optimising a MEC remains a concern for real-time applications. As a result, an optimised DNN architecture based on fine-tuned hyperparameters is required. This study investigates the optimal configuration of convolutional neural network (CNN)-based MEC by proposing an effective data segmentation technique and a generalised set of hyperparameters. Firstly, two segmentation strategies (disjoint and overlap) and various segment and overlap sizes were studied to optimise segmentation parameters. Secondly, to address the challenge of optimising the hyperparameters of a DNN-based MEC system, the problem has been abstracted as an optimisation problem, and Bayesian optimisation has been used to solve it. From 20 healthy people, ten surface electromyography (sEMG) grasping movements abstracted from daily life were chosen as the target gesture set. With an ideal segment size of 200 ms and an overlap size of 80%, the results show that the overlap segmentation technique outperforms the disjoint segmentation technique (p-value < 0.05). In comparison to manual (12.76 ± 4.66), grid (0.10 ± 0.03), and random (0.12 ± 0.05) search hyperparameters optimisation strategies, the proposed optimisation technique resulted in a mean classification error rate (CER) of 0.08 ± 0.03 across all subjects. In addition, a generalised CNN architecture with an optimal set of hyperparameters is proposed. When tested separately on all individuals, the single generalised CNN architecture produced an overall CER of 0.09 ± 0.03. This study's significance lies in its contribution to the field of EMG signal processing by demonstrating the superiority of the overlap segmentation technique, optimizing CNN hyperparameters through Bayesian optimization, and offering practical insights for improving prosthetic control and human–computer interfaces.

Circ2DGNN: circRNA-disease Association Prediction via Transformer-based Graph Neural Network.

Investigating the associations between circRNA and diseases is vital for comprehending the underlying mechanisms of diseases and formulating effective therapies. Computational prediction methods often rely solely on known circRNA-disease data, indirectly incorporating other biomolecules' effects by computing circRNA and disease similarities based on these molecules. However, this approach is limited, as other biomolecules also play significant roles in circRNA-disease interactions. To address this, we construct a comprehensive heterogeneous network incorporating data on human circRNAs, diseases, and other biomolecule interactions to develop a novel computational model, circ2DGNN, which is built upon a heterogeneous graph neural network. circ2DGNN directly takes heterogeneous networks as inputs and obtains the embedded representation of each node for downstream link prediction through graph representation learning. circ2DGNN employs a Transformer-like architecture, which can compute heterogeneous attention score for each edge, and perform message propagation and aggregation, using a residual connection to enhance the representation vector. It uniquely applies the same parameter matrix only to identical meta-relationships, reflecting diverse parameter spaces for different relationship types. After fine-tuning hyperparameters via five-fold cross-validation, evaluation conducted on a test dataset shows circ2DGNN outperforms existing state-of-the-art(SOTA) methods.

Utilizing Autoencoders for Analysis and Classification of Microscopic in Situ Hybridization Images

Currently, analysis of microscopic In Situ Hybridization (ISH) images is done manually by experts. Precise evaluation and classification of such microscopic images can ease experts' work and reveal further insights about the data. In this work, we propose a deep-learning workflow to detect and classify areas of microscopic images with similar levels of gene expression. Analysis of the data is done by employing a type of ANN – Deep Learning Autoencoders – suitable for unsupervised learning. The model's performance is optimised by balancing the latent layers' length and complexity and fine-tuning hyperparameters. The results are validated by adapting the mean-squared error (MSE) metric and comparison to expert's evaluation. Reconstruction of the whole-scale microscopic images is used to summarise and visualise the results.

Optimizing Photodegradation Rate Prediction of Organic Contaminants: Models with Fine-Tuned Hyperparameters and SHAP Feature Analysis for Informed Decision Making

Optimizing Photodegradation Rate Prediction of Organic Contaminants: Models with Fine-Tuned Hyperparameters and SHAP Feature Analysis for Informed Decision Making

KE: A Knowledge Enhancing Framework for Machine Learning Models.

Machine learning models are widely used in science and engineering to predict the properties of materials and solve complex problems. However, training large models can take days and fine-tuning hyperparameters can take months, making it challenging to achieve optimal performance. To address this issue, we propose a Knowledge Enhancing (KE) algorithm that enhances knowledge gained from a lower capacity model to a higher capacity model, enhancing training efficiency and performance. We focus on the problem of predicting the bandgap of an unknown material and present a theoretical analysis and experimental verification of our algorithm. Our experiments show that the performance of our knowledge enhancement model is improved by at least 10.21% compared to current methods on OMDB datasets. We believe that our generic idea of knowledge enhancement will be useful for solving other problems and provide a promising direction for future research.

Deep learning techniques for multi-class classification of asphalt damage based on hamburg-wheel tracking test results

In recent years, advancements in deep learning (DL) have been leveraged in civil engineering, but further exploration is necessary to apply DL techniques to asphalt research. These advances involve employing computer vision tasks and machine learning approaches to solve current challenges and develop innovative solutions for the conservation and monitoring of roads. In the laboratory, the Hamburg-Wheel Tracking (HWT) test simulates expected vehicle traffic and evaluates permanent deformation, such as rutting in asphalt mixtures. Current works focus on detecting the damages in the asphalt but do not classify the level of damage, which is the novelty proposed in the present work for rutting evaluation. This study aims to classify images of asphalt damage based on surface deformation caused by the HWT test. The dataset built and implemented in this study -called by the authors HWTBench2023- consists of 1198 images of asphalt samples subjected to controlled HWT conditions with varying levels of damage. The study employs a multi-class classification model based on convolutional neural networks (CNNs). The CNN was calibrated with transfer learning and fine-tuning hyperparameters. The outcomes demonstrate a high degree of accuracy, where the validation test showed an overall accuracy of 89 %, with F1-score values over 89 %. The model developed in this research identifies the presence or absence of damage and quantifies the damage level. In addition, the model was evaluated on a different dataset containing asphalt pavement photographs with rut damages. The model’s performance detecting rut damage on asphalt pavement under actual conditions indicates its adaptability to new images not included in the model’s learning stage. Therefore, this study improves methods for fast assessing rutting deterioration in asphalt pavements with transfer learning, provides accurate measurements of pavement deformation, and helps identify maintenance and rehabilitation needs.

Multivariate Multi-Step Long Short-Term Memory Neural Network for Simultaneous Stream-Water Variable Prediction

Implementing multivariate predictive analysis to ascertain stream-water (SW) parameters including dissolved oxygen, specific conductance, discharge, water level, temperature, pH, and turbidity is crucial in the field of water resource management. This is especially important during a time of rapid climate change, where weather patterns are constantly changing, making it difficult to forecast these SW variables accurately for different water-related problems. Various numerical models based on physics are utilized to forecast the variables associated with surface water (SW). These models rely on numerous hydrologic parameters and require extensive laboratory investigation and calibration to minimize uncertainty. However, with the emergence of data-driven analysis and prediction methods, deep-learning algorithms have demonstrated satisfactory performance in handling sequential data. In this study, a comprehensive Exploratory Data Analysis (EDA) and feature engineering were conducted to prepare the dataset, ensuring optimal performance of the predictive model. A neural network regression model known as Long Short-Term Memory (LSTM) was trained using several years of daily data, enabling the prediction of SW variables up to one week in advance (referred to as lead time) with satisfactory accuracy. The model’s performance was evaluated by comparing the predicted data with observed data, analyzing the error distribution, and utilizing error matrices. Improved performance was achieved by increasing the number of epochs and fine-tuning hyperparameters. By applying proper feature engineering and optimization, this model can be adapted to other locations to facilitate univariate predictive analysis and potentially support the real-time prediction of SW variables.

Predicting Compressive Strength of M20 Concrete with Partial Replacement of Coarse Aggregates by Nueva Ecija Sourced Recycled Portland Cement Concrete Pavement using XGBoost Algorithm

This research delves into the potential impact of replacing the coarse aggregates in M20 concrete with recycled concrete. The study aims to explore the possibility of recycling waste material from road construction to address sustainability challenges in the construction industry, especially in developing countries like the Philippines. The research adopts an experimental design involving preparing and testing recycled concrete aggregates, alongside sample preparation and testing. The study also employs the XGBoost machine learning algorithm to help predict the compressive strength of concrete. The XGBoost algorithm was trained using fine-tuned hyperparameters, and the model's evaluation was done using R-squared and Root Mean Squared Error. The research findings indicate a positive correlation between RCA content and compressive strength, with the average compressive strength of concrete mixtures increasing over time. The XGBoost outperforms the Multi Linear Regression model in predicting compressive strength, and incorporating RCA within the optimal range can enhance the strength properties of concrete. The study concludes that the XGBoost model with fine-tuned hyperparameters is reliable for predicting compressive strength, contributing to sustainable concrete production and reliable strength outcomes.

Using machine learning-based boosting algorithm to predict hepatotoxicity after local therapy and association with survival in patients undergoing liver directed therapy.

e16229 Background: Hepatocellular carcinoma typically arises in the setting of chronic liver disease. In carefully selected patients, liver directed therapy has been demonstrated to improve survival; however, this benefit from cancer specific survival may be offset by increased mortality from diminished hepatic functional reserve. Treatment-related hepatotoxicity also may vary between liver directed treatment modalities for a given tumor. Our aim was to determine if a machine learning approach could estimate hepatotoxicity prior to treatment. Methods: We identified all patients with hepatocellular carcinoma who underwent liver directed therapy at a large urban academic health system who also had post-treatment laboratory follow-up at 6 months. Patients were categorized as having worse liver function if they experienced a decline in albumin-bilirubin (ALBI) grade or more than 0.5 absolute decline in ALBI score at post treatment follow-up. A machine learning, extreme-gradient boosting (XGBoost) approach was employed to identify predictive features for hepatotoxicity using fine-tuned hyperparameters. Results: The study cohort consisted of 269 patients. 48 (18%) were categorized as having worsened hepatic function after liver directed therapy, with an average decline of -0.72 in ALBI score. Median survival after liver directed therapy was 5.8 years, with worsening hepatic function associated with a 5 month decline in overall survival. 158 (59%) patients received transarterial chemoembolization, 99 (37%) patients received stereotactic body radiotherapy and 12 (7%) patients received Y-90. A machine learning model was developed resulting in an Area Under the ROC Curve (AUC) of 0.894 in a 10-point decision tree model. On SHAP analysis, the features with the largest impact on hepatotoxicity prior to treatment was pretreatment T-stage followed by Child-Pugh Score. Conclusions: A machine learning model was developed to accurately classify patients who sustained treatment related hepatotoxicity. Significant decline in ALBI score or grade was associated with a worsening of overall survival. Future studies in larger prospective clinical trials utilizing this machine learning-based methodology of assessing liver dysfunction after focal therapy are required to validate these findings.

Hyperparameter Search for Machine Learning Algorithms for Optimizing the Computational Complexity

For machine learning algorithms, fine-tuning hyperparameters is a computational challenge due to the large size of the problem space. An efficient strategy for adjusting hyperparameters can be established with the use of the greedy search and Swarm intelligence algorithms. The Random Search and Grid Search optimization techniques show promise and efficiency for this task. The small population of solutions used at the outset, and the costly goal functions used by these searches, can lead to slow convergence or execution time in some cases. In this research, we propose using the machine learning model known as Support Vector Machine and optimizing it using four distinct algorithms—the Ant Bee Colony Algorithm, the Genetic Algorithm, the Whale Optimization, and the Particle Swarm Optimization—to evaluate the computational cost of SVM after hyper-tuning. Computational complexity comparisons of these optimization algorithms were performed to determine the most effective strategies for hyperparameter tuning. It was found that the Genetic Algorithm had a lower temporal complexity than other algorithms.

ENABLING FLEXIBLE AND ADAPTABLE NAVIGATION OF GROUND ROBOTS IN DYNAMIC ENVIRONMENTS WITH LIVE LEARNING

Federated learning is utilized for automated ground robot navigation, enabling decentralized training and continuous model adaptation. Strategies include hardware selection, ML model design, and hyperparameter fine-tuning. Real-world application involves optimizing communication protocols and evaluating performance with diverse network conditions. Federated learning shows promise for machine learning-based life learning systems in ground robot navigation. Research objective: to explore the use of federated learning in automated ground robot navigation and optimize the system for improved performance in dynamic environments. Materials and methods. The research utilizes federated learning to train machine learning models for ground robot navigation. Hardware selection, ML model design, and hyperparameter fine-tuning are employed. Communication protocols are optimized, and performance is evaluated using multiple gaming machine algorithms. Results. The results show that decreasing the learning rate and increasing hidden units improve model accuracy, while batch size has no significant impact. Communication protocols are evaluated, with Protocol A providing high efficiency but low security, Protocol B offering a balance, and Protocol C prioritizing security. Conclusion. The proposed approach using federated learning enables ground robots to navigate dynamic environments effectively. Optimizing the system involves selecting efficient communication protocols and fine-tuning hyperparameters. Future work includes integrating additional sensors, advanced ML models, and optimizing communication protocols for improved performance and integration with the control system. Overall, this approach enhances ground robot mobility in dynamic environments.

Increasing depth, distribution distillation, and model soup: erasing backdoor triggers for deep neural networks

Deep neural networks are vulnerable to backdoor attacks, in which the adversary injects a trigger embedded set into the training process. Inputs marked with the trigger provide incorrect predictions, whereas clean inputs remain unaffected. To erase the latent triggers in models, increasing depth, distribution distillation, and model soup (ID3MS), a defensive solution that operates without prior knowledge of triggers and relies on a small clean set is introduced. The depth of the backdoor model is increased by adding fully connected layer(s) at the penultimate layer. Without a classification layer, the original backdoor and increased depth models are considered as teacher and student, respectively. The student model applies distribution distillation to refit the distribution of the clean set and erase the backdoor triggers. The distilled student model is then recovered with the classification layer and model soup is used to ensemble a collection of models generated by various fine-tuning hyperparameters. The experimental results validate the superior performance of the ID3MS compared with existing defensive techniques against several attacks across datasets.