Hyperparameter Tuning Research Articles

Time series forecasting of price of the agricultural products using data science

Purpose. The purpose of our article is to research and forecast prices for agricultural products using the example of potato prices based on the most effective models using data science techniques. Methodology / approach. Various forecasting models are explored, starting from baseline models like decomposition and exponential smoothing models to more advanced techniques such as ARIMA, SARIMA, as well as deep learning models including neural network. The data is split into training and testing sets, and models are validated using cross-validation techniques and optimised through hyperparameter tuning. Model performance is evaluated using metrics such as MAE, MSE, RMSE, and MAPE. The selected model is then used to generate future price forecasts, with uncertainty quantified through confidence intervals. Results. The study successfully applied advanced data science techniques to forecast potato prices, leveraging a range of effective models. By analysing historical price data and using various forecasting methods, the research identified the most accurate models for predicting future price trends. The results demonstrate that the selected models can provide reliable forecasts. In particular, the results showed that the model could achieve good forecast results when applied to real problems and, thus, can be effectively used for forecasting tasks especially considering seasonality. In addition, it should be noted that the model has a higher prediction accuracy at the time intervals closest to the original data. The obtained results support using both models simultaneously for forecasting, which can compensate for the shortcomings of each of them. The models can be used separately, to more accurately predict the values for the required period, or a combination of them is also possible. Originality / scientific novelty. The study’s originality lies in development of methods for effectively accounting for seasonality in agricultural price data, such as using seasonal decomposition techniques or more advanced techniques that combine statistical and data science approaches. The novelty implies the implementation of real-time data processing and forecasting system allows for the timely prediction of price changes, enabling stakeholders to make more informed decisions. Practical value / implications. Forecasting potato prices holds significant practical value for various stakeholders. For farmers, accurate forecasts enable informed decisions on the optimal times to plant, harvest, and sell their crops, thereby optimising their profits. In the supply chain, distributors and retailers can use these forecasts to manage inventory more effectively and plan contracts, reducing waste and avoiding shortages. Policymakers benefit from forecasts by anticipating market fluctuations and stabilising prices, which supports both consumers and producers. For consumers, stable pricing ensures better budgeting and helps avoid sudden price spikes, making essential foods more affordable. Overall, accurate price forecasting enhances market efficiency by reducing uncertainty and aiding investors in managing risk.

Industrial activated sludge model identification using hyperparameter-tuned metaheuristics

This study focuses on the parameter estimation of an industrial activated sludge model using hyperparameter-tuned metaheuristic techniques. The data used in this study were collected on-site from a textile industry wastewater treatment plant. A Modified Activated Sludge Model (M-ASM) was the 'first-principle model’ selected and implemented with suitable assumptions. Advanced metaheuristic techniques, as Adaptive Tunicate Swarm Optimization (ATSO), Whale Optimization Algorithm (WOA), Rao-3 Optimization (Rao-3) and Driving Training Based Optimization (DTBO) were implemented. The hyperparameter tuning was performed with Bayesian Optimization (BO). Optimized metaheuristic algorithms were implemented for model-parameter identification. The Bayesian optimized Rao-3(BO-Rao-3) algorithm provided the best validation results, with a Mean Absolute Percentage Error (MAPE) value of 7.0141 and Normalized Root Mean Square Error (NRMSE) value of 0.2629. It also had the least execution time. BO-Rao-3 is 0.93% to 4.7% better than the other implemented hyperparameter-tuned metaheuristic techniques.

StackGridCov: a robust stacking ensemble learning-based model integrated with GridSearchCV hyperparameter tuning technique for mutation prediction of COVID-19 virus

StackGridCov: a robust stacking ensemble learning-based model integrated with GridSearchCV hyperparameter tuning technique for mutation prediction of COVID-19 virus

BayesAge 2.0: A Maximum Likelihood Algorithm to Predict Transcriptomic Age.

Aging is a complex biological process influenced by various factors, including genetic and environmental influences. In this study, we present BayesAge 2.0, an improved version of our maximum likelihood algorithm designed for predicting transcriptomic age (tAge) from RNA-seq data. Building on the original BayesAge framework, which was developed for epigenetic age prediction, BayesAge 2.0 integrates a Poisson distribution to model count-based gene expression data and employs LOWESS smoothing to capture non-linear gene-age relationships. BayesAge 2.0 provides significant improvements over traditional linear models, such as Elastic Net regression. Specifically, it addresses issues of age bias in predictions, with minimal age-associated bias observed in residuals. Its computational efficiency further distinguishes it from traditional models, as reference construction and cross-validation are completed more quickly compared to Elastic Net regression, which requires extensive hyperparameter tuning. Overall, BayesAge 2.0 represents a notable advance in transcriptomic age prediction, offering a robust, accurate, and efficient tool for aging research and biomarker development.

Automated gall bladder cancer detection using artificial gorilla troops optimizer with transfer learning on ultrasound images

The gallbladder (GB) is a small pouch and a deep tissue placed under the liver. GB Cancer (GBC) is a deadly illness that is complex to discover in an initial phase. Initial diagnosis can significantly enhance the existence rate. Non-ionizing energy, low cost, and convenience make the US a general non-invasive analytical modality for patients with GB diseases. Automatic recognition of GBC from US imagery is a significant issue that has gained much attention from researchers. Recently, machine learning (ML) techniques dependent on convolutional neural network (CNN) architectures have prepared transformational growth in radiology and medical analysis for illnesses like lung, pancreatic, breast, and melanoma. Deep learning (DL) is a region of artificial intelligence (AI), a functional medical tomography model that can help in the initial analysis of GBC. This manuscript presents an Automated Gall Bladder Cancer Detection using an Artificial Gorilla Troops Optimizer with Transfer Learning (GBCD-AGTOTL) technique on Ultrasound Images. The GBCD-AGTOTL technique examines the US images for the presence of gall bladder cancer using the DL model. In the initial stage, the GBCD-AGTOTL technique preprocesses the US images using a median filtering (MF) approach. The GBCD-AGTOTL technique applies the Inception module for feature extraction, which learns the complex and intrinsic patterns in the pre-processed image. Besides, the AGTO algorithm-based hyperparameter tuning procedure takes place, which optimally picks the hyperparameter values of the Inception technique. Lastly, the bidirectional gated recurrent unit (BiGRU) model helps classify gall bladder cancer. A series of simulation analyses were performed to ensure the performance of the GBCD-AGTOTL technique on the GBC dataset. The experimental outcomes inferred the enhanced abilities of the GBCD-AGTOTL in detecting gall bladder cancer.

Enhanced seagull optimization for enhanced accuracy in CUDA-accelerated Levenberg–Marquardt backpropagation neural networks for earthquake forecasting

Hyperparameter tuning is crucial for enhancing the accuracy and reliability of artificial neural networks (ANNs). This study presents an optimization of the Levenberg–Marquardt backpropagation neural network (LM-BPNN) by integrating an improved seagull optimization algorithm (ISOA). The proposed ISOA-LM-BPNN model is designed to forecast earthquakes in the Caribbean region. The study further explores the impact of data and model parallelism, revealing that hybrid parallelism effectively mitigates the limitations of both. This leads to substantial gains in throughput and overall performance. To address computational demands, this model leverages the compute unified device architecture (CUDA) framework, enabling hybrid parallelism on graphics processing units (GPUs). This approach significantly enhances the model’s computational speed. The experimental results demonstrate that the ISOA-LM-BPNN model achieves a 20% improvement in accuracy compared to four baseline algorithms across three diverse datasets. The integration of ISOA with LM-BPNN refines the neural network’s hyperparameters, leading to more precise earthquake predictions. Additionally, the model’s computational efficiency is evidenced by a 56% speed increase when utilizing a single GPU, and an even greater acceleration with dual GPUs connected via NVLink compared to traditional CPU-based computations. The findings underscore the potential of ISOA-LM-BPNN as a robust tool for earthquake forecasting, combining high accuracy with enhanced computational speed, making it suitable for real-time applications in seismic monitoring and early warning systems.

Development and validation of a machine learning-based framework for assessing metabolic-associated fatty liver disease risk

BackgroundThe existing predictive models for metabolic-associated fatty liver disease (MAFLD) possess certain limitations that render them unsuitable for extensive population-wide screening. This study is founded upon population health examination data and employs a comparison of eight distinct machine learning (ML) algorithms to construct the optimal screening model for identifying high-risk individuals with MAFLD in China.MethodsWe collected physical examination data from 5,171,392 adults residing in the northwestern region of China, during the year 2021. Feature selection was conducted through the utilization of the Least Absolute Shrinkage and Selection Operator (LASSO) regression. Additionally, class balancing parameters were incorporated into the models, accompanied by hyperparameter tuning, to effectively address the challenges posed by imbalanced datasets. This study encompassed the development of both tree-based ML models (including Classification and Regression Trees, Random Forest, Adaptive Boosting, Light Gradient Boosting Machine, Extreme Gradient Boosting, and Categorical Boosting) and alternative ML models (specifically, k-Nearest Neighbors and Artificial Neural Network) for the purpose of identifying individuals with MAFLD. Furthermore, we visualized the importance scores of each feature on the selected model.ResultsThe average age (standard deviation) of the 5,171,392 participants was 51.12 (15.00) years, with 52.47% of the participants being females. MAFLD was diagnosed by specialized physicians. 20 variables were finally included for analyses after LASSO regression model. Following ten rounds of cross-validation and parameter optimization for each algorithm, the CatBoost algorithm exhibited the best performance, achieving an Area Under the Receiver Operating Characteristic Curve (AUC) of 0.862. The ranking of feature importance indicates that age, BMI, triglyceride, fasting plasma glucose, waist circumference, occupation, high density lipoprotein cholesterol, low density lipoprotein cholesterol, total cholesterol, systolic blood pressure, diastolic blood pressure, ethnicity and cardiovascular diseases are the top 13 crucial factors for MAFLD screening.ConclusionThis study utilized a large-scale, multi-ethnic physical examination data from the northwestern region of China to establish a more accurate and effective MAFLD risk screening model, offering a new perspective for the prediction and prevention of MAFLD.

Unsupervised machine learning and cepstral analysis with 4D-STEM for characterizing complex microstructures of metallic alloys

Four-dimensional scanning transmission electron microscopy, coupled with a wide array of data analytics, has unveiled new insights into complex materials. Here, we introduce a straightforward unsupervised machine learning approach that entails dimensionality reduction and clustering with minimal hyperparameter tuning to semi-automatically identify unique coexisting structures in metallic alloys. Applying cepstral transformation to the original diffraction dataset improves this process by effectively isolating phase information from potential signal ambiguity caused by sample tilt and thickness variations, commonly observed in electron diffraction patterns. In a case study of a NiTiHfAl shape memory alloy, conventional scanning transmission electron microscopy imaging struggles to accurately identify a low-contrast precipitate at lower magnifications, posing challenges for microscale analyses. We find that our method efficiently separates multiple coherent structures while using objective means of determining hyperparameters. Furthermore, we demonstrate how the clustering result facilitates more robust strain mapping to provide immediate and quantitative structural insights.

Intelligence model on sequence-based prediction of PPI using AISSO deep concept with hyperparameter tuning process

Protein–protein interaction (PPI) prediction is vital for interpreting biological activities. Even though many diverse sorts of data and machine learning approaches have been employed in PPI prediction, performance still has to be enhanced. As a result, we adopted an Aquilla Influenced Shark Smell (AISSO)-based hybrid prediction technique to construct a sequence-dependent PPI prediction model. This model has two stages of operation: feature extraction and prediction. Along with sequence-based and Gene Ontology features, unique features were produced in the feature extraction stage utilizing the improved semantic similarity technique, which may deliver reliable findings. These collected characteristics were then sent to the prediction step, and hybrid neural networks, such as the Improved Recurrent Neural Network and Deep Belief Networks, were used to predict the PPI using modified score level fusion. These neural networks’ weight variables were adjusted utilizing a unique optimal methodology called Aquila Influenced Shark Smell (AISSO), and the outcomes showed that the developed model had attained an accuracy of around 88%, which is much better than the traditional methods; this model AISSO-based PPI prediction can provide precise and effective predictions.

Parametric optimization and comparative study of machine learning and deep learning algorithms for breast cancer diagnosis.

Breast Cancer is the leading form of cancer found in women and a major cause of increased mortality rates among them. However, manual diagnosis of the disease is time-consuming and often limited by the availability of screening systems. Thus, there is a pressing need for an automatic diagnosis system that can quickly detect cancer in its early stages. Data mining and machine learning techniques have emerged as valuable tools in developing such a system. In this study we investigated the performance of several machine learning models on the Wisconsin Breast Cancer (original) dataset with a particular emphasis on finding which models perform the best for breast cancer diagnosis. The study also explores the contrast between the proposed ANN methodology and conventional machine learning techniques. The comparison between the methods employed in the current study and those utilized in earlier research on the Wisconsin Breast Cancer dataset is also compared. The findings of this study are in line with those of previous studies which also highlighted the efficacy of SVM, Decision Tree, CART, ANN, and ELM ANN for breast cancer detection. Several classifiers achieved high accuracy, precision and F1 scores for benign and malignant tumours, respectively. It is also found that models with hyperparameter adjustment performed better than those without and boosting methods like as XGBoost, Adaboost, and Gradient Boost consistently performed well across benign and malignant tumours. The study emphasizes the significance of hyperparameter tuning and the efficacy of boosting algorithms in addressing the complexity and nonlinearity of data. Using the Wisconsin Breast Cancer (original) dataset, a detailed summary of the current status of research on breast cancer diagnosis is provided.

YOLOv8-Based Drone Detection: Performance Analysis and Optimization

The extensive utilization of drones has led to numerous scenarios that encompass both advantageous and perilous outcomes. By using deep learning techniques, this study aimed to reduce the dangerous effects of drone use through early detection of drones. The purpose of this study is the evaluation of deep learning approaches such as pre-trained YOLOv8 drone detection for security issues. This study focuses on the YOLOv8 model to achieve optimal performance in object detection tasks using a publicly available dataset collected by Mehdi Özel for a UAV competition that is sourced from GitHub. These images are labeled using Roboflow, and the model is trained on Google Colab. YOLOv8, known for its advanced architecture, was selected due to its suitability for real-time detection applications and its ability to process complex visual data. Hyperparameter tuning and data augmentation techniques were applied to maximize the performance of the model. Basic hyperparameters such as learning rate, batch size, and optimization settings were optimized through iterative experiments to provide the best performance. In addition to hyperparameter tuning, various data augmentation strategies were used to increase the robustness and generalization ability of the model. Techniques such as rotation, scaling, flipping, and color adjustments were applied to the dataset to simulate different conditions and variations. Among the augmentation techniques applied to the specific dataset in this study, rotation was found to deliver the highest performance. Blurring and cropping methods were observed to follow closely behind. The combination of optimized hyperparameters and strategic data augmentation allowed YOLOv8 to achieve high detection accuracy and reliable performance on the publicly available dataset. This method demonstrates the effectiveness of YOLOv8 in real-world scenarios, while also highlighting the importance of hyperparameter tuning and data augmentation in increasing model capabilities. To enhance model performance, dataset augmentation techniques including rotation and blurring are implemented. Following these steps, a significant precision value of 0.946, a notable recall value of 0.9605, and a considerable precision–recall curve value of 0.978 are achieved, surpassing many popular models such as Mask CNN, CNN, and YOLOv5.

Struggling Models: An Analysis of Logistic Regression and Random Forest in Predicting Repeat Buyers with Imbalanced Performance Metrics

Predicting repeat buyers is essential for businesses seeking to improve customer retention and maximize profitability. This study examines the effectiveness of logistic regression and random forest algorithms in forecasting repeat buyers, utilizing an e-commerce dataset from Kaggle. Despite the theoretical strengths of these models, our results indicate significant performance challenges. Both models were evaluated on key metrics: accuracy, precision, recall, F1 score, and ROC-AUC. The findings revealed that the models logistic regression and random forest performed poorly, with accuracy hovering around 50%, precision and recall demonstrating imbalanced performance, and ROC-AUC scores barely exceeding random guessing levels. Such metrics highlight the limited discriminative power of these models in identifying repeat buyers. The analysis suggests that issues such as data quality, feature relevance, and class imbalance contribute to these shortcomings. Specifically, the models struggled to effectively learn from the data, leading to suboptimal predictions. These results underscore the need for enhanced feature engineering, better handling of class imbalance, and possibly exploring more advanced algorithms. This study provides a critical assessment of the limitations inherent in using Logistic Regression and Random Forest for predicting repeat buyers, hence implements feature engineering, SMOTE and hyperparameter tuning using RandomSearchCV to get better result.

Advancing student outcome predictions through generative adversarial networks

Predicting student outcomes is essential in educational analytics for creating personalised learning experiences. The effectiveness of these predictive models relies on having access to sufficient and accurate data. However, privacy concerns and the lack of student consent often restrict data collection, limiting the applicability of predictive models. To tackle this obstacle, we employ Generative Adversarial Networks, a type of Generative AI, to generate tabular data replicating and enlarging the dimensions of two distinct publicly available student datasets. The ‘Math dataset’ has 395 observations and 33 features, whereas the ‘Exam dataset’ has 1000 observations and 8 features. Using advanced Python libraries, Conditional Tabular Generative Adversarial Networks and Copula Generative Adversarial Networks, our methodology consists of two phases. First, a mirroring approach where we produce synthetic data matching the volume of the real datasets, focusing on privacy and evaluating predictive accuracy. Second, augmenting the real datasets with newly created synthetic observations to fill gaps in datasets that lack student data. We validate the synthetic data before employing these approaches using Correlation Analysis, Density Analysis, Correlation Heatmaps, and Principal Component Analysis. We then compare the predictive accuracy of whether students will pass or fail their exams across original, synthetic, and augmented datasets. Employing Feedforward Neural Networks, Convolutional Neural Networks, and Gradient-boosted Neural Networks, and using Bayesian optimisation for hyperparameter tuning, this research methodically examines the impact of synthetic data on prediction accuracy. We implement and optimize these models using Python. Our mirroring approach aims to achieve accuracy rates that closely align with the original data. Meanwhile, our augmenting approach seeks to reach a slightly higher accuracy level than when solely learning from the original data. Our findings provide actionable insights into leveraging advanced Generative AI techniques to enhance educational outcomes and meet our objectives successfully.

Tuning parameters of deep neural network training algorithms pays off: a computational study

AbstractThe paper aims to investigate the impact of the optimization algorithms on the training of deep neural networks with an eye to the interaction between the optimizer and the generalization performance. In particular, we aim to analyze the behavior of state-of-the-art optimization algorithms in relationship to their hyperparameters setting to detect robustness with respect to the choice of a certain starting point in ending on different local solutions. We conduct extensive computational experiments using nine open-source optimization algorithms to train deep Convolutional Neural Network architectures on an image multi-class classification task. Precisely, we consider several architectures by changing the number of layers and neurons per layer, to evaluate the impact of different width and depth structures on the computational optimization performance. We show that the optimizers often return different local solutions and highlight the strong correlation between the quality of the solution found and the generalization capability of the trained network. We also discuss the role of hyperparameters tuning and show how a tuned hyperparameters setting can be re-used for the same task on different problems achieving better efficiency and generalization performance than a default setting.

A multi-layer nesting and integration approach for predicting groundwater levels in agriculturally intensive areas using data-driven models

Agricultural water demand, groundwater extraction, surface water delivery and climate exhibit complex nonlinear relationships with groundwater storage in agricultural regions. As alternatives to computationally intensive physical models, data-driven machine learning methods are frequently employed as surrogates to capture these complex relationships, owing to their high computational efficiency. Inevitably, reliance on a single machine learning model may lead to underestimation of prediction uncertainty and potentially result in reduced accuracy. This paper presents a multi-layer nesting and integration approach for predicting groundwater levels in agriculturally intensive areas using data-driven models. The key contributions of the research are threefold: 1) the development of a comprehensive input variable selection process considering the lag effects and driving mechanisms of groundwater level variations; 2) the implementation of an optimization-based hyperparameter tuning method to enhance the performance of individual machine learning models; and 3) the establishment of a multi-model integration framework based on a multi-layer nesting technique. This approach combines the outputs of multiple machine learning models to consolidate predictions and expand the hypothesis space. The effectiveness of the proposed framework is demonstrated through case studies in the Huaihe River Basin, a major agricultural region in China. The results show that the multi-layer nesting and integration approach outperforms the use of individual machine learning models, providing more accurate and reliable groundwater level predictions. This framework offers valuable insights for decision-makers and water resource managers, supporting sustainable groundwater management and addressing the challenges faced in agriculturally intensive areas.

Comparative Analysis of Hepatitis C virus Genotype 1a (Isolate 1) using Multiple Regression Algorithms and Fingerprinting Techniques

Approximately 70 million people worldwide have been infected with Hepatitis C virus (HCV), presenting a critical global health challenge. As a member of the Flaviviridae family, HCV can cause severe liver diseases such as cirrhosis, acute hepatitis, and chronic hepatitis. The Hepatitis C virus (HCV) genome encodes a single polyprotein consisting of 3010 amino acids, which when processed contains 10 polypeptides derived from cellular and viral proteases. These include structural proteins such as core protein, E1 and E2 envelope glycoproteins, and nonstructural proteins such as NS1, NS2, NS3, NS4A, NS4B, NS5A, and NS5B. Nonstructural proteins will be released by HCV NS2-3 and NS3-4A proteases, however, structural proteins will be released by host ER signaling peptidases. co-translationally and post-translationally form 10 individual structural proteins: 5'-C-E1-E2-p7-NS2-NS3-NS4A-NS4B-NS5A-NS5B-3'. Despite extensive research, there are significant gaps in predictive and analytical approaches to managing HCV, particularly in understanding the polyprotein structure and its implications for drug discovery. This study addresses these gaps by employing machine learning techniques to analyze HCV polyprotein using various fingerprinting methods and regression algorithms. The data was sourced from the ChEMBL database, and fingerprinting techniques such as PubChem, MACCS, and E-State were utilized. Regression algorithms, including Gradient Boosting Regression (GBR), Random Forest Regression (RFR), AdaBoost Regression (ABR), and Hist Gradient Boosting Regression (HSR), were applied. Model performance was evaluated using R² and Adjusted R² metrics, comparing default models with those enhanced by hyperparameter tuning. Feature importance analysis was conducted to identify key features influencing model performance, aiding in model simplification. The results show that although hyperparameter tuning does not significantly improve the predictive power of a model, it can provide an insight into model optimization. In particular, the default model showed higher R² and Adjusted R² values across different fingerprinting techniques compared to models with hyperparameterized features. Gradient Boosting Regression (GBR) and Random Forest Regression (RFR) consistently performed well, with GBR showing the highest R² values when using PubChem fingerprints. Although there was no significant improvement through hyperparameter tuning, this study was able to find out the features that strongly influenced the model performance by conducting a feature importance analysis. This analysis helped simplify the model and highlighted the potential of machine learning in improving the understanding of HCV polyprotein structure. This research identifies optimal regression models and fingerprinting techniques, providing a strong framework for future drug discovery efforts aimed at improving global health outcomes. The research also shows that it is important to date to advance drug discovery using machine learning.

ESCHR: a hyperparameter-randomized ensemble approach for robust clustering across diverse datasets

Clustering is widely used for single-cell analysis, but current methods are limited in accuracy, robustness, ease of use, and interpretability. To address these limitations, we developed an ensemble clustering method that outperforms other methods at hard clustering without the need for hyperparameter tuning. It also performs soft clustering to characterize continuum-like regions and quantify clustering uncertainty, demonstrated here by mapping the connectivity and intermediate transitions between MNIST handwritten digits and between hypothalamic tanycyte subpopulations. This hyperparameter-randomized ensemble approach improves the accuracy, robustness, ease of use, and interpretability of single-cell clustering, and may prove useful in other fields as well.

Evaluating the Effectiveness of Machine Learning-Based Intrusion Detection in Multi-Cloud Environments

In the dynamic landscape of cloud computing, multi-cloud environments have emerged as the prevalent ones due to offering a host of advantages including redundancy, flexibility, and increased security. These multi-cloud environments can, however, be vulnerable to stealthy cyber-attacks, thereby making it a challenging job to find efficient intrusion detection systems (IDS). The efficiency of the Machine Learning-based Intrusion Detection System in multi-cloud environments has been evaluated using a 1-D Convolutional Neural Network to classify network traffic based on the UNSW-NB15 dataset. The dataset included a wide variety of network features, and it was decided that it could be used directly after converting the categorical variables into their numerical forms and scaling them with MinMaxScaler. This was used for training and testing the 1D-CNN model, which had attained an accuracy of 84.02% during training and 82.79% on validation. Further, the performance of the model is elicited with metrics that include precision at 80.91%, recall at 82.79%, and an F1-score at 81.28%, showing its capability in identifying network intrusions effectively. The small difference between the accuracy of training and validation shows that overfitting was minimal; thus, the model will generalize well when applied to unseen data. This research underlines the possibility of using deep learning techniques, and particularly CNNs, for intrusion detection in complex multi-cloud infrastructures. Probably future work would be optimizing the model through hyperparameter tuning, using additional sources of data, and even exploring much more sophisticated architectures that will further improve the accuracy of intrusion detection.

In Search of Lost Online Test-Time Adaptation: A Survey

AbstractThis article presents a comprehensive survey of online test-time adaptation (OTTA), focusing on effectively adapting machine learning models to distributionally different target data upon batch arrival. Despite the recent proliferation of OTTA methods, conclusions from previous studies are inconsistent due to ambiguous settings, outdated backbones, and inconsistent hyperparameter tuning, which obscure core challenges and hinder reproducibility. To enhance clarity and enable rigorous comparison, we classify OTTA techniques into three primary categories and benchmark them using a modern backbone, the Vision Transformer. Our benchmarks cover conventional corrupted datasets such as CIFAR-10/100-C and ImageNet-C, as well as real-world shifts represented by CIFAR-10.1, OfficeHome, and CIFAR-10-Warehouse. The CIFAR-10-Warehouse dataset includes a variety of variations from different search engines and synthesized data generated through diffusion models. To measure efficiency in online scenarios, we introduce novel evaluation metrics, including GFLOPs, wall clock time, and GPU memory usage, providing a clearer picture of the trade-offs between adaptation accuracy and computational overhead. Our findings diverge from existing literature, revealing that (1) transformers demonstrate heightened resilience to diverse domain shifts, (2) the efficacy of many OTTA methods relies on large batch sizes, and (3) stability in optimization and resistance to perturbations are crucial during adaptation, particularly when the batch size is 1. Based on these insights, we highlight promising directions for future research. Our benchmarking toolkit and source code are available at https://github.com/Jo-wang/OTTA_ViT_survey.

Nonlinear Ensemble Deep Learning Model for Energy Consumption Prediction with Bayesian Optimization

Accurate prediction of electric energy consumption is crucial for efficient load dispatching, energy utilization, and grid operation. Traditional statistical and classical machine learning methods struggle with the nonlinear nature of energy consumption data, often leading to higher prediction errors. Additionally, deep learning models using a single approach face challenges such as convergence to local minima and poor generalization. This paper proposes a nonlinear ensemble deep learning model for residential energy consumption prediction, incorporating Bayesian optimization for hyperparameter tuning. The model combines Long Short-Term Memory (LSTM), Bidirectional LSTM (BiLSTM), and 1D Convolutional Neural Networks (1D-CNN), leveraging their powerful nonlinear feature learning capabilities. A k-means clustering approach is used to preprocess and reduce variability in the data, enhancing the ensemble model's performance. The ensemble model was tested on real energy consumption data from two districts in Addis Ababa, showing significant improvements in prediction accuracy with lower MAE, RMSE, and MAPE values compared to single models and un-clustered data. The integration of clustering and Bayesian optimization further enhanced model generalizability and minimized overfitting, demonstrating the effectiveness of a nonlinear approach in capturing complex energy consumption patterns.