Machine Learning Optimization Research Articles

Reducing the Power Consumption of Edge Devices Supporting Ambient Intelligence Applications

Having as a main objective the exploration of power efficiency of microcontrollers running machine learning models, this manuscript contrasts the performance of two types of state-of-the-art microcontrollers, namely ESP32 with an LX6 core and ESP32-S3 with an LX7 core, focusing on the impact of process acceleration technologies like cache memory and vectoring. The research employs experimental methods, where identical machine learning models are run on both microcontrollers under varying conditions, with particular attention to cache optimization and vector instruction utilization. Results indicate a notable difference in power efficiency between the two microcontrollers, directly linked to their respective process acceleration capabilities. The study concludes that while both microcontrollers show efficacy in running machine learning models, ESP32-S3 with an LX7 core demonstrates superior power efficiency, attributable to its advanced vector instruction set and optimized cache memory usage. These findings provide valuable insights for the design of power-efficient embedded systems supporting machine learning for a variety of applications, including IoT and wearable devices, ambient intelligence, and edge computing and pave the way for future research in optimizing machine learning models for low-power, embedded environments.

Application of machine learning and statistical approaches for optimization of heavy metals (Cd2+, Pb2+, Cu2+, and Zn2+) adsorption onto carbonized char prepared from PET plastic bottle waste

Abstract This study focuses on the probable use of carbonized char prepared from PET plastic bottles for heavy metals (HMs) adsorption (Cd2+, Pb2+, Cu2+, and Zn2+). The prepared adsorbent is characterized by field-emission scanning electron microscopy (FE-SEM), energy dispersive X-ray spectroscopy (EDX), and Fourier-transform infrared spectroscopy (FTIR). Batch adsorption experiments were conducted with the influencing of different operational conditions: contact time (1–180 min), adsorbate concentration (25–300 mg/L), adsorbent dose (0.5–6 g/L), pH (3–7), and temperature (25–60 ºC). High coefficient value [Cd2+ (R2 = 0.99), Pb2+ (R2 = 0.97), Cu2+ (R2 = 0.94), and Zn2+ (R2 = 0.98)] of process optimization model suggest that this model was significant, where pH and adsorbent dose expressively stimulus removal efficiency including 86.68, 73.66, 67.10, and 57.04% for Cd2+, Pb2+, Cu2+, and Zn2+ at pH (7), respectively. Furthermore, ANN and BB-RSM revealed a good association between the tested and projected values. The maximum monolayer adsorption capacity of Cd2+, Pb2+, Cu2+, and Zn2+ was 263.157, 78.740, 196.078, and 84.745 mg/g, respectively. Pseudo-second-order was the well-suited kinetics, where Langmuir and Freundlich isotherm could explain better for equilibrium adsorption data. Thermodynamic study shows HMs adsorption is favorable, exothermic, and spontaneous.

Predictive models for flexible pavement fatigue cracking based on machine learning

Pavement performance prediction is crucial for ensuring the longevity and safety of road networks. In our extensive study, we employ a diverse array of techniques to enhance fatigue performance models in flexible pavements. The methodology begins with Random Forest feature selection, identifying the top 15 critical variables that significantly impact pavement performance. These variables form the basis for subsequent model development. Our investigation into model performance indicates the superiority of advanced machine learning methods such as Regression Trees (RT), Gaussian Process Regression (GPR), Support Vector Machines (SVM), Ensemble Trees (ET), and Artificial Neural Networks (ANN) over traditional linear regression methods. This consistent outperformance underscores their potential to reshape forecasting accuracy. Through extensive model optimization, we reveal robust performance across both complete and selected feature sets, emphasizing the importance of meticulous feature selection in enhancing forecast accuracy. The accuracy of our best optimized machine learning model is highlighted by its Performance Measurement metrics: RMSE of 22.416, MSE of 502.46, R-squared of 0.80848, and MAE of 8.9958. Additionally, comparative analysis with previous empirical models demonstrates that our best optimized machine learning model outperforms existing empirical models. This work underscores the significance of feature curation in pavement performance prediction, highlighting the potential of sophisticated modeling methodologies. Embracing cutting-edge technologies facilitates data-driven decisions, ultimately contributing to the development of more robust road networks, enhancing safety, and prolonging lifespan.

Multi-criteria structural optimization of a hood inner structure using machine learning technique

This article presents an efficient multi-criteria structural optimization process based on machine learning for shape and size optimization of the hood inner. For this purpose, a low dimensional parameterization based on two-dimensional mapping and dimensionality reduction is used to decrease the number of design variables. There are many design parameters to be considered for designing this structure. In this study, the effect of three important criteria consisting of head injury, eigenfrequency, and static load were analyzed. A full vehicle finite element model is prepared for a newly developed product because head injury analysis requires a full vehicle model due to contact with the inner components of the engine compartment. Then, an experimental study is performed based on the real test scenario for model verification. A new rating method for analyzing the head injury criteria evaluated in different points is proposed and different machine learning techniques are compared in the approximation of the objective function. Finally, the problem is solved as both single and multi-objective optimization problems. Results present considerable improvement in accuracy and efficiency in the identified structure while the method has a low computational cost and the implementation is not difficult.

ML-Based Detection of DDoS Attacks Using Evolutionary Algorithms Optimization.

The escalating reliance of modern society on information and communication technology has rendered it vulnerable to an array of cyber-attacks, with distributed denial-of-service (DDoS) attacks emerging as one of the most prevalent threats. This paper delves into the intricacies of DDoS attacks, which exploit compromised machines numbering in the thousands to disrupt data services and online commercial platforms, resulting in significant downtime and financial losses. Recognizing the gravity of this issue, various detection techniques have been explored, yet the quantity and prior detection of DDoS attacks has seen a decline in recent methods. This research introduces an innovative approach by integrating evolutionary optimization algorithms and machine learning techniques. Specifically, the study proposes XGB-GA Optimization, RF-GA Optimization, and SVM-GA Optimization methods, employing Evolutionary Algorithms (EAs) Optimization with Tree-based Pipelines Optimization Tool (TPOT)-Genetic Programming. Datasets pertaining to DDoS attacks were utilized to train machine learning models based on XGB, RF, and SVM algorithms, and 10-fold cross-validation was employed. The models were further optimized using EAs, achieving remarkable accuracy scores: 99.99% with the XGB-GA method, 99.50% with RF-GA, and 99.99% with SVM-GA. Furthermore, the study employed TPOT to identify the optimal algorithm for constructing a machine learning model, with the genetic algorithm pinpointing XGB-GA as the most effective choice. This research significantly advances the field of DDoS attack detection by presenting a robust and accurate methodology, thereby enhancing the cybersecurity landscape and fortifying digital infrastructures against these pervasive threats.

Review on structural optimization techniques for additively manufactured implantable medical devices

Recent developments in additive manufacturing (AM) have led to significant opportunities in the design and fabrication of implantable medical devices due to the advantages that AM offers compared to conventional manufacturing, such as high customizability, the ability to fabricate highly complex shapes, good dimensional accuracy, a clean build environment, and reduced material usage. The study of structural design optimization (SDO) involves techniques such as Topology Optimization (TO), Shape Optimization (SHO), and Size Optimization (SO) that determine specific parameters to achieve the best measurable performance in a defined design space under a given set of loads and constraints. Integration of SDO techniques with AM leads to utmost benefits in designing and fabricating optimized implantable medical devices with enhanced functional performance. Research and development of various lattice structures represents a powerful method for unleashing the full potential of additive manufacturing (AM) technologies in creating medical implants with improved surface roughness, biocompatibility, and mechanical properties. Furthermore, the integration of artificial intelligence (AI) and machine learning (ML) in structural optimization has expanded opportunities to improve device performance, adaptability, and durability. The review is meticulously divided into two main sections, reflecting the predictability of the implant’s internal structure: (a) unpredictable interior topology, which explores topology-based optimization techniques, and (b) predictable inner topology, concentrating on lattice structures. The analysis of the reviewed literature highlights a common focus on addressing issues such as stress shielding, osseointegration enhancement, customization to individual needs, programmable functionalities, and weight reduction in implant designs. It emphasizes significant advances in reducing stress shielding effects, promoting osseointegration, and facilitating personalized implant creation. The review provides a detailed classification of optimization methods, with each approach scrutinized for its unique contribution to overcoming specific challenges in medical implant design, thus leading to more advanced, effective, and patient-oriented implantable devices.

High-density forest AGB estimation in tropical forest integrated with PolInSAR multidimensional features and optimized machine learning algorithms

Accurately estimating the above-ground biomass (AGB) of high-density tropical rainforests is a challenging issue. In this study, airborne multi-baseline PolInSAR data were used to estimate the tropical rainforest AGB in Gabon, Africa. The most suitable baseline combination of the PolInSAR data was initially determined through baseline selection, and the PolInSAR parameters related to forest height were obtained based on the forest canopy height estimation theory and microwave penetration depth theory. The height parameter, baseline parameter, and observation geometry parameter were then used as independent variables to construct the AGB regression model. Support vector regression (SVR) was chosen as the AGB estimation model, and the global best particle swarm algorithm (GLB-PSO) was used to optimize the SVR model’s parameters. The results show that the RFECV variable selection method is superior to the Pearson method. The GLB-PSO algorithm can also further improve the saturation point of the SVR model—the estimation results show that the saturation point of AGB estimation of PolInSAR multidimensional features combined with the SVR machine learning algorithm is up to 500 Mg/ha, while this saturation point can be increased to 650 Mg/ha when using the GLB-PSO-SVR algorithm.

A Novel Framework for Forest Above-Ground Biomass Inversion Using Multi-Source Remote Sensing and Deep Learning

The estimation of forest above-ground biomass (AGB) can be significantly improved by leveraging remote sensing (RS) and deep learning (DL) techniques. In this process, it is crucial to obtain appropriate RS features and develop a suitable model. However, traditional methods such as random forest (RF) feature selection often fail to adequately consider the complex relationships within high-dimensional RS feature spaces. Moreover, challenges related to parameter selection and overfitting inherent in DL models may compromise the accuracy of AGB estimation. Therefore, this study proposes a novel framework based on freely available Sentinel-1 synthetic aperture radar (SAR) and Sentinel-2 optical data. Firstly, we designed new indices through the formula analogous with vegetation index calculation to integrate multidimensional spectral and structural information. Then, leveraging the simplicity of computational principles, a pigeon-inspired optimization algorithm (PIO) was introduced into a bi-directional long short-term memory neural network (PIO-BiLSTM), which achieved the set objective function through repeated iteration and validation to obtain the optimal model parameters. Finally, to verify the framework’s effect, we conducted experiments in two different tree species and compared another seven classical optimization algorithms and machine learning models. The results indicated that the new indices significantly improved the inversion accuracy of all models in both categories, and the PIO-BiLSTM model achieved the highest accuracy (Category-1: R2 = 0.8055, MAE = 8.8475 Mg·ha−1, RMSE = 12.2876 Mg·ha−1, relative RMSE = 18.1715%; Category-2: R2 = 0.7956, MAE = 1.7103 Mg·ha−1, RMSE = 2.2887 Mg·ha−1, relative RMSE = 9.3000%). Compared with existing methods, the proposed framework greatly reduced the labor costs in parameter selection, and its potential uncertainty also decreased by up to 9.0%. Furthermore, the proposed method has a strong generalization ability and is independent of tree species, indicating its great potential for future forest AGB inversion in wider regions with diverse forest types.

Compact Data Learning for Machine Learning Classifications

This paper targets the area of optimizing machine learning (ML) training data by constructing compact data. The methods of optimizing ML training have improved and become a part of artificial intelligence (AI) system development. Compact data learning (CDL) is an alternative practical framework to optimize a classification system by reducing the size of the training dataset. CDL originated from compact data design, which provides the best assets without handling complex big data. CDL is a dedicated framework for improving the speed of the machine learning training phase without affecting the accuracy of the system. The performance of an ML-based arrhythmia detection system and its variants with CDL maintained the same statistical accuracy. ML training with CDL could be maximized by applying an 85% reduced input dataset, which indicated that a trained ML system could have the same statistical accuracy by only using 15% of the original training dataset.

Reinvent 4: Modern AI-driven generative molecule design.

REINVENT 4 is a modern open-source generative AI framework for the design of small molecules. The software utilizes recurrent neural networks and transformer architectures to drive molecule generation. These generators are seamlessly embedded within the general machine learning optimization algorithms, transfer learning, reinforcement learning and curriculum learning. REINVENT 4 enables and facilitates de novo design, R-group replacement, library design, linker design, scaffold hopping and molecule optimization. This contribution gives an overview of the software and describes its design. Algorithms and their applications are discussed in detail. REINVENT 4 is a command line tool which reads a user configuration in either TOML or JSON format. The aim of this release is to provide reference implementations for some of the most common algorithms in AI based molecule generation. An additional goal with the release is to create a framework for education and future innovation in AI based molecular design. The software is available from https://github.com/MolecularAI/REINVENT4 and released under the permissive Apache 2.0 license. Scientific contribution. The software provides an open-source reference implementation for generative molecular design where the software is also being used in production to support in-house drug discovery projects. The publication of the most common machine learning algorithms in one code and full documentation thereof will increase transparency of AI and foster innovation, collaboration and education.

Enhancing Predictions of Acetate and Ethanol Production from Microbial Electrosynthesis Using Optimized Machine Learning Models

Enhancing Predictions of Acetate and Ethanol Production from Microbial Electrosynthesis Using Optimized Machine Learning Models

Optimizing machine learning yield functions using query-by-committee for support vector classification with a dynamic stopping criterion

AbstractIn the field of materials engineering, the accurate prediction of material behavior under various loading conditions is crucial. Machine Learning (ML) methods have emerged as promising tools for generating constitutive models straight from data, capable of describing complex material behavior in a more flexible way than classical constitutive models. Yield functions, which serve as foundation of constitutive models for plasticity, can be properly described in a data-oriented manner using ML methods. However, the quality of these descriptions heavily relies on the availability of sufficient high-quality and representative training data that needs to be generated by fundamental numerical simulations, experiments, or a combination of both. The present paper addresses the issue of data selection, by introducing an active learning approach for Support Vector Classification (SVC) and its application in training an ML yield function with suitable data. In this regard, the Query-By-Committee (QBC) algorithm was employed, guiding the selection of new training data points in regions of the feature space where a committee of models shows significant disagreement. This approach resulted in a marked reduction in the variance of model predictions throughout the active learning process. It was also shown that the rate of decrease in the variance went along with an increase in the quality of the trained model, quantified by the Matthews Correlation Coefficient (MCC). This demonstrated the effectiveness of the approach and offered us the possibility to define a dynamic stopping criterion based on the variance in the committee results.

Machine learning optimization of bio-sandcrete brick modelling using response surface methodology

In this study, raw grinded groundnut shell (RGGNS) was used as a fine aggregate in the brick industry to reuse agricultural waste in building materials. In this study, an experimental approach was used to examine a new cement brick with raw groundnut shells integrated with compressive strength, water absorption and dry density optimization utilizing response surface methodology (RSM). The raw ground-nut shell content improved the fine aggregate performance of the 40%, 50%, and 60% samples. The 28-day high compressive strength with the raw ground-nut shell was 6.1 N/mm2 maximum, as needed by the technical standard. Samples made from 40%, 50%, and 60% raw groundnut shells yielded densities of 1.7, 2.2, and 1.9 kg/cm3 for groundnut shell (GNS) brick, respectively. A product's mechanical properties meet the IS code standard’s minimum requirements. RSM was then utilized to develop a model for the addition of raw groundnut shell to concrete. R-square and Adeq precision values indicated that the results are highly significant, and equations for predicting compressive strength, water absorption, and dry density have been developed. In addition, optimization was performed on the RSM findings to determine the efficiency optimization of the model. Following the optimization results, experiments were conducted to determine the applicability of the optimized model.

Applying Optimized Machine Learning Models for Predicting Unconfined Compressive Strength in Fine-Grained Soil

Applying Optimized Machine Learning Models for Predicting Unconfined Compressive Strength in Fine-Grained Soil

Optimized machine learning model discourse analysis

Optimized machine learning model discourse analysis

Optimized machine learning model for Alzheimer and epilepsy detection from EEG signals

ABSTRACT One of the common nervous system diseases in older adults is Alzheimer's and epilepsy, and the possibility of occurrence increases with age. The chances of seizure are high for patients with mild cognitive impairment and Alzheimer's disease. So, there is a bidirectional association between Alzheimer's and epilepsy, as both affect the neurodegenerative processes. Electroencephalogram (EEG) is a possible non-invasive measurement technique widely used to measure the variations in brain signals. EEG signal is analyzed to discriminate the Alzheimer and epilepsy. Numerous research works evaluated the clinical relevance of Alzheimer's and epilepsy. Specifically, machine learning-based evaluation models developed recently bring the facts by extracting features from the EEG signals. However, machine learning-based models lag in performance due to high dimensional EEG features. For initial feature selection particle swarm optimization is included in the proposed model and to reduce the computation complexity of the classifier, kernel PCA is incorporated for dimensionality reduction. Experimentations using benchmark Bon and Dementia datasets confirms the proposed model better performances in terms of precision, recall, f1-score and accuracy. The attained accuracy of 94% is much better than existing Gaussian Mixture Model (GMM), Relevance Vector Machine (RVM), Support Vector Machine (SVM), and Artificial Neural Network (ANN) methods.

Optimizing ensembles machine learning, genetic algorithms, and multivariate modeling for enhanced prediction of maize yield and stress tolerance index

The frequent occurrence of drought, halting from unpredictable climate-induced weather patterns, presents significant challenges in breeding drought-tolerant maize to identify adaptable genotypes. The study explores the optimization of machine learning (ML) to predict both the grain yield and stress tolerance index (STI) of maize under normal and drought-induced stress. In total, 35 genotypes, comprising 31 hybrid candidates and four commercial varieties, were meticulously evaluated across three normal and drought-treated sites. Three popular ML were optimized using a genetic algorithm (GA) and ensemble ML to enhance data capture. Additionally, a Multi-trait Genotype-Ideotype Distance (MGIDI) was also involved to identify superior maize hybrids well-suited for drought conditions. The results highlight that the ensemble meta-models optimized by grid search exhibit robust performance with high accuracy across the testing datasets (R2 = 0.92 for grain yield and 0.82 for STI). The RF optimized by GA algorithm demonstrates slightly lower performance (R2 = 0.91 for grain yield and 0.79 for STI), surpassing the predictive performance of individual SVM-GA and KNN-GA models. Selection of the best-performing hybrids indicated that out of the six hybrids with the highest STI values, both the ensemble and MGIDI can accurately predict four hybrids, namely H06, H10, H13, and H35. Thus, combining ML with MGIDI enables researchers to discern traits for each genotype and holds promise for advancing the field of drought-tolerant maize breeding and expediting the development of resilient varieties.

Advances in machine learning optimization for classical and quantum photonics

The development and optimization of photonic devices and various other nanostructure electromagnetic devices present a computationally intensive task. Much optimization relies on finite-difference time-domain or finite element analysis simulations, which can become very computationally demanding for finely detailed structures and dramatically reduce the available optimization space. In recent years, various inverse design machine learning (ML) techniques have been successfully applied to realize previously unexplored optimization spaces for photonic and quantum photonic devices. In this review, recent results using conventional optimization methods, such as the adjoint method and particle swarm, are examined along with ML optimization using convolutional neural networks, Bayesian optimizations with deep learning, and reinforcement learning in the context of new applications to photonics and quantum photonics.

Design Optimisation of Metastructure Configuration for Lithium-Ion Battery Protection Using Machine Learning Methodology

The market for electric vehicles (EVs) has been growing in popularity, and by 2027, it is predicted that the market valuation will reach $869 billion. To support the growth of EVs in public road safety, advances in battery safety research for EV application should achieve low-cost, lightweight, and high safety protection. In this research, the development of a lightweight, crashworthy battery protection system using an excellent energy absorption capability is carried out. The lightweight structure was developed by using metastructure constructions with an arrangement of repeated lattice cellular structures. Three metastructure configurations (bi-stable, star-shaped, double-U) with their geometrical variables (thickness, inner spacing, cell stack) and material types (stainless steel, aluminium, and carbon steel) were evaluated until the maximum Specific Energy Absorptions (SEA) value was attained. The Finite Element Method (FEM) is utilised to simulate the mechanics of impact and calculate the optimum SEA of the various designs using machine learning methodology. Latin Hypercube Sampling (LHS) was used to derive the design variation by dividing the variables into 100 samples. The machine learning optimisation method utilises the Artificial Neural Networks (ANN) and Non-dominated Sorting Genetic Algorithm-II (NSGA-II) to forecast the design that produces maximum SEA. The optimum control variables are star-shaped cells consisting of one vertical unit cell using aluminium material with a cross-section thickness of 2.9 mm. The optimum design increased the SEA by 5577% compared to the baseline design. The accuracy of the machine learning prediction is also verified using numerical simulation with a 2.83% error. Four different sandwich structure configurations are then constructed using the optimal geometry for prismatic battery protection subjected to ground impact loading conditions. An optimum configuration of 6×4×1 core cells arrangement results in a maximum displacement of 7.33 mm for the prismatic battery in the ground impact simulation, which is still less than the deformation threshold for prismatic battery safety of 10.423 mm. It is shown that the lightweight metastructure is very efficient for prismatic battery protection subjected to ground impact loading conditions.

Catalyzing net-zero carbon strategies: Enhancing CO2 flux Prediction from underground coal fires using optimized machine learning models

Underground coal fires release substantial carbon dioxide (CO2), posing significant environmental and health threats. Accurate prediction of surface CO2 emissions in these areas is crucial for understanding combustion zones and contributes to the global net zero carbon strategy. Traditional data analysis methods have been inadequate for CO2 flux prediction, highlighting the necessity for advanced machine learning (ML) techniques. This study introduces four optimized ML models—General Regression Neural Networks (GRNNs) and Radial Basis Function Neural Networks (RBFNNs) coupled with Grey Wolf Optimizer (GWO), Particle Swarm Optimization (PSO), and Whale Optimization Algorithm (WOA)—to rapidly predict CO2 flux in areas affected by underground coal fires. Utilizing 223 field test samples, these models consider six key variables: soil temperature at 30 cm depth (ST-30), ambient pressure/temperature/humidity (AP/AT/AH), soil water content (SWC), and wind speed (A-WS). The results underscore the superior predictive accuracy of the GRNN model, with an RMSE of 0.074 and an R2 of 0.9995. Sensitivity analysis reveals A-WS and ST-30 as the most influential factors. Compared to traditional methods, these ML models demonstrate enhanced accuracy and efficiency, marking a significant advancement in the field. The study's findings have broader applications beyond underground coal fires, suggesting potential for these ML models in other environmental monitoring contexts, such as emissions tracking in urban areas or integration with satellite data for global environmental assessment. This methodology represents a pivotal step in environmental management and monitoring, offering scalable and adaptable solutions for various ecological challenges. By rapidly and accurately estimating CO2 flux from underground coal fires, this study contributes significantly to achieving the global net zero carbon target and sets a new benchmark in environmental ML applications.