Optimal Learning Algorithm Research Articles

Capturing woody aboveground biomass historical change and potential under climate change using Landsat time-series for afforestation in dryland of China

Capturing long-term dynamics and the potential under climate change of woody aboveground biomass (AGB) is imperative for calculating and raising carbon sequestration of afforestation in dryland. It is always been a great challenge to accurately capture AGB dynamics of sparse woody vegetation mixed with grassland using only Landsat time-series, resulting in changing trajectory of woody AGB estimates cannot accurately reflect woody vegetation growth regularity in dryland. In this study, surface reflectance (SR) sensitive to woody AGB was firstly selected and interannual time-series of composited SR was smoothed using S-G filter for each pixel, and then optimal machine learning algorithm was selected to estimate woody AGB time-series. Pixels that have reached AGB potential were detected based on the AGB changing trajectory, and the potential was spatial-temporal extended using random forest model combining environmental variables under current climate condition and CMIP6 climate models. Results show that: 1) minimum value composite based on NIRv during Jul.-Sep. is more capable of explaining woody AGB variation in dryland (R = 0.87, p < 0.01), and Random Forest (RF) model has the best performance in estimating woody AGB (R2 = 0.75, RMSE = 4.74 t·ha−1) among sis commonly used machine learning models. 2) Annual woody AGB estimates can be perfectly fitted with a logistic growth curve (R2 = 0.97, p < 0.001) indicating explicit growth regularity of woody vegetation, which provides physiological foundation for determining woody AGB potential. 3) Woody AGB potential can be accurately simulated by RF combining environmental variables (R2 = 0.95, RMSE = 2.89 t·ha−1), and current woody AGB still has a potential of small increase, whereas the overall losses of woody AGB potential were observed in 2030, 2040 and 2050 under CMIP6 SSP-RCP scenarios.

From Time-Series to Hybrid Models: Advancements in Short-Term Load Forecasting Embracing Smart Grid Paradigm

This review paper is a foundational resource for power distribution and management decisions, thoroughly examining short-term load forecasting (STLF) models within power systems. The study categorizes these models into three groups: statistical approaches, intelligent-computing-based methods, and hybrid models. Performance indicators are compared, revealing the superiority of heuristic search and population-based optimization learning algorithms integrated with artificial neural networks (ANNs) for STLF. However, challenges persist in ANN models, particularly in weight initialization and susceptibility to local minima. The investigation underscores the necessity for sophisticated predictive models to enhance forecasting accuracy, advocating for the efficacy of hybrid models incorporating multiple predictive approaches. Acknowledging the changing landscape, the focus shifts to STLF in smart grids, exploring the transformative potential of advanced power networks. Smart measurement devices and storage systems are pivotal in boosting STLF accuracy, enabling more efficient energy management and resource allocation in evolving smart grid technologies. In summary, this review provides a comprehensive analysis of contemporary predictive models and suggests that ANNs and hybrid models could be the most suitable methods to attain reliable and accurate STLF. However, further research is required, including considerations of network complexity, improved training techniques, convergence rates, and highly correlated inputs to enhance STLF model performance in modern power systems.

Integrating Multiscale Simulation with Machine Learning to Screen and Design FIL@COFs for Ethane-Selective Separation.

Efficient and economical separation of C2H6/C2H4 is an imperative and extremely challenging process in the petrochemical industry. The C2H6-selective adsorbents with high working capacity and high selectivity are highly desirable from a practical application standpoint. In this study, we constructed a database of fluorinated ionic liquid@covalent organic frameworks (FIL@COFs) and screened out the high-performing FIL@COFs for C2H6-selective separation. Utilizing the optimal machine learning (ML) algorithm (XGBoost) and hyperparameters, we further revealed the key factors influencing the separation performance. The multiscale simulation not only validated the prediction accuracy of ML but also demonstrated that adjusting the largest cavity diameter of COFs with FILs could yield FIL@COFs with high performance for C2H6-selective separation. Our work provides essential guidance for designing new FIL@COF adsorbents for value-added gas purification.

An Improved BGE-Adam Optimization Algorithm Based on Entropy Weighting and Adaptive Gradient Strategy

This paper introduces an enhanced variant of the Adam optimizer—the BGE-Adam optimization algorithm—that integrates three innovative technologies to augment the adaptability, convergence, and robustness of the original algorithm under various training conditions. Firstly, the BGE-Adam algorithm incorporates a dynamic β parameter adjustment mechanism that utilizes the rate of gradient variations to dynamically adjust the exponential decay rates of the first and second moment estimates (β1 and β2), the adjustment of β1 and β2 is symmetrical, which means that the rules that the algorithm considers when adjusting β1 and β2 are the same. This design helps to maintain the consistency and balance of the algorithm, allowing the optimization algorithm to adaptively capture the trending movements of gradients. Secondly, it estimates the direction of future gradients by a simple gradient prediction model, combining historic gradient information with the current gradient. Lastly, entropy weighting is integrated into the gradient update step. This strategy enhances the model’s exploratory nature by introducing a certain amount of noise, thereby improving its adaptability to complex loss surfaces. Experimental results on classical datasets, MNIST and CIFAR10, and gastrointestinal disease medical datasets demonstrate that the BGE-Adam algorithm has improved convergence and generalization capabilities. In particular, on the specific medical image gastrointestinal disease test dataset, the BGE-Adam optimization algorithm achieved an accuracy of 69.36%, a significant improvement over the 67.66% accuracy attained using the standard Adam algorithm; on the CIFAR10 test dataset, the accuracy of the BGE-Adam algorithm reached 71.4%, which is higher than the 70.65% accuracy of the Adam optimization algorithm; and on the MNIST dataset, the BGE-Adam algorithm’s accuracy was 99.34%, surpassing the Adam optimization algorithm’s accuracy of 99.23%. The BGE-Adam optimization algorithm exhibits better convergence and robustness. This research not only demonstrates the effectiveness of the combination of these three technologies but also provides new perspectives for the future development of deep learning optimization algorithms.

Variable Neighborhood Search Based Human Learning Optimization Algorithm for Secure Data Analysis and Computing

Variable Neighborhood Search Based Human Learning Optimization Algorithm for Secure Data Analysis and Computing

Predicting scale deposition in oil reservoirs using machine learning optimization algorithms

Scale deposition, a form of formation damage, not only affects the reservoir but also damages the well and equipment. This phenomenon occurs due to changes in temperature, pressure, and the injection of incompatible salt water, leading to ionic reactions. This study investigated permeability reduction due to scale deposition and examined how parameters such as temperature, pressure drop, and ion concentration affect the prediction accuracy. The scale deposits investigated in this study include CaSO4, BaSO4, and SrSO4. This paper uses Python to employ different machine-learning algorithms to predict the results. Each machine learning model has certain hyper-parameters that need adjustment. Failure to do so will result in reduced accuracy and incomplete interpretation of input data. The accuracy of the support vector regression (SVR) algorithm was significantly affected by the variation of the epsilon parameter in the dataset used. Therefore, before hyperparameter optimization, SVR had the lowest accuracy at 0.575. After adjusting the hyper-parameters, our findings show that SVR had the highest increase in R-squared value, which was 0.900, and the most minor growth in KNN, which went from 0.995 to 0.996. Additionally, the highest accuracy value for K-Nearest Neighbor is 0.996. Furthermore, most errors were related to SVR and XGBoost algorithms, while the most negligible errors were for the Decision Tree and KNN algorithms.

Prediction of Glass Chemical Composition and Type Identification Based on Machine Learning Algorithms

Ancient glass artifacts were susceptible to weathering from the environment, causing changes in their chemical composition, which pose significant obstacles to the identification of glass products. Analyzing the chemical composition of ancient glass has been beneficial for evaluating their weathering status and proposing measures to reduce glass weathering. The objective of this study was to explore the optimal machine learning algorithm for glass type classification based on chemical composition. A set of glass artifact data including color, emblazonry, weathering, and chemical composition was employed and various methods including logistic regression and machine learning techniques were used. The results indicated that a significant correlation (p &lt; 0.05) could only observed between surface weathering and the glass types (high-potassium and lead–barium). Based on the random forest and logistic regression models, the primary chemical components that signify glass types and weathering status were determined using PbO, K2O, BaO, SiO2, Al2O3, and P2O5. The random forest model presented a superior ability to identify glass types and weathering status, with a global accuracy of 96.3%. This study demonstrates the great potential of machine learning for glass chemical component estimation and glass type and weathering status identification, providing technical guidance for the appraisal of ancient glass artifacts.

What if the Covid-19 Pandemic Never Happened Estimation of the Tourist Arrivals for 2020 Via Levenberg-Marquardt Optimization and K-Star (K ) Machine Learning Algorithms

Based on the probability theory, which is used to model uncertainty and randomness in real-world situations, the study aims to understand the impact of uncertain conditions, such as the Covid-19 pandemic, on the accuracy of the algorithms and the resulting losses to a country's tourism industry. The contribution of this paper to the international body of knowledge is twofold: firstly, it advances theoretical understanding of the use of probability theory in modelling real-world problems; and secondly, it offers a methodological approach for estimating tourist arrivals that accounts for the impact of extreme events. To achieve these aims, the Levenberg-Marquardt optimization was first applied to determine the optimal coefficients of the exponential function for estimating tourist arrivals from 1950 to 2020. Next, the K-Star machine learning algorithm was applied to the dataset with and without Covid-19 cases to estimate tourist arrivals.

Learning rate burst for superior SGDM and AdamW integration

Current mainstream deep learning optimization algorithms can be classified into two categories: non-adaptive optimization algorithms, such as Stochastic Gradient Descent with Momentum (SGDM), and adaptive optimization algorithms, like Adaptive Moment Estimation with Weight Decay (AdamW). Adaptive optimization algorithms for many deep neural network models typically enable faster initial training, whereas non-adaptive optimization algorithms often yield better final convergence. Our proposed Adaptive Learning Rate Burst (Adaburst) algorithm seeks to combine the strengths of both categories. The update mechanism of Adaburst incorporates elements from AdamW and SGDM, ensuring a seamless transition between the two. Adaburst modifies the learning rate of the SGDM algorithm based on a cosine learning rate schedule, particularly when the algorithm encounters an update bottleneck, which is called learning rate burst. This approach helps the model to escape current local optima more effectively. The results of the Adaburst experiment underscore its enhanced performance in image classification and generation tasks when compared with alternative approaches, characterized by expedited convergence and elevated accuracy. Notably, on the MNIST, CIFAR-10, and CIFAR-100 datasets, Adaburst attained accuracies that matched or exceeded those achieved by SGDM. Furthermore, in training diffusion models on the DeepFashion dataset, Adaburst achieved convergence in fewer epochs than a meticulously calibrated AdamW optimizer while avoiding abrupt blurring or other training instabilities. Adaburst augmented the final training set accuracy on the MNIST, CIFAR-10, and CIFAR-100 datasets by 0.02%, 0.41%, and 4.18%, respectively. In addition, the generative model trained on the DeepFashion dataset demonstrated a 4.62-point improvement in the Frechet Inception Distance (FID) score, a metric for assessing generative model quality. Consequently, this evidence suggests that Adaburst introduces an innovative optimization algorithm that simultaneously updates AdamW and SGDM and incorporates a learning rate burst mechanism. This mechanism significantly enhances deep neural networks’ training speed and convergence accuracy.

Development of a deep learning model for the automated detection of green pixels indicative of gout on dual energy CT scan

BackgroundDual-energy CT (DECT) is a non-invasive way to determine the presence of monosodium urate (MSU) crystals in the workup of gout. Color-coding distinguishes MSU from calcium following material decomposition and post-processing. Most software labels MSU as green and calcium as blue. There are limitations in the current image processing methods of segmenting green-encoded pixels. Additionally, identifying green foci is tedious, and automated detection would improve workflow. This study aimed to determine the optimal deep learning (DL) algorithm for segmenting green-encoded pixels of MSU crystals on DECTs. MethodsDECT images of positive and negative gout cases were retrospectively collected. The dataset was split into train (N = 28) and held-out test (N = 30) sets. To perform cross-validation, the train set was split into seven folds. The images were presented to two musculoskeletal radiologists, who independently identified green-encoded voxels. Two 3D Unet-based DL models, Segresnet and SwinUNETR, were trained, and the Dice similarity coefficient (DSC), sensitivity, and specificity were reported as the segmentation metrics. ResultsSegresnet showed superior performance, achieving a DSC of 0.9999 for the background pixels, 0.7868 for the green pixels, and an average DSC of 0.8934 for both types of pixels, respectively. According to the post-processed results, the Segresnet reached voxel-level sensitivity and specificity of 98.72 % and 99.98 %, respectively. ConclusionIn this study, we compared two DL-based segmentation approaches for detecting MSU deposits in a DECT dataset. The Segresnet resulted in superior performance metrics. The developed algorithm provides a potential fast, consistent, highly sensitive and specific computer-aided diagnosis tool. Ultimately, such an algorithm could be used by radiologists to streamline DECT workflow and improve accuracy in the detection of gout.

Enhanced Whale Optimization and Ensemble Learning Algorithm for Autism Spectrum Disorder Classification

Enhanced Whale Optimization and Ensemble Learning Algorithm for Autism Spectrum Disorder Classification

Inkjet-printed MOS-based MEMS sensor array combined with one-dimensional convolutional neural network algorithm for identifying indoor harmful gases

To achieve a highly accurate and efficient analysis of mixed gases, it is crucial to develop an electronic nose system with high sensitivity of gas sensors and low data processing complexity. In this study, the metal oxide semiconductor (MOS) based micro-electromechanical system (MEMS) gas sensor array was prepared by inkjet printing sensing materials onto a micro-hotplate. The pattern recognition unit employed a one-dimensional convolutional neural network (1D-CNN) to identify 7 types of gases. While the optimal traditional machine learning algorithm achieved an 80% recognition accuracy, the 1D-CNN can achieve 99.8%. Furthermore, the impact of varying time series input lengths on the model accuracy was investigated, pinpointing an optimal sampling time of 15 s. This work showed that integrating the MEMS sensor array with the 1D-CNN algorithm might offer a promising approach for intricate gas classification and identification.

Reinvent 4: Modern AI-driven generative molecule design.

REINVENT 4 is a modern open-source generative AI framework for the design of small molecules. The software utilizes recurrent neural networks and transformer architectures to drive molecule generation. These generators are seamlessly embedded within the general machine learning optimization algorithms, transfer learning, reinforcement learning and curriculum learning. REINVENT 4 enables and facilitates de novo design, R-group replacement, library design, linker design, scaffold hopping and molecule optimization. This contribution gives an overview of the software and describes its design. Algorithms and their applications are discussed in detail. REINVENT 4 is a command line tool which reads a user configuration in either TOML or JSON format. The aim of this release is to provide reference implementations for some of the most common algorithms in AI based molecule generation. An additional goal with the release is to create a framework for education and future innovation in AI based molecular design. The software is available from https://github.com/MolecularAI/REINVENT4 and released under the permissive Apache 2.0 license. Scientific contribution. The software provides an open-source reference implementation for generative molecular design where the software is also being used in production to support in-house drug discovery projects. The publication of the most common machine learning algorithms in one code and full documentation thereof will increase transparency of AI and foster innovation, collaboration and education.

Distributed scheduling for multi-energy synergy system considering renewable energy generations and plug-in electric vehicles: A level-based coupled optimization method

Multi-energy synergy systems integrating high-penetration large-scale plug-in electric vehicles, distributed renewable energy generations, and battery energy storage systems have great potential to reduce the reliance of the grid on traditional fossil fuels. However, the random charging characteristics of plug-in electric vehicles and the uncertainty of photovoltaics may impose an additional burden on the grid and affect the supply–demand equilibrium. To address this issue, judicious scheduling optimization offers an effective solution. In this study, considering charge and discharge management of plug-in electric vehicles and intermittent photovoltaics, a novel Multi-energy synergy systems scheduling framework is developed for solving grid instability and unreliability issues. This formulates a large-scale mixed-integer problem, which calls for a powerful and effective optimizer. The new binary level-based learning optimization algorithm is proposed to address nonlinear large-scale high-coupling unit commitment problems. To investigate the feasibility of the proposed scheme, numerical experiments have been carried out considering multiple scales of unit numbers and various scenarios. Finally, the results confirm that the proposed scheduling framework is reasonable and effective in solving unit commitment problems, can achieve 3.3% cost reduction and demonstrates superior performance in handling large-scale energy optimization problems. The integration of plug-in electric vehicles, distributed renewable energy generations, and battery energy storage systems is verified to reduce the output power of 192.72 MW units during peak periods to improve grid stability. Therefore, optimizing energy utilization and distribution will become an indispensable part of future power systems.

Predicting the risk of hospital readmissions using a machine learning approach: a case study on patients undergoing skin procedures.

Even with modern advancements in medical care, one of the persistent challenges hospitals face is the frequent readmission of patients. These recurrent admissions not only escalate healthcare expenses but also amplify mental and emotional strain on patients. This research delved into two primary areas: unraveling the pivotal factors causing the readmissions, specifically targeting patients who underwent dermatological treatments, and determining the optimal machine learning algorithms that can foresee potential readmissions with higher accuracy. Among the multitude of algorithms tested, including logistic regression (LR), support vector machine (SVM), random forest (RF), Naïve Bayesian (NB), artificial neural network (ANN), xgboost (XG), and k-nearest neighbor (KNN), it was noted that two models-XG and RF-stood out in their prediction prowess. A closer inspection of the data brought to light certain patterns. For instance, male patients and those between the ages of 21 and 40 had a propensity to be readmitted more frequently. Moreover, the months of March and April witnessed a spike in these readmissions, with ~6% of the patients returning within just a month after their first admission. Upon further analysis, specific determinants such as the patient's age and the specific hospital where they were treated emerged as key indicators influencing the likelihood of their readmission.

Fake News Detection using Python and Machine Learning

In an era dominated by digital information, the unchecked proliferation of false information poses a critical threat. This has motivated to study for techniques which can tackle issues related to it, and hence the study uses machine learning algorithms to detect fake news. By systematically exploring and evaluating various machine learning models, the research enhances decision-making, fortifies information integrity and addresses the adverse impacts of misinformation on a broader scale. Going beyond mere algorithmic efficacy, the study introduces a multifaceted approach by using evaluation metrics like precision and accuracy. This strategic analysis identifies the optimal machine learning algorithm for classifying articles as real or fake news. The contents covered by the study include literature review, dataset selection, data preprocessing and cleaning, vectorization, model and metric selection, model training, optimization and evaluation. The study extends its impact beyond technical proficiency, actively contributing to bolstering trust across media, democratic processes, and ensuring content authenticity. This inclusive approach benefits a wide array of stakeholders including news organizations, social media platforms and government entities. Through rigorous evaluation, the study significantly enhances information credibility, providing a countermeasure against the growing threat of misinformation.

Hyper-parameter Optimization in the context of Smart Manufacturing: a Systematic Literature Review

Smart manufacturing involves the use of a variety of automation solutions, such as robotics, machines with embedded software, and advanced sensors collecting vast quantities of data. Efficient control of a complex composition of such solutions, as well as the analysis of the collected data, are essential for improving the efficiency of the production processes and decision-making. Data analysis and process optimization are enabled through the application of state-of-the-art optimization and machine learning algorithms. However, the efficient use of these algorithms often depends on the careful selection of the parameters, which on its own is a process that requires a high degree of expertise and time. Therefore, another class of algorithms can be applied that is designed to discover the optimal parameter configuration given the specific nature of the manufacturing process and the used algorithm. In this work, we systematically analyze the published literature to discover which parameter selection techniques are used in the context of Industry 4.0, for which processes, and how these benefit from automated parameter selection. Within our literature review, we discover nine relevant publications, most of which concentrate on parameter selection for machine learning algorithms through various numerical optimization and metaheuristic techniques.

Energy management of integrated energy system in the park under multiple time scales

&lt;abstract&gt; &lt;p&gt;Considering the problem of time scale differences among subsystems in the integrated energy system of a park, as well as the increasing complexity of the system structure and number of control variables, there may be a deep reinforcement learning (DRL) "curse of dimensionality" problem, which hinders the further improvement of economic benefits and energy utilization efficiency of park-level integrated energy systems (PIES). This article proposes a reinforcement learning optimization algorithm for comprehensive energy PPO (Proximal Policy Optimization) in industrial parks considering multiple time scales for energy management. First, PIES are divided into upper and lower layers, the first containing power and thermal systems, and the second containing gas systems. The upper and lower layers of energy management models are built based on the PPO; then, both layers formulate the energy management schemes of the power, thermal, and gas systems in a long (30 min) and short time scale (6 min). Through confirmatory and comparative experiments, it is shown that the proposed method can not only effectively overcome the curse of dimensionality in DRL algorithms during training but can also develop different energy system management plans for PIES on a differentiated time scale, improving the overall economic benefits of the system and reducing carbon emissions.&lt;/p&gt; &lt;/abstract&gt;

Human Learning Optimization Algorithm With Diversified Searches

Human Learning Optimization Algorithm With Diversified Searches

A study on automated improvement of securities trading strategies using machine learning optimization algorithms

Abstract Automation in securities trading offers advantages over human subjective trading, such as immunity to subjective emotional factors, high efficiency, and the ability to monitor multiple stocks simultaneously, making it a cutting-edge development path in the securities trading industry. In this paper, we first apply the concept of time-frequency decomposition, gradually moving from the first-order moments of securities prices to the higher-order moments. We then combine this with the EMD time-frequency decomposition method to analyze the securities price sequence and extract the characteristics of the securities price fluctuations. Finally, we use the differential long- and short-term memory network to construct an automatic optimization trading system. We compare the system’s performance with traditional technical analysis indexes, as well as the annualized returns of PPO and A2C models on various securities, to verify its performance under unilateral rising, oscillating rising, and plummeting quotes. Finally, we conducted a live test on 1000 GEM stocks. The system in this paper outperforms all traditional technical indicators, with an average annualized return of 71.85% at the lowest and 127.27% at the highest among 5 securities, demonstrating excellent performance. In the three quotes of Ningde Times, Aier Dental, and Goldfish that are rising one way, rising and falling over time, and rising again, the annualized returns of this paper’s system are 77.13%, 67.16%, and 12.66%, which are higher than those of the PPO and A2C models.