Genetic Reinforcement Learning Research Articles

Unveiling Genetic Reinforcement Learning (GRLA) and Hybrid Attention-Enhanced Gated Recurrent Unit with Random Forest (HAGRU-RF) for Energy-Efficient Containerized Data Centers Empowered by Solar Energy and AI

The adoption of renewable energy sources has seen a significant rise in recent years across various industrial sectors, with solar energy standing out due to its eco-friendly characteristics. This shift from conventional fossil fuels to solar power is particularly noteworthy in energy-intensive environments such as cloud data centers. These centers, which operate continuously to support active servers via virtual instances, present a critical opportunity for the integration of sustainable energy solutions. In this study, we introduce two innovative approaches that substantially advance data center energy management. Firstly, we introduce the Genetic Reinforcement Learning Algorithm (GRLA) for energy-efficient container placement, representing a pioneering approach in data center management. Secondly, we propose the Hybrid Attention-enhanced GRU with Random Forest (HAGRU-RF) model for accurate solar energy prediction. This model combines GRU neural networks with Random Forest algorithms to forecast solar energy production reliably. Our primary focus is to evaluate the feasibility of solar energy in meeting the energy demands of cloud data centers that utilize containerization for virtualization, thereby promoting green cloud computing. Leveraging a robust German photovoltaic energy dataset, our study demonstrates the effectiveness and adaptability of these techniques across diverse environmental contexts. Furthermore, comparative analysis against traditional methods highlights the superior performance of our models, affirming the potential of solar-powered data centers as a sustainable and environmentally responsible solution.

Multi-modal framework to model wet milling through numerical simulations and artificial intelligence (part 2)

Modelling of stirred media mills is crucial because of their broad utilisation in various industries, ranging from mechanochemistry and mining to the production of batteries and pharmaceuticals. Stirred media mills are responsible for a considerable portion of the global energy demand. However, requirements exist regarding highly specific or uniform particle sizes, process conditions, and reduced wear or abrasion. Multi-modal modelling, which is the intelligent integration of different approaches, such as experiments, simulations, and AI, benefits from respective advantages of each approach. In the first study, results of an experiment conducted via magnetic tracking of a tracer bead was compared with those of simulations, and the inner mill mechanisms were investigated. The two-way coupled computer fluid dynamics discrete element method (CFD-DEM) simulations allowed the investigation of subsequent modelling through AI methods [1]. A novel AI training technique called “genetic reinforcement learning” (hereinafter, GRL), which combines neural nets with genetic algorithms, was demonstrated for cases with limited data. Furthermore, genetic programming was applied to derive transparent mathematical equations based on the generated data. Using these methods and experimentally validated simulation data, predictive models were trained, and mathematical equations were derived. Relative velocity distributions in the entire simulation domain as well as spatial distributions via heatmaps were predicted and evaluated for independent cases. Systematic predictions for the characteristic relative velocity values were generated instantaneously for varying tip speeds and bead diameters in a parameter space, which would have required 1–10 years through simulations. Finally, a transparent equation was generated via genetic programming.

Optimising Performance for NB-IoT UE Devices through Data Driven Models

This paper presents a data driven framework for performance optimisation of Narrow-Band IoT user equipment. The proposed framework is an edge micro-service that suggests one-time configurations to user equipment communicating with a base station. Suggested configurations are delivered from a Configuration Advocate, to improve energy consumption, delay, throughput or a combination of those metrics, depending on the user-end device and the application. Reinforcement learning utilising gradient descent and genetic algorithm is adopted synchronously with machine and deep learning algorithms to predict the environmental states and suggest an optimal configuration. The results highlight the adaptability of the Deep Neural Network in the prediction of intermediary environmental states, additionally the results present superior performance of the genetic reinforcement learning algorithm regarding its performance optimisation.

Cloud Detection of SuperView-1 Remote Sensing Images Based on Genetic Reinforcement Learning

Cloud pixels have massively reduced the utilization of optical remote sensing images, highlighting the importance of cloud detection. According to the current remote sensing literature, methods such as the threshold method, statistical method and deep learning (DL) have been applied in cloud detection tasks. As some cloud areas are translucent, areas blurred by these clouds still retain some ground feature information, which blurs the spectral or spatial characteristics of these areas, leading to difficulty in accurate detection of cloud areas by existing methods. To solve the problem, this study presents a cloud detection method based on genetic reinforcement learning. Firstly, the factors that directly affect the classification of pixels in remote sensing images are analyzed, and the concept of pixel environmental state (PES) is proposed. Then, PES information and the algorithm’s marking action are integrated into the “PES-action” data set. Subsequently, the rule of “reward–penalty” is introduced and the “PES-action” strategy with the highest cumulative return is learned by a genetic algorithm (GA). Clouds can be detected accurately through the learned “PES-action” strategy. By virtue of the strong adaptability of reinforcement learning (RL) to the environment and the global optimization ability of the GA, cloud regions are detected accurately. In the experiment, multi-spectral remote sensing images of SuperView-1 were collected to build the data set, which was finally accurately detected. The overall accuracy (OA) of the proposed method on the test set reached 97.15%, and satisfactory cloud masks were obtained. Compared with the best DL method disclosed and the random forest (RF) method, the proposed method is superior in precision, recall, false positive rate (FPR) and OA for the detection of clouds. This study aims to improve the detection of cloud regions, providing a reference for researchers interested in cloud detection of remote sensing images.

Intelligent learning agent for collaborative virtual workspace

PurposeIntelligent agents are becoming an essential part of collaborative virtual environments. The purpose of this paper is to present an architecture of a learning agent that is able to utilize machine learning techniques to monitor the user's actions.Design/methodology/approachA learning agent is developed and integrated into federated collaborative virtual workspace.FindingsThe experimental results showed that the combination of genetic algorithms and reinforcement learning algorithms provides the agent with better learning capability resulting in better predictions for the user.Originality/valueThis paper provides experimental results and a performance analysis in terms of accuracy of predictions, processing time, and memory utilization of the agent.

Les systèmes de classeurs. Une présentation générale

This article concerns the presentation of classifiers systems built to learn interactions. Different kinds of classifiers systems are presented, using a chronological approach introducing new challenges or concepts approached by every model. We describe used concepts: genetic algorithms and reinforcement learning. The article presents basics systems such as ZCS or XCS, systems for anticipation, as well as hierarchicals or heterogenous classifiers.

A novel genetic reinforcement learning for nonlinear fuzzy control problems

Unlike a supervise learning, a reinforcement learning problem has only very simple “evaluative” or “critic” information available for learning, rather than “instructive” information. A novel genetic reinforcement learning, called reinforcement sequential-search-based genetic algorithm (R-SSGA), is proposed for solving the nonlinear fuzzy control problems in this paper. Unlike the traditional reinforcement genetic algorithm, the proposed R-SSGA method adopts the sequential-search-based genetic algorithms (SSGA) to tune the fuzzy controller. Therefore, the better chromosomes will be initially generated while the better mutation points will be determined for performing efficient mutation. The adjustable parameters of fuzzy controller are coded as real number components. We formulate a number of time steps before failure occurs as a fitness function. Simulation results have shown that the proposed R-SSGA method converges quickly and minimizes the population size.

Combination of online clustering and Q-value based GA for reinforcement fuzzy system design

This paper proposes a combination of online clustering and Q-value based genetic algorithm (GA) learning scheme for fuzzy system design (CQGAF) with reinforcements. The CQGAF fulfills GA-based fuzzy system design under reinforcement learning environment where only weak reinforcement signals such as success and failure are available. In CQGAF, there are no fuzzy rules initially. They are generated automatically. The precondition part of a fuzzy system is online constructed by an aligned clustering-based approach. By this clustering, a flexible partition is achieved. Then, the consequent part is designed by Q-value based genetic reinforcement learning. Each individual in the GA population encodes the consequent part parameters of a fuzzy system and is associated with a Q-value. The Q-value estimates the discounted cumulative reinforcement information performed by the individual and is used as a fitness value for GA evolution. At each time step, an individual is selected according to the Q-values, and then a corresponding fuzzy system is built and applied to the environment with a critic received. With this critic, Q-learning with eligibility trace is executed. After each trial, GA is performed to search for better consequent parameters based on the learned Q-values. Thus, in CQGAF, evolution is performed immediately after the end of one trial in contrast to general GA where many trials are performed before evolution. The feasibility of CQGAF is demonstrated through simulations in cart-pole balancing, magnetic levitation, and chaotic system control problems with only binary reinforcement signals.

System for foreign exchange trading using genetic algorithms and reinforcement learning

Foreign exchange trading has emerged recently as a significant activity in many countries. As with most forms of trading, the activity is influenced by many random parameters so that the creation of a system that effectively emulates the trading process will be very helpful. A major issue for traders in the deregulated Foreign Exchange Market is when to sell and when to buy a particular currency in order to maximize profit. This paper presents novel trading strategies based on the machine learning methods of genetic algorithms and reinforcement learning.

Predicting direction shifts on Canadian–US exchange rates with artificial neural networks

AbstractThe paper presents a variety of neural network models applied to Canadian–US exchange rate data. Networks such as backpropagation, modular, radial basis functions, linear vector quantization, fuzzy ARTMAP, and genetic reinforcement learning are examined. The purpose is to compare the performance of these networks for predicting direction (sign change) shifts in daily returns. For this classification problem, the neural nets proved superior to the naïve model, and most of the neural nets were slightly superior to the logistic model. Using multiple previous days' returns as inputs to train and test the backpropagation and logistic models resulted in no increased classification accuracy. The models were not able to detect a systematic affect of previous days' returns up to fifteen days prior to the prediction day that would increase model performance. Copyright © 2001 John Wiley & Sons, Ltd.

Genetic reinforcement learning through symbiotic evolution for fuzzy controller design

An efficient genetic reinforcement learning algorithm for designing fuzzy controllers is proposed in this paper. The genetic algorithm (GA) adopted in this paper is based upon symbiotic evolution which, when applied to fuzzy controller design, complements the local mapping property of a fuzzy rule. Using this Symbiotic-Evolution-based Fuzzy Controller (SEFC) design method, the number of control trials, as well as consumed CPU time, are considerably reduced when compared to traditional GA-based fuzzy controller design methods and other types of genetic reinforcement learning schemes. Moreover, unlike traditional fuzzy controllers, which partition the input space into a grid, SEFC partitions the input space in a flexible way, thus creating fewer fuzzy rules. In SEFC, different types of fuzzy rules whose consequent parts are singletons, fuzzy sets, or linear equations (TSK-type fuzzy rules) are allowed. Further, the free parameters (e.g., centers and widths of membership functions) and fuzzy rules are all tuned automatically. For the TSK-type fuzzy rule especially, which put the proposed learning algorithm in use, only the significant input variables are selected to participate in the consequent of a rule. The proposed SEFC design method has been applied to different simulated control problems, including the cart-pole balancing system, a magnetic levitation system, and a water bath temperature control system. The proposed SEFC has been verified to be efficient and superior from these control problems, and from comparisons with some traditional GA-based fuzzy systems.

Genetic reinforcement learning approach to the heterogeneous machine scheduling problem

Focuses on the development of a learning-based heuristic for scheduling heterogeneous machines. Although list scheduling methods have been widely used for a large class of scheduling problems, including the heterogeneous machine scheduling problem, they involve designing priority rules, which usually require a fair amount of insights on the characteristics of the problem to be solved. Instead of elaborate design of priority rules in a single step, we propose an iterative list scheduling process, which refines priority rules while generating a number of schedules. The proposed iterative list scheduling is formulated as a reinforcement learning problem, with states and actions defined in list scheduling. Due to the large number of possible states, reinforcement learning algorithms which use value functions in constructing an optimal policy may not be suitable for scheduling problems. Thus, to directly work with policies rather than the values of states, we propose genetic reinforcement learning (GRL), in which the policies of reinforcement learning are encoded into the chromosomes of genetic algorithms and a near-optimal policy is searched for by genetic algorithms. A GRL-based scheduler, called evolutionary intracell scheduler (EVIS), has been developed and applied to various scheduling problems such as the heterogeneous machine scheduling, the processor scheduling, the job-shop scheduling, the flow-shop scheduling, and the open-shop scheduling problems. The proposed model of EVIS, which has a linear order of population-fitness convergence, is verified by computer experiments. Even without fine tuning EVIS, the quality of solutions achieved by EVIS is comparable to that of problem-tailored heuristics for most of the problem instances.

Genetic reinforcement learning for neurocontrol problems

Empirical tests indicate that at least one class of genetic algorithms yields good performance for neural network weight optimization in terms of learning rates and scalability. The successful application of these genetic algorithms to supervised learning problems sets the stage for the use of genetic algorithms in reinforcement learning problems. On a simulated inverted-pendulum control problem, “genetic reinforcement learning” produces competitive results with AHC, another well-known reinforcement learning paradigm for neural networks that employs the temporal difference method. These algorithms are compared in terms of learning rates, performance-based generalization, and control behavior over time.