Reinforcement Learning Mechanism Research Articles

Deep reinforcement learning can promote sustainable human behaviour in a common-pool resource problem

A canonical social dilemma arises when resources are allocated to people, who can either reciprocate with interest or keep the proceeds. The right resource allocation mechanisms can encourage levels of reciprocation that sustain the commons. Here, in an iterated multiplayer trust game, we use deep reinforcement learning (RL) to design a social planner that promotes sustainable contributions from human participants. We first trained neural networks to behave like human players, creating a stimulated economy that allows us to study the dynamics of receipt and reciprocation. We use RL to train a mechanism to maximise aggregate return to players. The RL mechanism discovers a redistributive policy that leads to a large but also more equal surplus. The mechanism outperforms baseline mechanisms by conditioning its generosity on available resources and temporarily sanctioning defectors. Examining the RL policy allows us to develop a similar but explainable mechanism that is more popular among players.

Intelligent Resilient Security Control for Fractional-Order Multiagent Networked Systems Using Reinforcement Learning and Event-Triggered Communication Mechanism.

The main objective of this study is to develop an intelligent, resilient event-triggered control method for fractional-order multiagent networked systems (FOMANSs) using reinforcement learning (RL) to address challenges resulting from unknown dynamics, actuator faults, and denial-of-service (DoS) attacks. First, the challenge of unknown system dynamics within their environment must be addressed to achieve desired system stability in the face of unknown dynamics or to optimize consensus in FOMANSs. To address this problem, an adaptive learning law is implemented to handle unknown nonlinear dynamics, parameterized by a neural network, which establishes weights for a fuzzy logic system utilized in cooperative tracking protocols. A novel distributed control policy facilitates signal sharing through RL among agents, reducing error variables through learning. Moreover, this study combines an RL algorithm with the sliding mode control strategy to optimize the parameterization of the distributed control protocol, thereby eliminating its constraints on initial conditions. Second, realizing that DoS attacks typically make the actuator signal inaccessible for distributed control protocols, an innovative intelligent dual-event-triggered control strategy is formulated to reduce the effects of DoS attacks. By coordinating nested event triggers across various channels, the distributed control input is protected from incorrect signals from DoS attacks, thus ensuring its resilience. To address this problem, an intelligent security dual-event-triggered control protocol guarantees Mittag-Leffler stability of the closed-loop system and ensures effective sliding motion conditions. This distributed control protocol ensures robust tracking of control tasks and mitigates "Zeno behavior" during event triggering. The proposed control strategy is validated using a single-link flexible-joint robotic manipulator system.

A Co-Evolution Algorithm With Dueling Reinforcement Learning Mechanism for the Energy-Aware Distributed Heterogeneous Flexible Flow-Shop Scheduling Problem

A Co-Evolution Algorithm With Dueling Reinforcement Learning Mechanism for the Energy-Aware Distributed Heterogeneous Flexible Flow-Shop Scheduling Problem

Policy Complexity Suppresses Dopamine Responses.

Limits on information processing capacity impose limits on task performance. We show that male and female mice achieve performance on a perceptual decision task that is near-optimal given their capacity limits, as measured by policy complexity (the mutual information between states and actions). This behavioral profile could be achieved by reinforcement learning with a penalty on high complexity policies, realized through modulation of dopaminergic learning signals. In support of this hypothesis, we find that policy complexity suppresses midbrain dopamine responses to reward outcomes. Furthermore, neural and behavioral reward sensitivity were positively correlated across sessions. Our results suggest that policy compression shapes basic mechanisms of reinforcement learning in the brain.

Utilizing reinforcement learning and support vector machine to enhance feedback mechanism in English speech training

This study proposes an innovative approach to enhance the feedback mechanism in English speech training by combining reinforcement learning and support vector machine (SVM) technologies. Traditional feedback mechanisms for English speech training face challenges such as feedback delay, subjectivity, insufficient personalization, technical limitations, and lack of long-term analysis. To address these issues, the SVM model is used to accurately recognize English phonemes or words, enabling real-time evaluation of students’ pronunciation. The reinforcement learning mechanism leverages these real-time evaluations to make personalized adjustments, offering immediate feedback and dynamic training plans. The integration of these two technologies resulted in a multi-dimensional feedback system for assessing pronunciation performance and providing continuous improvement suggestions through long-term data analysis. Experimental results show that the reinforcement learning system using a Q-learning model maintained a high reward mean, significantly improving training effectiveness. Additionally, the SVM model achieved a pronunciation classification accuracy of over 86.54%, with an F1-score of 94.65%. In an 8-week experiment, the experimental group (using the combined Q-learning and SVM system) outperformed the control group (using traditional methods) in pronunciation scores. The experimental group’s score range expanded to 85–94, with a median increase to 89, while the control group’s range was 78–87, with a median increase to 82. The learning progress in the experimental group was significantly higher, demonstrating the effectiveness and superiority of the proposed method.

Spatial reciprocity under reinforcement learning mechanism.

At present, the research on the dynamics of cooperative behavior of agents under reinforcement learning mechanism either assumes that agents have global interaction, that is, agents interact with all other agents in the population, or directly study the influence of relevant factors on cooperation evolution based on the local interaction in a network structure. It neglects to formally study how the limitation of agents that only interact with local agents affects their strategy choice. Thus, in this paper, we study the cooperative behavior of agents in a typical social decision-making environment with conflicts between individual interests and collective interests. On the one hand, a programmed game model in game theory, namely, prisoner's dilemma game, is used to capture the essence of real-world dilemmas. On the other hand, the effects of local and global strategy learning on the cooperative evolution of agents are investigated separately, and the nature of spatial reciprocity under the reinforcement learning mechanism is found. Specifically, when there is no inherent connection between the interacting agents and the learning agents within the system, the network structure has a limited effect on promoting cooperation. It is only when there is an overlap between the interacting agents and the learning agents that the spatial reciprocity effect observed in the traditional evolutionary game theory can be fully realized.

Q-Learning-Driven Butterfly Optimization Algorithm for Green Vehicle Routing Problem Considering Customer Preference.

This paper proposes a Q-learning-driven butterfly optimization algorithm (QLBOA) by integrating the Q-learning mechanism of reinforcement learning into the butterfly optimization algorithm (BOA). In order to improve the overall optimization ability of the algorithm, enhance the optimization accuracy, and prevent the algorithm from falling into a local optimum, the Gaussian mutation mechanism with dynamic variance was introduced, and the migration mutation mechanism was also used to enhance the population diversity of the algorithm. Eighteen benchmark functions were used to compare the proposed method with five classical metaheuristic algorithms and three BOA variable optimization methods. The QLBOA was used to solve the green vehicle routing problem with time windows considering customer preferences. The influence of decision makers' subjective preferences and weight factors on fuel consumption, carbon emissions, penalty cost, and total cost are analyzed. Compared with three classical optimization algorithms, the experimental results show that the proposed QLBOA has a generally superior performance.

A Hybrid Adaptive Dynamic Programming for Optimal Tracking Control of USVs.

This article presents an efficient method for solving the optimal tracking control policy of unmanned surface vehicles (USVs) using a hybrid adaptive dynamic programming (ADP) approach. This approach integrates data-driven integral reinforcement learning (IRL) and dynamic event-driven (DED) mechanisms into the solution of the control policy of the established augmented system while obtaining both the feedforward and feedback components of the tracking controller. For the USV model and the reference trajectory, an augmented system is established, and the tracking Hamilton-Jacobi-Bellman (HJB) equation is derived based on IRL, aiming to fully utilize system data information and reduce model dependency. For the solution of the tracking HJB equation, the DED-based controller update rule is used to further reduce the burden of network transmission. In implementing the ADP method, the DED experience replay-based weight update rule is utilized to recycle data resources. Experiments show that compared with the static event-driven (SED) approach, the DED approach reduces the sample size by and increases the average interval by about four times.

Altered Neural Responses to Punishment Learning in Conduct Disorder.

Altered Neural Responses to Punishment Learning in Conduct Disorder.

Weight Transfer in the Reinforcement Learning Model of Songbird Acquisition.

Song acquisition behavior observed in the songbird system provides a notable example of learning through trial- and-error which parallels human speech acquisition. Studying songbird vocal learning can offer insights into mechanisms underlying human language. We present a computational model of song learning that integrates reinforcement learning (RL) and Hebbian learning and agrees with known songbird circuitry. The song circuit outputs activity from nucleus RA, which receives two primary inputs: timing information from area HVC and stochastic activity from nucleus LMAN. Additionally, song learning relies on Area X, a basal ganglia area that receives dopaminergic inputs from VTA. In our model, song is first acquired in the HVC-to-Area X connectivity, employing an RL mechanism that involves node perturbation. This information is then consolidated into HVC-to-RA synapses through a Hebbian mechanism. The transfer of weights from Area X to RA takes place via the thalamus, utilizing a specific form of spike-timing-dependent plasticity (STDP). Thus, we present a computational model grounded in songbird circuitry in which the optimal policy is initially guided by RL and subsequently transferred to another circuit through Hebbian plasticity.

Opportunities, challenges and future perspectives of using bioinformatics and artificial intelligence techniques on tropical disease identification using omics data.

Tropical diseases can often be caused by viruses, bacteria, parasites, and fungi. They can be spread over vectors. Analysis of multiple omics data types can be utilized in providing comprehensive insights into biological system functions and disease progression. To this end, bioinformatics tools and diverse AI techniques are pivotal in identifying and understanding tropical diseases through the analysis of omics data. In this article, we provide a thorough review of opportunities, challenges, and future directions of utilizing Bioinformatics tools and AI-assisted models on tropical disease identification using various omics data types. We conducted the review from 2015 to 2024 considering reliable databases of peer-reviewed journals and conference articles. Several keywords were taken for the article searching and around 40 articles were reviewed. According to the review, we observed that utilization of omics data with Bioinformatics tools like BLAST, and Clustal Omega can make significant outcomes in tropical disease identification. Further, the integration of multiple omics data improves biomarker identification, and disease predictions including disease outbreak predictions. Moreover, AI-assisted models can improve the precision, cost-effectiveness, and efficiency of CRISPR-based gene editing, optimizing gRNA design, and supporting advanced genetic correction. Several AI-assisted models including XAI can be used to identify diseases and repurpose therapeutic targets and biomarkers efficiently. Furthermore, recent advancements including Transformer-based models such as BERT and GPT-4, have been mainly applied for sequence analysis and functional genomics. Finally, the most recent GeneViT model, utilizing Vision Transformers, and other AI techniques like Generative Adversarial Networks, Federated Learning, Transfer Learning, Reinforcement Learning, Automated ML and Attention Mechanism have shown significant performance in disease classification using omics data.

A Linear Programming-Based Reinforcement Learning Mechanism for Incomplete-Information Games

A Linear Programming-Based Reinforcement Learning Mechanism for Incomplete-Information Games

Reinforcement Learning for Synchronization of Heterogeneous Multiagent Systems by Improved Q-Functions.

This article dedicates to investigating a methodology for enhancing adaptability to environmental changes of reinforcement learning (RL) techniques with data efficiency, by which a joint control protocol is learned using only data for multiagent systems (MASs). Thus, all followers are able to synchronize themselves with the leader and minimize their individual performance. To this end, an optimal synchronization problem of heterogeneous MASs is first formulated, and then an arbitration RL mechanism is developed for well addressing key challenges faced by the current RL techniques, that is, insufficient data and environmental changes. In the developed mechanism, an improved Q-function with an arbitration factor is designed for accommodating the fact that control protocols tend to be made by historic experiences and instinctive decision-making, such that the degree of control over agents' behaviors can be adaptively allocated by on-policy and off-policy RL techniques for the optimal multiagent synchronization problem. Finally, an arbitration RL algorithm with critic-only neural networks is proposed, and theoretical analysis and proofs of synchronization and performance optimality are provided. Simulation results verify the effectiveness of the proposed method.

Optimized dead-zone inverse control using reinforcement learning and sliding-mode mechanism for a class of high-order nonlinear systems

Optimized dead-zone inverse control using reinforcement learning and sliding-mode mechanism for a class of high-order nonlinear systems

Enhanced reinforcement learning-based two-way transmit-receive directional antennas neighbor discovery in wireless ad hoc networks

Enhanced reinforcement learning-based two-way transmit-receive directional antennas neighbor discovery in wireless ad hoc networks

Modeling and simulation of Fuzzy-rule based WBAN using multi-level fuzzy colored petri-nets and reinforcement learning

Modeling and simulation of Fuzzy-rule based WBAN using multi-level fuzzy colored petri-nets and reinforcement learning

Multi-UAVs task allocation method based on MPSO-SA-DQN

Multi-UAVs play an important role in the battlefield. Although many methods are proposed to solve the Multi-UAV task allocation, there still existing the problems of complex time constraints and uncertain solution space. The reason is that multi-UAVs usually face changing environmental factors. Aiming at solving such problem, this paper proposes a multi-UAV task assignment method based on Deep Q-based evolutionary reinforcement learning algorithms (MPSO-SA-DQN). Specifically, this method builds a multi-agent training framework based on the deep evolutionary reinforcement learning mechanism and SA-DQN. Its aim is to improve the global exploration and optimization capabilities of multi-agents. At the same time, the multi-dimensional particle swarm optimization algorithm is introduced to optimize the state space. Based on task priority mapping, the MPSO-SA-DQN algorithm framework is proposed. As a result, multi-agents can optimize the execution state in real time in the environment interaction. Besides, it also has the ability to reach optimal state and maximum reward. According to the characteristics of multi-UAV global task assignment, this paper designs a priority state space autoencoder strategy and global task feature. A multi-UAVs tasks allocation and iterative optimization method based on MPSO-SA-DQN algorithm is proposed, so as to continuously optimize the task allocation scheme. The simulation results show that the multi-UAV task allocation method based on MPSO-SA-DQN can effectively solve the problem of uncertainty in the optimal solution space of task allocation. At the same time, the algorithm achieves faster convergence result, and a good prospect of promotion in the field of UAV swarm cooperative task planning.

Disentangling negative reinforcement, working memory, and deductive reasoning deficits in elevated BMI

Disentangling negative reinforcement, working memory, and deductive reasoning deficits in elevated BMI

A novel energy-efficiency framework for UAV-assisted networks using adaptive deep reinforcement learning

In the air-to-ground transmissions, the lifespan of the network is based on the “unmanned aerial vehicle's (UAV)” life span because of the limited battery capacity. Thus, the enhancement of energy efficiency and the outage of the ground candidate’s minimization are significant factors of the network functionality. UAV-aided transmission can highly enhance the spectrum efficacy and coverage. Because of their flexible deployment and the high maneuverability, the UAVs can be the best alternative for the situations where the “Internet of Things (IoT)” systems utilize more energy to attain the essential information rate, when they are far away from the terrestrial base station. Therefore, it is significant to win over the few troubles in the conventional UAV-aided efficiency approaches. Thus, this proposed work is aimed to design an innovative energy efficiency framework in the UAV-assisted network using a reinforcement learning mechanism. The energy efficiency optimization in the UAV offers better wireless coverage to the static and mobile ground user. Presently, reinforcement learning techniques effectively optimize the energy efficiency rate of the system by employing the 2D trajectory mechanism, which effectively removes the interference rate attained in the nearby UAV cells. The main objective of the recommended framework is to maximize the energy efficiency rate of the UAV network by performing the joint optimization using UAV 3D trajectory, with the energy utilized during interference accounting, and connected user counts. Hence, an efficient Adaptive Deep Reinforcement Learning with Novel Loss Function (ADRL-NLF) framework is designed to provide a better energy efficiency rate to the UAV network. Moreover, the parameter of ADRL is tuned using the Hybrid Energy Valley and Hermit Crab (HEVHC) algorithm. Various experimental observations are performed to observe the effectualness rate of the recommended energy efficiency model for UAV-based networks over the classical energy efficiency framework in UAV Networks.

How to improve “construct, merge, solve and adapt"? Use reinforcement learning!

AbstractIn this work, we propose a new variant of construct, merge, solve, and adapt (CMSA), which is a recently introduced hybrid metaheuristic for combinatorial optimization. Our newly proposed variant, named reinforcement learning CMSA (RL-CMSA), makes use of a reinforcement learning (RL) mechanism trained online with data gathered during the search process. In addition to generally outperforming standard CMSA, this new variant proves to be more flexible as it does not require a greedy function for the evaluation of solution components at each solution construction step. We present RL-CMSA as a general framework for enhancing CMSA by leveraging a simple RL learning process. Moreover, we study a range of specific designs for the employed learning mechanism. The advantages of the introduced CMSA variant are demonstrated in the context of the far from most string and minimum dominating set problems, showing the improvement in performance and simplicity with respect to standard CMSA. In particular, the best performing RL-CMSA variant proposed is statistically significantly better than the standard algorithm for both problems, obtaining 1.28% and 0.69% better results on average respectively.