Reinforcement Learning Research Articles

Reinforcement learning for online testing of autonomous driving systems: a replication and extension study

Reinforcement learning for online testing of autonomous driving systems: a replication and extension study

Energy‐Aware Routing in WSN Utilizing Self‐Attention–Based Cycle‐Consistent Generative Adversarial Network With Honey Badger Algorithm

ABSTRACTEnergy efficiency and secure data transmission are considered as the main design targets of wireless sensor network (WSN). Previous energy‐aware cluster head (CH) selection approaches could not provide secured data transmission. To address this problem, a self‐attention–based cycle‐consistent generative adversarial network (SaCyCsGAN) with honey badger algorithm (HyBA)–adopted energy‐aware routing (EgAR) in WSN is proposed. Initially, the proposed system uses a CH to carry out the routing practice. Accordingly, SaCyCsGAN classifier is exploited for CH selection under firm fitness function: delay, distance, energy, cluster density, and traffic rate. Afterward CH selection, a spiteful node may enter the cluster and acts as a CH. Hence, best path selection is needed. For that, HyBA is utilized, it selects the best path depending upon triplet parameter: trust, connectivity, and service degree. Finally, data are conveying into base station (BS) and vice versa under best path. Finally, the proposed EgAR‐WSN‐SaCyCsGAN‐HyBA method attains 24.13%, 28.57%, 5.88%, and 20.37% higher throughput and 18.50%, 21.67%, 13.33%, and 22.22% lesser energy consumption evaluated to the existing methods such as multiple‐objective CH utilizing self‐attention basis progressive generative adversarial network for safe data aggregation (EgAR‐WSN‐SAPrGAN‐ArVOA), reinforcement learning–based energy‐efficient optimized routing protocol in WSN (EgAR‐WSN‐RLGT‐BCSA), data aggregation including clustering protocol in WSN under machine learning (EgAR‐WSN‐AfNN), and optimum cluster along trusted path for routing formation with categorization of intrusion under machine learning categorization (EgAR‐WSN‐RsBPDT‐HoCSOA).

Optimizing container relocation operations by using deep reinforcement learning

Optimizing container relocation operations by using deep reinforcement learning

Optimization of High-Frequency Trading Strategies Using Deep Reinforcement Learning

This study presents a new method for optimising high-risk trading (HFT) strategies using deep learning (DRL). We propose a multi-time DRL framework integrating advanced neural network architectures with sophisticated business data processing techniques. The framework employs a combination of convolutional neural networks for manual order analysis, short-term memory networks for time series processing, and a multi-head listening mechanism for body fusion. We formulate the HFT problem based on Markov Decision Processes and use the Proximal Policy Optimization algorithm for training. The model is evaluated using tick-by-tick data from the NASDAQ exchange, including ten liquid stocks in 6 months. The experimental results show the superiority of our method, achieving a Sharpe ratio of 3.42, outperforming the learning model and machine learning based on benchmarks up to 33%. The proposed strategy has demonstrated strong performance across a wide range of regulatory markets and has shown potential for strategic objectives. Sensitivity analysis confirms the model's stability across a range of hyperparameters. Our findings suggest that the DRL-based approach can improve HFT performance and provide better market adaptation and risk management. This research leads to the continuous evolution of algorithmic trading strategies and shows the potential of AI-driven approaches in financial markets.

Hybrid offline-online reinforcement learning for obstacle avoidance in autonomous underwater vehicles

Hybrid offline-online reinforcement learning for obstacle avoidance in autonomous underwater vehicles

Autonomous ecologies of construction: Collaborative modular robotic material eco-systems with deep multi-agent reinforcement learning

To address the unprecedented challenges of construction pressurized by the global climate crisis, housing shortage, and growing shortage of skilled labor, this research presents a radical shift in the construction lifecycle of buildings, from linear processes that produce static continuous buildings to interrelational processes linking adaptative eco-systems of collaborative robots and reconfigurable building parts. Inspired by natural builders, the interdisciplinary field of collective robotic construction (CRC) offers the potential for scalable, adaptive, and resilient construction with simple robots. We establish a design framework for autonomous collaborative robotic construction (ACRC) through modular robotic material eco-systems (MRMES) trained with deep multi-agent reinforcement learning (DMARL). This involves the integration of three core aspects: (1) modular robotic material eco-systems (2) cyber-physical simulation and control with bidirectional feedback (3) adaptive intelligence through deep multi-agent reinforcement learning. The framework is implemented through three comparable case studies for collaborative modular robotic assembly of reconfigurable building parts.

DGN: influence maximization based on deep reinforcement learning

DGN: influence maximization based on deep reinforcement learning

An efficient deep reinforcement learning based task scheduler in cloud-fog environment

Efficient task scheduling in cloud and fog computing environments remains a significant challenge due to the diverse nature and critical processing requirements of tasks originating from heterogeneous devices. Traditional scheduling methods often struggle with high latency and inadequate processing times, especially in applications demanding strict computational efficiency. To address these challenges, this paper proposes an advanced fog-cloud integration approach utilizing a deep reinforcement learning-based task scheduler, DRLMOTS (Deep Reinforcement Learning based Multi Objective Task Scheduler in Cloud Fog Environment). This novel scheduler intelligently evaluates task characteristics, such as length and processing capacity, to dynamically allocate computation to either fog nodes or cloud resources. The methodology leverages a Deep Q-Learning Network model and includes extensive simulations using both randomized workloads and real-world Google Jobs Workloads. Comparative analysis demonstrates that DRLMOTS significantly outperforms existing baseline algorithms such as CNN, LSTM, and GGCN, achieving a substantial reduction in makespan by up to 26.80%, 18.84, and 13.83% and decreasing energy consumption by up to 39.60%, 30.29%, and 27.11%. Additionally, the proposed scheduler enhances fault tolerance, showcasing improvements of up to 221.89%, 17.05%, and 11.05% over conventional methods. These results validate the efficiency and robustness of DRLMOTS in optimizing task scheduling in fog-cloud environments.

Joint Learning of Volume Scheduling and Order Placement Policies for Optimal Order Execution

Order execution is an extremely important problem in the financial domain, and recently, more and more researchers have tried to employ reinforcement learning (RL) techniques to solve this challenging problem. There are a lot of difficulties for conventional RL methods to tackle the order execution problem, such as the large action space including price and quantity, and the long-horizon property. As naturally order execution is composed of a low-frequency volume scheduling stage and a high-frequency order placement stage, most existing RL-based order execution methods treat these stages as two distinct tasks and offer a partial solution by addressing either one individually. However, the current literature fails to model the non-negligible mutual influence between these two tasks, leading to impractical order execution solutions. To address these limitations, we propose a novel automatic order execution approach based on the hierarchical RL framework (OEHRL), which jointly learns the policies for volume scheduling and order placement. OEHRL first extracts the state embeddings at both the macro and micro levels with a sequential variational auto-encoder model. Based on the effective embeddings, OEHRL generates a hindsight expert dataset, which is used to train a hierarchical order execution policy. In the hierarchical structure, the high-level policy is in charge of the target volume and the low-level learns to determine the prices for a series of the allocated sub-orders from the high level. These two levels collaborate seamlessly and contribute to the optimal order execution policy. Extensive experiment results on 200 stocks across the US and China A-share markets validate the effectiveness of the proposed approach.

Deephive: A Reinforcement Learning Approach for Automated Discovery of Swarm-Based Optimization Policies

We present an approach for designing swarm-based optimizers for the global optimization of expensive black-box functions. In the proposed approach, the problem of finding efficient optimizers is framed as a reinforcement learning problem, where the goal is to find optimization policies that require a few function evaluations to converge to the global optimum. The state of each particle within the swarm is defined as its current position and function value within a design space, and the particles learn to take favorable actions that maximize the reward, which is based on the final value of the objective function. The proposed approach is tested on 50 benchmark optimization functions and compared to the performance of other global optimization strategies. Furthermore, the generalization capabilities of the trained particles on the four categories of optimization benchmark functions are investigated. The results show superior performance compared to the other optimizers, desired scaling when the dimension of the functions is varied, and acceptable performance even when applied to unseen functions. On a broader scale, the results show promise for the rapid development of domain-specific optimizers.

Root Cause Attribution of Delivery Risks via Causal Discovery with Reinforcement Learning

Managing delivery risks is a critical challenge in modern supply chain management due to the increasing complexity and interdependencies of global supply networks. Existing methods often rely on correlation-based approaches, which fail to uncover the true causes behind delivery delays. This limitation makes it difficult for supply chain managers to identify actionable factors that can mitigate risks effectively. To address these challenges, we propose a novel method that integrates causal discovery with reinforcement learning to identify the root causes of delivery risks. Unlike traditional correlation-based methods, our approach uncovers both the direction and strength of causal relationships between variables, allowing for more accurate identification of the key drivers behind delivery delays. By applying causal strength quantification, we further measure the impact of each factor on delivery performance. Using real-world supply chain data, our results demonstrate that the proposed method reveals hidden causal relationships between factors such as shipping mode, order size, and delivery status. These insights enable supply chain managers to implement more targeted interventions, significantly improving risk mitigation strategies.

A Review of Deep Learning-Based Remote Sensing Image Caption: Methods, Models, Comparisons and Future Directions

Remote sensing images contain a wealth of Earth-observation information. Efficient extraction and application of hidden knowledge from these images will greatly promote the development of resource and environment monitoring, urban planning and other related fields. Remote sensing image caption (RSIC) involves obtaining textual descriptions from remote sensing images through accurately capturing and describing the semantic-level relationships between objects and attributes in the images. However, there is currently no comprehensive review summarizing the progress in RSIC based on deep learning. After defining the scope of the papers to be discussed and summarizing them all, the paper begins by providing a comprehensive review of the recent advancements in RSIC, covering six key aspects: encoder–decoder framework, attention mechanism, reinforcement learning, learning with auxiliary task, large visual language models and few-shot learning. Subsequently a brief explanation on the datasets and evaluation metrics for RSIC is given. Furthermore, we compare and analyze the results of the latest models and the pros and cons of different deep learning methods. Lastly, future directions of RSIC are suggested. The primary objective of this review is to offer researchers a more profound understanding of RSIC.

Path Tracking Control for Four-Wheel Independent Steering and Driving Vehicles Based on Improved Deep Reinforcement Learning

We propose a compound control framework to improve the path tracking accuracy of a four-wheel independent steering and driving (4WISD) vehicle in complex environments. The framework consists of a deep reinforcement learning (DRL)-based auxiliary controller and a dual-layer controller. Samples in the 4WISD vehicle control framework have the issues of skewness and sparsity, which makes it difficult for the DRL to converge. We propose a group intelligent experience replay (GER) mechanism that non-dominantly sorts the samples in the experience buffer, which facilitates within-group and between-group collaboration to achieve a balance between exploration and exploitation. To address the generalization problem in the complex nonlinear dynamics of 4WISD vehicles, we propose an actor-critic architecture based on the method of two-stream information bottleneck (TIB). The TIB method is used to remove redundant information and extract high-dimensional features from the samples, thereby reducing generalization errors. To alleviate the overfitting of DRL to known data caused by IB, the reverse information bottleneck (RIB) alters the optimization objective of IB, preserving the discriminative features that are highly correlated with actions and improving the generalization ability of DRL. The proposed method significantly improves the convergence and generalization capabilities of DRL, while effectively enhancing the path tracking accuracy of 4WISD vehicles in high-speed, large-curvature, and complex environments.

Message Action Adapter Framework in Multi-Agent Reinforcement Learning

Multi-agent reinforcement learning (MARL) has demonstrated significant potential in enabling cooperative agents. The communication protocol, which is responsible for message exchange between agents, is crucial in cooperation. However, communicative MARL systems still face challenges due to the noisy messages in complex multi-agent decision processes. This issue often stems from the entangled representation of observations and messages in policy networks. To address this, we propose the Message Action Adapter Framework (MAAF), which first trains individual agents without message inputs and then adapts a residual action based on message components. This separation isolates the effect of messages on action inference. We explore how training the MAAF framework with model-agnostic message types and varying optimization strategies influences adaptation performance. The experimental results indicate that MAAF achieves competitive performance across multiple baselines despite utilizing only half of the available communication, and shows an average improvement of 7.6% over the full attention-based communication approach. Additional findings suggest that different message types result in significant performance variations, emphasizing the importance of environment-specific message types. We demonstrate how the proposed architecture separates communication channels, effectively isolating message contributions.

Integrated Early Flood Prediction using Sentinel-2 Imagery with VANET-MARL-based Deep Neural RNN

<p style="line-height:200%;"><span style="font-family:"Times New Roman","serif";font-size:12.0pt;line-height:200%;" lang="EN-US">Floods are a major contributor to the destruction of infrastructure and the overall economy of afflicted countries, leading to loss of life and significant damage. Remote sensing, satellite imagery photography, global positioning system, and geographic information system (GIS) are commonly used to identify floods and analyze the associated damages. The research presented here integrates Sentinel-2 satellite images, VANET with Multi-Agent Reinforcement Learning (MARL), and a deep neural RNN for early flood prediction. Sentinel-2 imaging delivers extensive geographical and temporal data regarding land cover and water bodies, while VANET-MARL provides real-time ground-truth information and distributed decision-making capabilities. The Deep Neural RNN efficiently acquires intricate patterns from the combined data to forecast the likelihood, intensity, and scope of floods. The experimental findings clearly show that the suggested system outperforms standard methods in terms of accuracy, precision, recall, and lead time. VANET-MARL integration improves the system's ability to adapt and remain strong in changing circumstances. The results showed that the method was 94.8% accurate in early predicting floods.</span></p>

TopoSense: agent driven topological graph extraction from remote sensing image

Abstract. Automatic topological graph extraction is critical for intelligent remote sensing image interpretation and cartographic representation. However, existing approaches neither adopt segmentation-based post-processing nor directly predict the graph, thereby suffering from limited scalability and poor adaptability to complex spatial structures. To address these issues, we introduce TopoSense, an innovative framework for extracting topological graphs from remote sensing images through an agent-driven approach. By employing a novel combination of reinforcement learning and neural network architectures, TopoSense autonomously navigates through pixel-level data, efficiently constructing topological representations. It not only enhances the accuracy of spatial feature detection, but also significantly reduces processing time. Experiments on the TOP-BOUNDARY and REALSCENE demonstrate its superiority in capturing intricate spatial relationships compared to traditional methods.

Retraction Note: Analysis on quantum reinforcement learning algorithms for prediction of protein sequence

Retraction Note: Analysis on quantum reinforcement learning algorithms for prediction of protein sequence

Implementing an intelligent diagnosis and treatment system for in-hospital cardiac arrest in the Utstein style: a multi-center case study

BackgroundCardiac arrest presents a variety of causes and complexities, making it challenging to develop targeted treatment plans. Often, the original data are either inadequate or lack essential patient information. In this study, we introduce an intelligent system for diagnosing and treating in-hospital cardiac arrest (IHCA), aimed at improving the success rate of cardiopulmonary resuscitation and restoring spontaneous circulation.MethodsTo compensate for insufficient or incomplete data, a hybrid mega trend diffusion method was used to generate virtual samples, enhancing system performance. The core of the system is a modified episodic deep reinforcement learning module, which facilitates the diagnosis and treatment process while improving sample efficiency. Uncertainty analysis was performed using Monte Carlo simulations, and dependencies between different parameters were assessed using regular vine copula. The system's effectiveness was evaluated using ten years of data from Utstein-style IHCA registries across seven hospitals in China's Hebei Province.ResultsThe system demonstrated improved performance compared to other models, particularly in scenarios with inadequate data or missing patient information. The average reward scores in two key stages increased by 2.3–9 and 9.9–23, respectively.ConclusionsThe intelligent diagnosis and treatment effectively addresses IHCA, providing reliable diagnosis and treatment plans in IHCA scenarios. Moreover, it can effectively induce cardiopulmonary resuscitation and restoration of spontaneous circulation processes even when original data are insufficient or basic patient information is missing.

An optimized task offloading strategy based on deep reinforcement learning combined with channel reliability prediction

An optimized task offloading strategy based on deep reinforcement learning combined with channel reliability prediction

Integrated Architecture for Smart Grid Energy Management: Deep Attention-Enhanced Sequence-to-Sequence Model with Energy-Aware Optimized Reinforcement Learning for Demand Response

Integrated Architecture for Smart Grid Energy Management: Deep Attention-Enhanced Sequence-to-Sequence Model with Energy-Aware Optimized Reinforcement Learning for Demand Response