State-action Space Research Articles

Deep reinforcement learning for collision avoidance of autonomous ships in inland river

Due to the characteristics of large inertia, large time delay, and nonlinearity in ship motion, the maneuverability of ships is an important issue related to the safety of ship navigation. The manipulation of ships significantly affects the safety of ship navigation, and its importance will gradually increase with the development of autonomous ships. Collision accidents are common due to the complexity of inland waterways and density of the ships. Herein, an autonomous learning framework with deep reinforcement learning (DRL) is constructed for autonomous driving tasks. The state space, action space, reward function and neural network structure of DRL are designed based on the ship’s maneuvering characteristics and control requirements. A DDPG algorithm is then used to implement the controller. Finally, some representative route segments of the inland waterway is selected for simulation research based on the virtual simulation environment. The designed DRL controller can quickly converge from the training and learning process to meet the control requirements. By comparing with the previous experimental results, the effectiveness of this algorithm is verified. Therefore, this study provides a reference for future research on the collision avoidance technology of autonomous ships in inland rivers.

Joint channel allocation and transmit power control for underlay EH‐CRNs: A clustering‐based multi‐agent DDPG approach

AbstractTo address the concerns of energy supply and spectrum scarcity for wireless devices, energy harvesting cognitive radio networks have been proposed. To improve spectrum utilization, secondary users (SUs) access the licensed spectrum in underlay mode, which may cause severe interference to primary users and SUs. The focus is on the underlay energy harvesting cognitive radio networks with multiple pairs of SUs, and formulate the long‐term secondary throughput maximization problem as a mixed‐integer non‐linear programming problem. As traditional approaches could hardly solve the mixed‐integer non‐linear programming problem well, a centralized deep deterministic policy gradient (C‐DDPG) approach is proposed that achieves satisfactory throughput performance. To reduce the computational complexity of C‐DDPG, we further propose a clustering‐based multi‐agent DDPG (CMA‐DDPG) approach that combines the advantages of the centralized deep reinforcement learning approach and the distributed deep reinforcement learning approach. In the CMA‐DDPG, a novel interference‐based clustering algorithm is proposed, which partitions the SUs that cause severe mutual interference into one cluster, and the sizes of state space and action space are smaller than those in C‐DDPG. Numerical results validate the superiority of the proposed approaches in terms of the throughput and outage probability, and validate the clustering performance of the interference‐based clustering algorithm in terms of the outage probability of the secondary network.

A path planning method based on deep reinforcement learning for AUV in complex marine environment

The potential of autonomous underwater vehicle (AUV) on future applications is significant due to advancements in autonomy and intelligence. Path planning is a critical technology for AUV to perform operational missions in complex marine environments. To this end, this paper proposes a path planning method for AUV based on deep reinforcement learning. Initially, considering actual requirements, a complex marine environment model containing underwater terrain, sonobuoy detection, and ocean currents is established. Subsequently, the corresponding state space, action space, and reward function are formulated. Furthermore, to address the inherent limitations of existing deep reinforcement learning algorithms in terms of training efficiency, a mixed experience replay (MER) strategy is proposed. This strategy aims to enhance the efficiency of sample learning by integrating prior knowledge and exploration experience. Lastly, a novel HMER-SAC algorithm for AUV path planning is proposed by integrating the Soft Actor–Critic (SAC) algorithm with the hierarchical reinforcement learning strategy and the MER strategy. The results of the simulation and experiment demonstrate that the method is capable of efficiently planning executable paths in complex marine environments and exhibits superior training efficiency, stability, and performance.

Gain-Based Feedback for Shape Control of Antenna Reflectors Using Deep Q-Network Algorithm

An intelligent shape control method for antenna reflectors is proposed using gain feedback instead of displacement feedback, considering the limitations of high-precision deformation measurement on orbit. Moreover, the finite element method is used to establish an analysis model of a grid reflector with embedded piezoelectric actuators, and the antenna gain formula is derived. To solve the multidimensional control laws from a single gain dimension, an active shape control model training framework is proposed based on the deep Q-network algorithm, including the state space, action space, and reward function. Numerical simulations, with errors stemming from two typical thermal loads as the initial errors, are conducted to illustrate the effect of the proposed control method. The influence of the voltage step size on the shape control precision is analyzed, and further online training is discussed for implementing control. The results indicate that the proposed method reduces the root-mean-square (RMS) error by more than 50% The control effect of the trained control model on other error conditions does not perform as desired. However, online training can effectively solve this problem, achieving faster convergence and a significant reduction of the RMS error.

Real-time scheduling for two-stage assembly flowshop with dynamic job arrivals by deep reinforcement learning

Assembly flowshops differ from flowshops because their jobs have several components that are manufactured in the processing stage and then joined together in the assembly stage. Real-time scheduling of two-stage assembly flowshops is of great significance for many discrete manufacturing companies in customized order fulfilment. Therefore, we study two-stage assembly flowshop scheduling problem with the objective of total tardiness minimization considering dynamic job arrivals using deep reinforcement learning (DRL). First, an architecture of DRL-based real-time scheduling is proposed. Second, we establish a DRL-based real-time scheduling model by designing proper state space, action space and reward function. A scheduling agent is constructed and trained through the proximal policy optimization (PPO) using interactive simulation data. The experimental results show that the trained scheduling agent can learn to choose the appropriate dispatching rules efficiently so as to make a real-time scheduling quickly and effectively. The PPO algorithm outperforms six dispatching rules, genetic algorithm and two DRL algorithms (i.e., DDQN and DDPG) in terms of solution quality and computing speed. The proposed approach is highly applicable to many practical scheduling situations that requires real-time and quick decisions.

A Deep Reinforcement Learning Approach to Injection Speed Control in Injection Molding Machines with Servomotor-Driven Constant Pump Hydraulic System

The control of the injection speed in hydraulic injection molding machines is critical to product quality and production efficiency. This paper analyzes servomotor-driven constant pump hydraulic systems in injection molding machines to achieve optimal tracking control of the injection speed. We propose an efficient reinforcement learning (RL)-based approach to achieve fast tracking control of the injection speed within predefined time constraints. First, we construct a precise Markov decision process model that defines the state space, action space, and reward function. Then, we establish a tracking strategy using the Deep Deterministic Policy Gradient RL method, which allows the controller to learn optimal policies by interacting with the environment. Careful attention is also paid to the network architecture and the definition of states/actions to ensure the effectiveness of the proposed method. Extensive numerical results validate the proposed approach and demonstrate accurate and efficient tracking of the injection velocity. The controller’s ability to learn and adapt in real time provides a significant advantage over the traditional Proportion Integration Differentiation controller. The proposed method provides a practical solution to the challenge of maintaining accurate control of the injection speed in the manufacturing process.

Adaptive decision-making with deep Q-network for heterogeneous unmanned aerial vehicle swarms in dynamic environments

With the continuous advancement of Unmanned Aerial Vehicle (UAV) swarm technology, the field of adaptive decision-making for heterogeneous UAV swarms across a spectrum of tasks is rapidly emerging as a focal point of research. This domain holds significant importance for enhancing the operational efficiency and responsiveness of UAV swarms in complex and dynamic environments. This paper addresses this challenge by introducing an innovative adaptive task decision-making method, which leverages Deep Q-Network (DQN) algorithms, specifically tailored to meet the unique requirements of heterogeneous UAV swarms. We initially construct a mathematical model that transforms the intricate swarm decision-making issues into a standard Markov Decision Process (MDP), and meticulously design the state space, action space, reward function, and state transition function to cater to the decision-making needs of UAV swarms. Utilizing this model, we further integrate a hybrid experience replay mechanism, coupled with DQN algorithms for offline training, enabling the efficient transformation of training outcomes into a network model that is directly applicable to real-time online decision-making. This approach markedly enhances the decision-making adaptability and dynamic adaptability of UAV swarms when confronted with a variety of tasks. During the offline training phase, we also employ a hybrid experience replay mechanism and target network approximation techniques to bolster the network's stability and training efficiency, ensuring high reliability in online decision-making. Comparative simulations with existing DQN algorithms reveal the distinct superiority of our method in addressing adaptive task decision-making challenges. Moreover, through 1000 comprehensive simulation experiments, we further validate the stability and effectiveness of our method across diverse task environments, providing a solid theoretical foundation and technical support for the practical application and proliferation of UAV swarm technology.

Model predictive complex system control from observational and interventional data

Complex systems, characterized by intricate interactions among numerous entities, give rise to emergent behaviors whose data-driven modeling and control are of utmost significance, especially when there is abundant observational data but the intervention cost is high. Traditional methods rely on precise dynamical models or require extensive intervention data, often falling short in real-world applications. To bridge this gap, we consider a specific setting of the complex systems control problem: how to control complex systems through a few online interactions on some intervenable nodes when abundant observational data from natural evolution is available. We introduce a two-stage model predictive complex system control framework, comprising an offline pre-training phase that leverages rich observational data to capture spontaneous evolutionary dynamics and an online fine-tuning phase that uses a variant of model predictive control to implement intervention actions. To address the high-dimensional nature of the state-action space in complex systems, we propose a novel approach employing action-extended graph neural networks to model the Markov decision process of complex systems and design a hierarchical action space for learning intervention actions. This approach performs well in three complex system control environments: Boids, Kuramoto, and Susceptible-Infectious-Susceptible (SIS) metapopulation. It offers accelerated convergence, robust generalization, and reduced intervention costs compared to the baseline algorithm. This work provides valuable insights into controlling complex systems with high-dimensional state-action spaces and limited intervention data, presenting promising applications for real-world challenges.

Demonstration-enhanced policy search for space multi-arm robot collaborative skill learning

The increasing complexity of on-orbit tasks imposes great demands on the flexible operation of space robotic arms, prompting the development of space robots from single-arm manipulation to multi-arm collaboration. In this paper, a combined approach of Learning from Demonstration (LfD) and Reinforcement Learning (RL) is proposed for space multi-arm collaborative skill learning. The combination effectively resolves the trade-off between learning efficiency and feasible solution in LfD, as well as the time-consuming pursuit of the optimal solution in RL. With the prior knowledge of LfD, space robotic arms can achieve efficient guided learning in high-dimensional state-action space. Specifically, an LfD approach with Probabilistic Movement Primitives (ProMP) is firstly utilized to encode and reproduce the demonstration actions, generating a distribution as the initialization of policy. Then in the RL stage, a Relative Entropy Policy Search (REPS) algorithm modified in continuous state-action space is employed for further policy improvement. More importantly, the learned behaviors can maintain and reflect the characteristics of demonstrations. In addition, a series of supplementary policy search mechanisms are designed to accelerate the exploration process. The effectiveness of the proposed method has been verified both theoretically and experimentally. Moreover, comparisons with state-of-the-art methods have confirmed the outperformance of the approach.

Method for Collaborative Layout Optimization of Ship Equipment and Pipe Based on Improved Multi-Agent Reinforcement Learning and Artificial Fish Swarm Algorithm

The engine room is the core area of a ship, critical to its operation, safety, and efficiency. Currently, many researchers merely address the ship engine room layout design (SERLD) problem using optimization algorithms and independent layout strategies. However, the engine room environment is complex, involving two significantly different challenges: equipment layout and pipe layout. Traditional methods fail to achieve optimal collaborative layout objectives. To address this research gap, this paper proposes a collaborative layout method that combines improved reinforcement learning and heuristic algorithms. For equipment layout, the engine room space is first discretized into a grid, and a Markov decision process (MDP) framework suitable for equipment layout is proposed, including state space, action space, and reward mechanisms suitable for equipment layout. An improved adaptive guided multi-agent Q-learning (AGMAQL) algorithm is employed to train the layout model in a centralized manner, with enhancements made to the agent’s exploration state, exploration action, and learning strategy. For pipe layout, this paper proposes an improved adaptive trajectory artificial fish swarm algorithm (ATAFSA). This algorithm incorporates a hybrid encoding method, adaptive strategy, scouting strategy, and parallel optimization strategy, resulting in enhanced stability, accuracy, and problem adaptability. Subsequently, by comprehensively considering layout objectives and engine room attributes, a collaborative layout method incorporating hierarchical and adaptive weight strategies is proposed. This method optimizes in phases according to the layout objectives and priorities of different stages, achieving multi-level optimal layouts and providing designers with various reference schemes with different focuses. Finally, based on a typical real-world engine room engineering case, various leading algorithms and strategies are tested and compared. The results show that the proposed AGMAQL-ATAFSA (AGMAQL-ATA) exhibits robustness, efficiency, and engineering practicality. Compared to previous research methods and algorithms, the final layout quality improved overall: equipment layout effectiveness increased by over 4.0%, pipe optimization efficiency improved by over 40.4%, and collaborative layout effectiveness enhanced by over 2.2%.

Adaptive meta-reinforcement learning for AUVs 3D guidance and control under unknown ocean currents

Given the challenges posed by underwater communication constraints, Autonomous Underwater Vehicles (AUVs) must operate with enhanced autonomy while balancing operational safety and efficiency within complex marine settings. AUVs face a 3D guidance and control challenge, requiring complex adjustments to unknown ocean currents that affect their stability, maneuverability, and Guidance-Navigation-Control system efficiency. This study introduces and implements an off-policy meta-reinforcement learning framework to address AUVs’ 3D guidance and control challenges under unknown ocean currents. Our method, APE-SAC, utilizes an inference network to identify latent variables, streamlining the adaptation to unknown ocean currents by optimizing their strength and direction. Consequently, this framework redefines the issue of guidance and control under unknown current disturbances into a manageable learning task across multiple predefined scenarios. Utilizing the 3D dynamics of the REMUS AUV, we construct meta-Markov Decision Processes to detail the comprehensive state space, action space, and robust reward function tailored to the task’s specific demands. Furthermore, we develop a highly realistic simulation environment specifically designed for comprehensive experiments. Extensive results demonstrate that, compared to deep reinforcement learning baselines, APE-SAC exhibits outstanding performance in guiding and controlling AUVs under unknown ocean currents.

A frequency cooperative control strategy for multimicrogrids with EVs based on improved evolutionary-deep reinforcement learning

The construction of microgrid has fully promoted the large-scale access of distributed power and the rapid development of electric vehicles (EVs). And microgrid system faces operational issues such as strong random disturbances from distributed power sources and loads, as well as unexpected events during operation. These issues can lead to unstable microgrid frequency, excessive discharge of EVs, and increased control costs. To solve the issues, a frequency cooperative control strategy for multimicrogrids with EVs based on improved evolutionary-deep reinforcement learning (EDRL) is proposed in this paper. First, consider the effect of the vehicle-to-grid (V2G) process on the minimum time required to fully charge an EV, and the impact of the output distribution of micro turbines (MTs) on the regulation cost, and based on the coupling relationship between generator terminal voltage regulation and system frequency control, a multimicrogrid comprehensive control model is constructed. Second, in order to deal with engineering tasks with deceptive rewards and sparse rewards such as integrated frequency control, evolutionary algorithms and deep reinforcement learning algorithms are combined to effectively assist the training process to jump out of local optimal solutions and approach optimal control strategies. Meanwhile, the algorithm is improved based on the novelty search and intelligent partition strategy, so that the strategy has better convergence characteristics and can reduce the cost of information transmission and computational complexity under the premise of ensuring the control effect. Furthermore, the state space, action space and reward function of the controller are defined. Finally, the simulation results show that the proposed controller has the ability of coordinated control, can effectively reduce the adjustment cost of MT unit and the unnecessary discharging of EVs while ensuring the frequency regulation requirements of each submicrogrid, which is far superior to PID control, fuzzy control, and traditional deep reinforcement learning control.

Continuous control of structural vibrations using hybrid deep reinforcement learning policy

An efficient active structural control scheme by utilizing the collective concepts of data-driven and physics-inspired reinforcement learning (RL) approaches is being introduced here. The controller is designed to deliver optimal feedback forces based on the full-state information, wherein the training process involves the use of deep neural networks (NNs) and a specially designed gradient descent-based sequence within the RL framework. This integration of algorithms results in a unique active control policy that accelerates the learning process, thereby demands considerably less computational resources to develop an optimal and stable controller as compared to existing data-driven approaches. Most importantly, the mentioned data-driven and physics-inspired approaches encompass deep deterministic policy gradient (DDPG) and an iterative gradient-based state feedback control (SFSC) algorithms, respectively, to establish the architecture of the hybrid control policy, referred to as hybrid RL-controller. Additional advantage of the hybrid RL-controller is its ability to operate in continuous state–action spaces, which allows the designed controller to address diverse structural control problems. The staggered performance of the hybrid RL-controller, both in continuous and discrete time, is evaluated in three case studies that involve structures in linear and nonlinear regimes. The outcomes include a detailed comparison of the hybrid RL-controller with the individually designed DDPG and SFSC RL control strategies, as well as the uncontrolled scenario. Furthermore, this study thoroughly investigates the real-life implementation concerns of control strategies, such as perturbations in model parameters and input forces, and time delays in the feedback loop. Finally, the results from this study corroborate that the designed RL-controller showcases superior performance and faster execution time in the feedback loop, making it suitable for the vibration control of multidimensional complex structures.

Dynamic resource allocation for virtual machine migration optimization using machine learning

This article delves into the importance of applying machine learning and deep reinforcement learning techniques in cloud resource management and virtual machine migration optimization, highlighting the role of these advanced technologies in dealing with the dynamic changes and complexities of cloud computing environments. Through environment modeling, policy learning, and adaptive enhancement, machine learning methods, especially deep reinforcement learning, provide effective solutions for dynamic resource allocation and virtual intelligence migration. These technologies can help cloud service providers improve resource utilization, reduce energy consumption, and improve service reliability and performance. Effective strategies include simplifying state space and action space, reward shaping, model lightweight and acceleration, and accelerating the learning process through transfer learning and meta-learning techniques. With the continuous progress of machine learning and deep reinforcement learning technologies, combined with the rapid development of cloud computing technology, it is expected that the application of these technologies in cloud resource management and virtual machine migration optimization will be more extensive and in-depth. Researchers will continue to explore more efficient algorithms and models to further improve the accuracy and efficiency of decision making. In addition, with the integration of edge computing, Internet of Things and other technologies, cloud computing resource management will face more new challenges and opportunities, and the application scope and depth of machine learning and deep reinforcement learning technology will also expand, opening new possibilities for building a more intelligent, efficient and reliable cloud computing service system.

Defense penetration strategy for unmanned surface vehicle based on modified soft actor–critic

This paper investigates the defense penetration problem for unmanned surface vehicle (USV) with limited detection range under unknown defenders. A novel end-to-end defense penetration strategy is proposed for the USV based on modified soft actor–critic (MSAC). Specifically, the state–action space of the attacker is designed by combining the maneuvering model of USV and detection information. To guide the USV to reach the target area under the interception of defenders, the penetration reward is constructed. Then, the long short-term memory network (LSTM) is applied in the actor–critic network design. A dynamic training mechanism is proposed by combining curriculum learning and multi-memory pools, which aims to improve the success rate of defense penetration. Finally, simulation results validate the effectiveness and superiority of the proposed defense penetration method for the USV suffering from unknown defenders. The higher success rate can be guaranteed by using MSAC compared to the previous methods.

Advancements in Reinforcement Learning Algorithms for Autonomous Systems

Reinforcement learning, often known as RL, has developed as a strong paradigm to teach autonomous software agents to make choices in contexts that are both complicated and dynamic. This abstract investigates recent developments and uses of RL in a variety of fields, showing both its transformational potential and the constraints that it faces at present. Recent developments in reinforcement learning (RL) algorithms, in particular deep reinforcement learning (DRL), have made it possible to make major advancements in autonomous decision- making tasks. DRL algorithms can learn complicated representations of state-action spaces by using deep neural networks. This allows for more efficient exploration and exploitation methods to be implemented. Additionally, advancements in algorithmic enhancements, such as prioritized experience replay and distributional reinforcement learning, have improved the stability and sample efficiency of reinforcement learning algorithms, which has made it possible for these algorithms to be used in real-world applications. Robotics, autonomous cars, game playing, finance, and healthcare are just a few of the many fields that may benefit from the use of RL. In the field of robotics, reinforcement learning (RL) makes it possible for autonomous agents to learn how to navigate, manipulate, and move about in environments that are both complicated and unstructured. To improve both safety and efficiency on the road, autonomous cars make use of reinforcement learning (RL) to make decisions in dynamic traffic situations. In finance, RL algorithms are used for portfolio optimization, algorithmic trading, and risk management. These applications serve to improve investment techniques and decision-making procedures. Furthermore, in the field of healthcare, RL supports individualized treatment planning, clinical decision support, and medical image analysis, which enables physicians to provide patients with care that is specifically suited to their needs. Despite the promising improvements and applications, RL is still confronted with several difficulties that restrict its capacity to be widely adopted and scaled. Among these problems are the inefficiency of the sample, the trade-offs between exploration and exploitation, concerns about safety and dependability, and the need for explainability and interpretability in decision-making processes. To effectively address these difficulties, it is necessary to engage in collaborative efforts across several disciplines, conduct research on algorithmic developments, and establish extensive assessment frameworks (Anon, 2022).

A Transfer Approach Using Graph Neural Networks in Deep Reinforcement Learning

Transfer learning (TL) has shown great potential to improve Reinforcement Learning (RL) efficiency by leveraging prior knowledge in new tasks. However, much of the existing TL research focuses on transferring knowledge between tasks that share the same state-action spaces. Further, transfer from multiple source tasks that have different state-action spaces is more challenging and needs to be solved urgently to improve the generalization and practicality of the method in real-world scenarios. This paper proposes TURRET (Transfer Using gRaph neuRal nETworks), to utilize the generalization capabilities of Graph Neural Networks (GNNs) to facilitate efficient and effective multi-source policy transfer learning in the state-action mismatch setting. TURRET learns a semantic representation by accounting for the intrinsic property of the agent through GNNs, which leads to a unified state embedding space for all tasks. As a result, TURRET achieves more efficient transfer with strong generalization ability between different tasks and can be easily combined with existing Deep RL algorithms. Experimental results show that TURRET significantly outperforms other TL methods on multiple continuous action control tasks, successfully transferring across robots with different state-action spaces.

PA2D-MORL: Pareto Ascent Directional Decomposition Based Multi-Objective Reinforcement Learning

Multi-objective reinforcement learning (MORL) provides an effective solution for decision-making problems involving conflicting objectives. However, achieving high-quality approximations to the Pareto policy set remains challenging, especially in complex tasks with continuous or high-dimensional state-action space. In this paper, we propose the Pareto Ascent Directional Decomposition based Multi-Objective Reinforcement Learning (PA2D-MORL) method, which constructs an efficient scheme for multi-objective problem decomposition and policy improvement, leading to a superior approximation of Pareto policy set. The proposed method leverages Pareto ascent direction to select the scalarization weights and computes the multi-objective policy gradient, which determines the policy optimization direction and ensures joint improvement on all objectives. Meanwhile, multiple policies are selectively optimized under an evolutionary framework to approximate the Pareto frontier from different directions. Additionally, a Pareto adaptive fine-tuning approach is applied to enhance the density and spread of the Pareto frontier approximation. Experiments on various multi-objective robot control tasks show that the proposed method clearly outperforms the current state-of-the-art algorithm in terms of both quality and stability of the outcomes.

A novel approach for self-driving car in partially observable environment using life long reinforcement learning

Despite ground-breaking advancements in robotics, gaming, and other challenging domains, reinforcement learning still faces significant challenges in solving dynamic, open-world problems. Since reinforcement learning algorithms usually perform poorly when exposed to new tasks outside of their data distribution, continuous learning algorithms have drawn significant attention. In parallel with work on lifelong learning algorithms, there is a need for challenging environments, properly planned trials, and metrics to measure research success. In this context, a Deep Asynchronous Autonomous Learning System (DAALS) is proposed in this paper for training a self-driving car in a partially observable environment for real-world scenarios in a continuous state-action space. To cater to three different use cases, three different algorithms were used. To train their agents for learning and upgrading discrete state policies, DAALS used the Asynchronous Advantage Stager Reviewer (AASR) algorithm. To train its agent for continuous state spaces, DAALS also uses an Extensive Deterministic Policy Gradient (EDPG) algorithm. To train the agent in a lifelong form of learning for partially observable environments, DAALS uses a Deep Deterministic Policy Gradient Novel Lifelong Learning Algorithm (DDPGNLLA). The system offers flexibility to the user to train the agents for both discrete and continuous state-action spaces. Compared to previous models in continuous state-action spaces, Deep deterministic policy gradient lifelong learning algorithm outperforms previous models by 46.09%. Furthermore, the Deep Asynchronous Autonomous System tends to outperform all previous reinforcement learning algorithms, making our proposed approach a real-world solution. As DAALS has tested on number of different environments it provides the insights on how modern Artificial Intelligence (AI) solutions can be generalized making it one of the better solutions for AI general domain problems.

Traffic pattern-aware elevator dispatching via deep reinforcement learning

This study addresses the elevator dispatching problem using deep reinforcement learning, with a specific emphasis on traffic pattern awareness. Previous studies on reinforcement learning-based elevator dispatching have largely focused on training separate models for single traffic patterns, such as up-peak, down-peak, lunch-peak, and inter-floor. This separate training approach not only introduces practical complexities by requiring an auxiliary model to predict traffic patterns for guiding dispatching decisions but is also computationally burdensome. In contrast, our goal is to develop a unified, traffic pattern-aware dispatching model. We formulate the elevator dispatching problem as a Semi-Markov Decision Process (SMDP) with novel state representation, action space, and reward function designs. To solve the formulated SMDP, we propose a Dueling Double Deep Q-Network (D3QN) architecture associated with the training algorithm. To ensure traffic pattern awareness, we train our model in a unified ‘All in One’ traffic scenario, employing two practical techniques to enhance the training process: (1) temporal grouping with gradient surgery and (2) incorporation of passenger arrival information. Empirical evaluations confirm the superiority of our model over multiple benchmarks, including those relying on separate, pattern-specific models. Remarkably, our unified model demonstrates robust performance across unseen traffic scenarios and performs exceptionally well in single traffic patterns despite being trained solely on the unified ‘All in One’ scenario. The short inference time for decision-making further solidifies the model’s practical viability. Additionally, the incremental benefits contributed by each of our introduced techniques are also investigated. Our code is available at https://github.com/jswan95/RL-based-traffic-pattern-aware-elevator-dispatching