Deep Reinforcement Learning Agent Research Articles

Task-driven Risk-bounded Hierarchical Reinforcement Learning Based on Iterative Refinement

Deep Reinforcement Learning (DRL) has garnered substantial acclaim for its versatility and widespread applications across diverse domains. Aligned with human-like learning, DRL is grounded in the fundamental principle of learning from interaction, wherein agents dynamically adjust behavior based on environmental feedback in the form of rewards. This iterative trial-and-error process, mirroring human learning, underscores the importance of observation, experimentation, and feedback in shaping understanding and behavior. DRL agents, trained to navigate complex surroundings, refine their knowledge through hierarchical and abstract representations, empowered by deep neural networks. These representations enable efficient handling of long-horizon tasks and flexible adaptation to novel situations, akin to the human ability to construct mental models for comprehending complex concepts and predicting outcomes. Hence, abstract representation building emerges as a critical aspect in the learning processes of both artificial agents and human learners, particularly in long-horizon tasks. Furthermore, human decision-making, deeply rooted in evolutionary history, exhibits a remarkable capacity to balance the tradeoff between risk and cost across various domains. This cognitive process involves assessing potential negative consequences, evaluating factors such as the likelihood of adverse outcomes, severity of potential harm, and overall uncertainty. Humans intuitively gauge inherent risks and adeptly weigh associated costs, extending beyond monetary expenses to include time, effort, and opportunity costs. The nuanced ability of humans to consider the tradeoff between risk and cost highlights the complexity and adaptability of human decision-making, a skill lacking in typical DRL agents. Principles like these derived from human-like learning present an avenue for inspiring advancements in DRL, fostering the development of more adaptive and intelligent artificial agents. Motivated by these observations and focusing on practical challenges in robotics, our efforts target risk-aware stochastic sequential decision-making problem which is crucial for tasks with extended time frames and varied strategies. A novel integration of model-based conditional planning with DRL is proposed, inspired by hierarchical techniques. This approach breaks down complex tasks into manageable subtasks(motion primitives), ensuring safety constraints and informed decision-making. Unlike existing methods, our approach addresses motion primitive improvement iteratively, employing diverse prioritization functions to guide the search process effectively. This risk-bounded planning algorithm seamlessly integrates conditional planning and motion primitive learning, prioritizing computational efforts for enhanced efficiency within specified time limits.

Deep-Reinforcement-Learning-Based Collision Avoidance of Autonomous Driving System for Vulnerable Road User Safety

The application of autonomous driving system (ADS) technology can significantly reduce potential accidents involving vulnerable road users (VRUs) due to driver error. This paper proposes a novel hierarchical deep reinforcement learning (DRL) framework for high-performance collision avoidance, which enables the automated driving agent to perform collision avoidance maneuvers while maintaining appropriate speeds and acceptable social distancing. The novelty of the DRL method proposed here is its ability to accommodate dynamic obstacle avoidance, which is necessary as pedestrians are moving dynamically in their interactions with nearby ADSs. This is an improvement over existing DRL frameworks that have only been developed and demonstrated for stationary obstacle avoidance problems. The hybrid A* path searching algorithm is first applied to calculate a pre-defined path marked by waypoints, and a low-level path-following controller is used under cases where no VRUs are detected. Upon detection of any VRUs, however, a high-level DRL collision avoidance controller is activated to prompt the vehicle to either decelerate or change its trajectory to prevent potential collisions. The CARLA simulator is used to train the proposed DRL collision avoidance controller, and virtual raw sensor data are utilized to enhance the realism of the simulations. The model-in-the-loop (MIL) methodology is utilized to assess the efficacy of the proposed DRL ADS routine. In comparison to the traditional DRL end-to-end approach, which combines high-level decision making with low-level control, the proposed hierarchical DRL agents demonstrate superior performance.

Deep-reinforcement-learning-based RMSCA for space division multiplexing networks with multi-core fibers [Invited Tutorial

The escalating demands for network capacities catalyze the adoption of space division multiplexing (SDM) technologies. With continuous advances in multi-core fiber (MCF) fabrication, MCF-based SDM networks are positioned as a viable and promising solution to achieve higher transmission capacities in multi-dimensional optical networks. However, with the extensive network resources offered by MCF-based SDM networks comes the challenge of traditional routing, modulation, spectrum, and core allocation (RMSCA) methods to achieve appropriate performance. This paper proposes an RMSCA approach based on deep reinforcement learning (DRL) for MCF-based elastic optical networks (MCF-EONs). Within the solution, a novel state representation with essential network information and a fragmentation-aware reward function were designed to direct the agent in learning effective RMSCA policies. Additionally, we adopted a proximal policy optimization algorithm featuring an action mask to enhance the sampling efficiency of the DRL agent and speed up the training process. The performance of the proposed algorithm was evaluated with two different network topologies with varying traffic loads and fibers with different numbers of cores. The results confirmed that the proposed algorithm outperforms the heuristics and the state-of-the-art DRL-based RMSCA algorithm in reducing the service blocking probability by around 83% and 51%, respectively. Moreover, the proposed algorithm can be applied to networks with and without core switching capability and has an inference complexity compatible with real-world deployment requirements.

Online energy management strategy for ammonia-hydrogen hybrid electric vehicles harnessing deep reinforcement learning

Powertrain electrification and fuel decarbonization play pivotal roles in the pursuit of carbon peak and carbon neutrality within the realm of transportation. Ammonia and hydrogen are essential carbon-neutral fuels suited for deployment in heavy-duty, long-haul transportation applications. The technological complementarity of ammonia and hydrogen can be realized in the current context by combining various energy conversion devices. Energy management strategy (EMS) is one of the critical technologies in hybrid systems. In this study, a novel EMS based on deep reinforcement learning (DRL) is proposed for a heavy-duty automotive hybrid system containing an ammonia-hydrogen internal combustion engine (ICE), a fuel cell system (FCS), an ammonia electrolysis cell (AEC), and Li-ion batteries (LB). To address the protracted training times and convergence challenges intrinsic to DRL, a fuzzy logic control (FLC) is introduced to provide expert demonstrations during the DRL agent's learning process. The incorporation of FLC not only facilitates the convergence speed of the DRL algorithm, but also strikes a good balance between the operation efficiency improvement and battery SOC maintenance. The simulation results show that the proposed EMS improves average efficiency by 2% with a modest 0.75% SOC reduction across four operational scenarios compared to the FLC-based EMS. After thorough validation of the proposed EMS, a parametric study is conducted to examine three key parameters: charge mode threshold, FCS power change rate limit, and hybrid mode threshold. The study aims to offer valuable insights for formulating the EMS of the ammonia-hydrogen hybrid system.

Low Earth Orbit Satellite Network Routing Algorithm Based on Graph Neural Networks and Deep Q-Network

Low Earth orbit (LEO) satellite networks are characterized by rapid topological changes, numerous network nodes and varying states of node resource constraints, which have resulted in traditional routing algorithms no longer being suitable for LEO satellite network routing. Therefore, this paper proposes an inductive learning architecture based on Graph Sample and Aggregate (GraphSAGE), which can significantly reduce the number of topology nodes to be trained, thereby reducing the computational complexity of the nodes. Then deep reinforcement learning (DRL) is employed for the continuous learning optimization of routing algorithms, and its generalization is improved by selecting GraphSAGE to construct the DRL agent. In the proposed graph neural-network-based routing optimization algorithm for LEO satellite networks, each Deep Q-Network (DQN) agent independently generates the hidden states of the nodes through the GraphSAGE model and uses them as inputs to the DRL model to make routing decisions. After a simulation and comparison, the proposed algorithm not only improves the overall network throughput, but also reduces the average end-to-end delay. The average throughput of the proposed algorithm increases by 29.47% and 18.42% compared to that of Dijkstra and the DQN, respectively. The average end-to-end delay is reduced by 39.76% and 15.29%, respectively, and can also adapt to changing topologies.

Deep reinforcement learning for PID parameter tuning in greenhouse HVAC system energy Optimization: A TRNSYS-Python cosimulation approach

The control of indoor temperature in greenhouses is crucial as it directly impacts the crop's thermal comfort and the performance of heating, ventilation, and air-conditioning (HVAC) systems. Conventional feedback controllers, like on/off, can sometimes make HVAC system work at full capacity when only half that capacity is needed. In contrast, the proportional-integral-derivative (PID) controller, provides precise control based on its P, I, and D parameters. However, it lacks a formal design procedure for optimizing a specified objective function. Previous studies have utilized conventional PID tuning approaches to track room setpoint temperature for residential buildings, data centers, and office buildings, with limited research in greenhouse applications. To address this gap, this study proposes a flexible PID controller that employs a deep reinforcement learning (DRL) algorithm to optimize its parameters, by tracking the setpoints and energy consumption of a greenhouse planted with tomatoes. This approach is different from the typical method of using the trained RL agent directly in HVAC controls. Through a self-made TRNSYS-Python cosimulation framework, the DRL agent interacts directly and in real time with the greenhouse and its plants. Consequently, optimized PID parameters were established and tested in the simulated environment. The resulting performance, in terms of both energy consumption and its ability to maintain the crop’s comfort temperature, was compared with the simulated on/off and manually tuned PID controllers. Compared to the on/off baseline control, the proposed PID optimized parameters reduce energy use by 8.81% to 12.99% and the manually tuned PID parameters with the Ziegler-Nichols tuning method reduce energy use by 7.17 %. Additionally, the proposed method had a deviation of 2.07% to 3.13%, while the manually tuned PID controller and the on/off controller had deviations of 7.27% and 3.27%, respectively, from the minimum comfortable temperature. This study serves as a framework for improving the energy efficiency of greenhouse HVAC system operations.

An improved energy management strategy for hybrid electric powered aircraft based on deep reinforcement learning

Electric airplanes have attracted a great deal of attentions because of their strategic commercial values and low emissions. One of the essentials for efficient control and management of hybrid electric propulsion aircraft is a well-designed energy management strategy (EMS). A new EMS approach based on deep learning is introduced in this study to realize thrust distribution and power distribution, consequently lowering aircraft fuel consumption. A model of hybrid electric propulsion power system including turbine engines and lithium-ion battery pack is constructed to act as an application environment. An intelligent energy management strategy based on deep reinforcement learning (DRL) is then proposed in this paper. Two DRL agents based on twin delayed deep deterministic policy gradient (TD3) are combined in the hierarchical framework. The multi-agents are trained to handle the energy management problem in this paper. In a finite period of time, the training process is accomplished. After training, the simulation result of the test shows that this DRL-based EMS reduces the equivalent fuel consumption by 2.2% compared with that using the existing method, and preserves 99.9% of the battery state of health (SOH). The battery pack's state of charge (SOC) regresses precisely to the reference value at the end. Particularly, the proposed EMS is able to work properly for different flight profiles and various initial SOCs. The results of this paper provide preliminary supports for the feasibility and superiority of deep reinforcement learning applied to the energy management strategy of series hybrid electric aircraft.

Explain the Explainer: Interpreting Model-Agnostic Counterfactual Explanations of a Deep Reinforcement Learning Agent

Counterfactual examples (CFs) are one of the most popular methods for attaching post-hoc explanations to machine learning (ML) models. However, existing CF generation methods either exploit the internals of specific models or depend on each sample's neighborhood; thus, they are hard to generalize for complex models and inefficient for large datasets. This work aims to overcome these limitations and introduces RELAX, a model-agnostic algorithm to generate optimal counterfactual explanations. Specifically, we formulate the problem of crafting CFs as a sequential decision-making task. We then find the optimal CFs via deep reinforcement learning (DRL) with discretecontinuous hybrid action space. In addition, we develop a distillation algorithm to extract decision rules from the DRL agent's policy in the form of a decision tree to make the process of generating CFs itself interpretable. Extensive experiments conducted on six tabular datasets have shown that RELAX outperforms existing CF generation baselines, as it produces sparser counterfactuals, is more scalable to complex target models to explain, and generalizes to both classification and regression tasks. Finally, we show the ability of our method to provide actionable recommendations and distill interpretable policy explanations in two practical, real-world use cases.

Scheduling of twin automated stacking cranes based on Deep Reinforcement Learning

Effective scheduling of twin automated stacking cranes (ASCs) in automated storage yard is critical to maximize operational efficiency. While Deep Reinforcement Learning (DRL) is promising in solving NP-hard scheduling problems, twin ASCs scheduling is challenging due to its unique properties including sequence-dependent setup and potential ASC interferences. In this paper, we propose a novel DRL method to learn high-quality policy for scheduling twin ASCs. We propose a Markov Decision Process model that enables the DRL agent to learn to minimize makespan and possible interferences. Based on the problem characteristics, we design a self-attention based neural architecture to effectively capture the relationships between containers under certain block state. Experiments show that the agent with the proposed feature extraction network can learn high-quality policies from training instances. These learned policies can be employed to produce effective scheduling solutions within seconds. Compared to traditional scheduling methods, the learned policy performs best in most problem sizes, and the performance improvement amplifies as the scales increase. Moreover, the policies show remarkable generalization ability on unseen instances with different distributions or scales.

Simulation of autonomous resource allocation through deep reinforcement learning-based portfolio-project integration

Resource allocation has always been a critical challenge for construction project planning, and it affects the cost, duration, and quality of the projects. However, current methods mainly focus on a single project and lack integrated planning and optimization across a construction company's multiple projects. This paper describes a simulation of an Autonomous Resource Allocation (ARA) model using Deep Reinforcement Learning (DRL) agents and methods like Double Deep Q-Networks and combined experience replay to develop and test ARA algorithms based on data harvesting from the Internet of Things (IoT) devices. The results show that DRL can successfully perform ARA by capturing the complex interactions among resource allocation features, without needing retraining when situations change. It shows promising future possibilities for construction companies to improve resource utilization and project performance for larger and more complex construction projects.

Improve Robustness of Reinforcement Learning against Observation Perturbations via l∞ Lipschitz Policy Networks

Deep Reinforcement Learning (DRL) has achieved remarkable advances in sequential decision tasks. However, recent works have revealed that DRL agents are susceptible to slight perturbations in observations. This vulnerability raises concerns regarding the effectiveness and robustness of deploying such agents in real-world applications. In this work, we propose a novel robust reinforcement learning method called SortRL, which improves the robustness of DRL policies against observation perturbations from the perspective of the network architecture. We employ a novel architecture for the policy network that incorporates global $l_\infty$ Lipschitz continuity and provide a convenient method to enhance policy robustness based on the output margin. Besides, a training framework is designed for SortRL, which solves given tasks while maintaining robustness against $l_\infty$ bounded perturbations on the observations. Several experiments are conducted to evaluate the effectiveness of our method, including classic control tasks and video games. The results demonstrate that SortRL achieves state-of-the-art robustness performance against different perturbation strength.

Contextual Pre-planning on Reward Machine Abstractions for Enhanced Transfer in Deep Reinforcement Learning

Recent studies show that deep reinforcement learning (DRL) agents tend to overfit to the task on which they were trained and fail to adapt to minor environment changes. To expedite learning when transferring to unseen tasks, we propose a novel approach to representing the current task using reward machines (RMs), state machine abstractions that induce subtasks based on the current task’s rewards and dynamics. Our method provides agents with symbolic representations of optimal transitions from their current abstract state and rewards them for achieving these transitions. These representations are shared across tasks, allowing agents to exploit knowledge of previously encountered symbols and transitions, thus enhancing transfer. Empirical results show that our representations improve sample efficiency and few-shot transfer in a variety of domains.

I-Rebalance: Personalized Vehicle Repositioning for Supply Demand Balance

Ride-hailing platforms have been facing the challenge of balancing demand and supply. Existing vehicle reposition techniques often treat drivers as homogeneous agents and relocate them deterministically, assuming compliance with the reposition. In this paper, we consider a more realistic and driver-centric scenario where drivers have unique cruising preferences and can decide whether to take the recommendation or not on their own. We propose i-Rebalance, a personalized vehicle reposition technique with deep reinforcement learning (DRL). i-Rebalance estimates drivers' decisions on accepting reposition recommendations through an on-field user study involving 99 real drivers. To optimize supply-demand balance and enhance preference satisfaction simultaneously, i-Rebalance has a sequential reposition strategy with dual DRL agents: Grid Agent to determine the reposition order of idle vehicles, and Vehicle Agent to provide personalized recommendations to each vehicle in the pre-defined order. This sequential learning strategy facilitates more effective policy training within a smaller action space compared to traditional joint-action methods. Evaluation of real-world trajectory data shows that i-Rebalance improves driver acceptance rate by 38.07% and total driver income by 9.97%.

Testing of Deep Reinforcement Learning Agents with Surrogate Models

Deep Reinforcement Learning (DRL) has received a lot of attention from the research community in recent years. As the technology moves away from game playing to practical contexts, such as autonomous vehicles and robotics, it is crucial to evaluate the quality of DRL agents. In this article, we propose a search-based approach to test such agents. Our approach, implemented in a tool called Indago , trains a classifier on failure and non-failure environment (i.e., pass) configurations resulting from the DRL training process. The classifier is used at testing time as a surrogate model for the DRL agent execution in the environment, predicting the extent to which a given environment configuration induces a failure of the DRL agent under test. The failure prediction acts as a fitness function, guiding the generation towards failure environment configurations, while saving computation time by deferring the execution of the DRL agent in the environment to those configurations that are more likely to expose failures. Experimental results show that our search-based approach finds 50% more failures of the DRL agent than state-of-the-art techniques. Moreover, such failures are, on average, 78% more diverse; similarly, the behaviors of the DRL agent induced by failure configurations are 74% more diverse.

Dynamic scheduling of decentralized high-end equipment R&D projects via deep reinforcement learning

Distributed research and development (R&D) plays a pivotal role in high-end equipment manufacturing enterprises, significantly impacting interdisciplinary innovation and expediting development cycles. In the uncertain environment characterized by random arrivals of new projects, achieving dynamic scheduling and real-time coordination for decentralized R&D projects is a challenging problem. This paper studies a dynamic decentralized resource-constrained multi-project scheduling problem with product transfers (DDRCMPSP-PT). The objective is to minimize the development cycle after initial local schedules are generated to minimize the individual project makespan. To tackle the problem, we develop a decentralized multi-agent system using the dynamic coordination mechanism for resource assignment (DMAS/DCMRA). An up-to-date deep reinforcement learning (DRL) algorithm, dueling double deep Q-learning (D3QN) with prioritized replay, is employed to select the optimal strategy to resolve the global resource conflicts adaptively. Two priority rules are presented based on the structural properties and make up the action space of the DRL agent together with ten high-quality priority rules. Extensive simulation experiment results based on real enterprise cases reveal that the DMAS/DCMRA outperforms all traditional priority rule-based methods. The proposed priority rules also show competitive performance compared to other priority rules, which validates the applicability and superiority of the proposed method. This research offers a new agile project management technique for organizations running multiple R&D projects and managing crucial bottleneck resources.

Shovel allocation and scheduling for open-pit mining using deep reinforcement learning

ABSTRACT The open-pit production system is a highly dynamic and uncertain environment with complex interactions between haulage and loading equipment on a shared road network. One of the key decisions in open-pit short-term planning is the allocation sequence of shovels to mining faces to meet the production targets established by long- and medium-term strategic plans. Deep Reinforcement Learning(DRL) techniques are commonly used in dynamic production environments. In this approach, an agent is trained on a simulation of the production system to learn the optimal decisions based on the system’s current state. This paper proposes a DRL approach based on the Deep Q-Learning algorithm to obtain a robust shovel allocation plan for open-pit short-term planning. First, a discrete-event simulation of the mining production system incorporating trucks, shovels, crushers, waste dumps, and the road network is created. This simulation models the uncertainties of each component’s operating cycle based on historical activity records, and it is used to train the DRL agent. The goal is to learn a robust shovel allocation strategy for the next production quarter, 3 months, to meet the tonnes per hour (TPH) production target to be delivered to the crusher feeds by interacting with the production simulator. The framework is tested in an iron ore open-pit mine case study where the shovel allocation agent successfully learns a strategy that consistently delivers the desired production target.

Smart urban windcatcher: Conception of an AI-empowered wind-channeling system for real-time enhancement of urban wind environment

The dense configuration and rapid proliferation of high-rise buildings in central Hong Kong have led to increasing stagnation of pedestrian-level airflows, lowering the wind speed and exacerbating issues in wind and thermal comfort. Addressing these problems can be difficult without significant building renovations and urban re-planning. This paper introduces a novel framework for the real-time enhancement of the local urban wind environment using motion-controlled billboards, or smart urban windcatchers, managed by a deep reinforcement learning (DRL) agent. The DRL agent determines the windcatchers' optimal positions based on street-level sensor data, independent of meteorological data. The framework's effectiveness was assessed using a simplified city grid model, where two windcatchers aimed to optimize the pedestrian-level wind environment on a specific street. Simulation results indicated that the windcatchers could effectively alter the flow direction in the streets, promoting or diverting air passages per demand. The DRL agent also gave accurate instructions to the windcatchers under various weather conditions, achieving near-optimal wind environment scores. This paper introduces the conception and confirms the feasibility of the windcatcher system, laying the groundwork for future research—as it is much needed and welcomed—to enhance the system and overcome the acknowledged challenges in large-scale, real-world implementation.

An innovative heterogeneous transfer learning framework to enhance the scalability of deep reinforcement learning controllers in buildings with integrated energy systems

AbstractDeep Reinforcement Learning (DRL)-based control shows enhanced performance in the management of integrated energy systems when compared with Rule-Based Controllers (RBCs), but it still lacks scalability and generalisation due to the necessity of using tailored models for the training process. Transfer Learning (TL) is a potential solution to address this limitation. However, existing TL applications in building control have been mostly tested among buildings with similar features, not addressing the need to scale up advanced control in real-world scenarios with diverse energy systems. This paper assesses the performance of an online heterogeneous TL strategy, comparing it with RBC and offline and online DRL controllers in a simulation setup using EnergyPlus and Python. The study tests the transfer in both transductive and inductive settings of a DRL policy designed to manage a chiller coupled with a Thermal Energy Storage (TES). The control policy is pre-trained on a source building and transferred to various target buildings characterised by an integrated energy system including photovoltaic and battery energy storage systems, different building envelope features, occupancy schedule and boundary conditions (e.g., weather and price signal). The TL approach incorporates model slicing, imitation learning and fine-tuning to handle diverse state spaces and reward functions between source and target buildings. Results show that the proposed methodology leads to a reduction of 10% in electricity cost and between 10% and 40% in the mean value of the daily average temperature violation rate compared to RBC and online DRL controllers. Moreover, online TL maximises self-sufficiency and self-consumption by 9% and 11% with respect to RBC. Conversely, online TL achieves worse performance compared to offline DRL in either transductive or inductive settings. However, offline Deep Reinforcement Learning (DRL) agents should be trained at least for 15 episodes to reach the same level of performance as the online TL. Therefore, the proposed online TL methodology is effective, completely model-free and it can be directly implemented in real buildings with satisfying performance.

Deep reinforcement learning-assisted extended state observer for run-to-run control in the semiconductor manufacturing process

In the semiconductor manufacturing process, extended state observer (ESO)-based run-to-run (RtR) control is an intriguing solution. Although an ESO-RtR control strategy can effectively compensate for the lumped disturbance, appropriate gains are required. In this article, a cutting-edge deep reinforcement learning (DRL) technique is integrated into ESO-RtR, and a composite control framework of DRL-ESO-RtR is developed. In particular, the well-trained DRL agent serves as an assisted controller, which produces appropriate gains of ESO. The optimized ESO then presents a preferable control recipe for the manufacturing process. Under the RtR framework, the gain adjustment problem of ESO is formulated as a Markov decision process. An efficient state space and reward function are wisely designed using the system’s observable information. Correspondingly, the gain of the ESO is adaptively adjusted to cope with changing environmental disturbances. Finally, a twin-delayed deep deterministic policy gradient algorithm is employed to implement the suggested scheme. The feasibility and superiority of the developed method are validated in a deep reactive ion etching process. Comparative results demonstrate that the presented scheme outperforms the ordinary ESO-RtR controller in terms of disturbance rejection.

A K-means Supported Reinforcement Learning Framework to Multi-dimensional Knapsack

AbstractIn this paper, we address the difficulty of solving large-scale multi-dimensional knapsack instances (MKP), presenting a novel deep reinforcement learning (DRL) framework. In this DRL framework, we train different agents compatible with a discrete action space for sequential decision-making while still satisfying any resource constraint of the MKP. This novel framework incorporates the decision variable values in the 2D DRL where the agent is responsible for assigning a value of 1 or 0 to each of the variables. To the best of our knowledge, this is the first DRL model of its kind in which a 2D environment is formulated, and an element of the DRL solution matrix represents an item of the MKP. Our framework is configured to solve MKP instances of different dimensions and distributions. We propose a K-means approach to obtain an initial feasible solution that is used to train the DRL agent. We train four different agents in our framework and present the results comparing each of them with the CPLEX commercial solver. The results show that our agents can learn and generalize over instances with different sizes and distributions. Our DRL framework shows that it can solve medium-sized instances at least 45 times faster in CPU solution time and at least 10 times faster for large instances, with a maximum solution gap of 0.28% compared to the performance of CPLEX. Furthermore, at least 95% of the items are predicted in line with the CPLEX solution. Computations with DRL also provide a better optimality gap with respect to state-of-the-art approaches.