Deep Reinforcement Learning Agent Research Articles

I-Rebalance: Personalized Vehicle Repositioning for Supply Demand Balance

Ride-hailing platforms have been facing the challenge of balancing demand and supply. Existing vehicle reposition techniques often treat drivers as homogeneous agents and relocate them deterministically, assuming compliance with the reposition. In this paper, we consider a more realistic and driver-centric scenario where drivers have unique cruising preferences and can decide whether to take the recommendation or not on their own. We propose i-Rebalance, a personalized vehicle reposition technique with deep reinforcement learning (DRL). i-Rebalance estimates drivers' decisions on accepting reposition recommendations through an on-field user study involving 99 real drivers. To optimize supply-demand balance and enhance preference satisfaction simultaneously, i-Rebalance has a sequential reposition strategy with dual DRL agents: Grid Agent to determine the reposition order of idle vehicles, and Vehicle Agent to provide personalized recommendations to each vehicle in the pre-defined order. This sequential learning strategy facilitates more effective policy training within a smaller action space compared to traditional joint-action methods. Evaluation of real-world trajectory data shows that i-Rebalance improves driver acceptance rate by 38.07% and total driver income by 9.97%.

Testing of Deep Reinforcement Learning Agents with Surrogate Models

Deep Reinforcement Learning (DRL) has received a lot of attention from the research community in recent years. As the technology moves away from game playing to practical contexts, such as autonomous vehicles and robotics, it is crucial to evaluate the quality of DRL agents. In this article, we propose a search-based approach to test such agents. Our approach, implemented in a tool called Indago , trains a classifier on failure and non-failure environment (i.e., pass) configurations resulting from the DRL training process. The classifier is used at testing time as a surrogate model for the DRL agent execution in the environment, predicting the extent to which a given environment configuration induces a failure of the DRL agent under test. The failure prediction acts as a fitness function, guiding the generation towards failure environment configurations, while saving computation time by deferring the execution of the DRL agent in the environment to those configurations that are more likely to expose failures. Experimental results show that our search-based approach finds 50% more failures of the DRL agent than state-of-the-art techniques. Moreover, such failures are, on average, 78% more diverse; similarly, the behaviors of the DRL agent induced by failure configurations are 74% more diverse.

Dynamic scheduling of decentralized high-end equipment R&D projects via deep reinforcement learning

Distributed research and development (R&D) plays a pivotal role in high-end equipment manufacturing enterprises, significantly impacting interdisciplinary innovation and expediting development cycles. In the uncertain environment characterized by random arrivals of new projects, achieving dynamic scheduling and real-time coordination for decentralized R&D projects is a challenging problem. This paper studies a dynamic decentralized resource-constrained multi-project scheduling problem with product transfers (DDRCMPSP-PT). The objective is to minimize the development cycle after initial local schedules are generated to minimize the individual project makespan. To tackle the problem, we develop a decentralized multi-agent system using the dynamic coordination mechanism for resource assignment (DMAS/DCMRA). An up-to-date deep reinforcement learning (DRL) algorithm, dueling double deep Q-learning (D3QN) with prioritized replay, is employed to select the optimal strategy to resolve the global resource conflicts adaptively. Two priority rules are presented based on the structural properties and make up the action space of the DRL agent together with ten high-quality priority rules. Extensive simulation experiment results based on real enterprise cases reveal that the DMAS/DCMRA outperforms all traditional priority rule-based methods. The proposed priority rules also show competitive performance compared to other priority rules, which validates the applicability and superiority of the proposed method. This research offers a new agile project management technique for organizations running multiple R&D projects and managing crucial bottleneck resources.

Shovel allocation and scheduling for open-pit mining using deep reinforcement learning

ABSTRACT The open-pit production system is a highly dynamic and uncertain environment with complex interactions between haulage and loading equipment on a shared road network. One of the key decisions in open-pit short-term planning is the allocation sequence of shovels to mining faces to meet the production targets established by long- and medium-term strategic plans. Deep Reinforcement Learning(DRL) techniques are commonly used in dynamic production environments. In this approach, an agent is trained on a simulation of the production system to learn the optimal decisions based on the system’s current state. This paper proposes a DRL approach based on the Deep Q-Learning algorithm to obtain a robust shovel allocation plan for open-pit short-term planning. First, a discrete-event simulation of the mining production system incorporating trucks, shovels, crushers, waste dumps, and the road network is created. This simulation models the uncertainties of each component’s operating cycle based on historical activity records, and it is used to train the DRL agent. The goal is to learn a robust shovel allocation strategy for the next production quarter, 3 months, to meet the tonnes per hour (TPH) production target to be delivered to the crusher feeds by interacting with the production simulator. The framework is tested in an iron ore open-pit mine case study where the shovel allocation agent successfully learns a strategy that consistently delivers the desired production target.

Smart urban windcatcher: Conception of an AI-empowered wind-channeling system for real-time enhancement of urban wind environment

The dense configuration and rapid proliferation of high-rise buildings in central Hong Kong have led to increasing stagnation of pedestrian-level airflows, lowering the wind speed and exacerbating issues in wind and thermal comfort. Addressing these problems can be difficult without significant building renovations and urban re-planning. This paper introduces a novel framework for the real-time enhancement of the local urban wind environment using motion-controlled billboards, or smart urban windcatchers, managed by a deep reinforcement learning (DRL) agent. The DRL agent determines the windcatchers' optimal positions based on street-level sensor data, independent of meteorological data. The framework's effectiveness was assessed using a simplified city grid model, where two windcatchers aimed to optimize the pedestrian-level wind environment on a specific street. Simulation results indicated that the windcatchers could effectively alter the flow direction in the streets, promoting or diverting air passages per demand. The DRL agent also gave accurate instructions to the windcatchers under various weather conditions, achieving near-optimal wind environment scores. This paper introduces the conception and confirms the feasibility of the windcatcher system, laying the groundwork for future research—as it is much needed and welcomed—to enhance the system and overcome the acknowledged challenges in large-scale, real-world implementation.

An innovative heterogeneous transfer learning framework to enhance the scalability of deep reinforcement learning controllers in buildings with integrated energy systems

AbstractDeep Reinforcement Learning (DRL)-based control shows enhanced performance in the management of integrated energy systems when compared with Rule-Based Controllers (RBCs), but it still lacks scalability and generalisation due to the necessity of using tailored models for the training process. Transfer Learning (TL) is a potential solution to address this limitation. However, existing TL applications in building control have been mostly tested among buildings with similar features, not addressing the need to scale up advanced control in real-world scenarios with diverse energy systems. This paper assesses the performance of an online heterogeneous TL strategy, comparing it with RBC and offline and online DRL controllers in a simulation setup using EnergyPlus and Python. The study tests the transfer in both transductive and inductive settings of a DRL policy designed to manage a chiller coupled with a Thermal Energy Storage (TES). The control policy is pre-trained on a source building and transferred to various target buildings characterised by an integrated energy system including photovoltaic and battery energy storage systems, different building envelope features, occupancy schedule and boundary conditions (e.g., weather and price signal). The TL approach incorporates model slicing, imitation learning and fine-tuning to handle diverse state spaces and reward functions between source and target buildings. Results show that the proposed methodology leads to a reduction of 10% in electricity cost and between 10% and 40% in the mean value of the daily average temperature violation rate compared to RBC and online DRL controllers. Moreover, online TL maximises self-sufficiency and self-consumption by 9% and 11% with respect to RBC. Conversely, online TL achieves worse performance compared to offline DRL in either transductive or inductive settings. However, offline Deep Reinforcement Learning (DRL) agents should be trained at least for 15 episodes to reach the same level of performance as the online TL. Therefore, the proposed online TL methodology is effective, completely model-free and it can be directly implemented in real buildings with satisfying performance.

Deep reinforcement learning-assisted extended state observer for run-to-run control in the semiconductor manufacturing process

In the semiconductor manufacturing process, extended state observer (ESO)-based run-to-run (RtR) control is an intriguing solution. Although an ESO-RtR control strategy can effectively compensate for the lumped disturbance, appropriate gains are required. In this article, a cutting-edge deep reinforcement learning (DRL) technique is integrated into ESO-RtR, and a composite control framework of DRL-ESO-RtR is developed. In particular, the well-trained DRL agent serves as an assisted controller, which produces appropriate gains of ESO. The optimized ESO then presents a preferable control recipe for the manufacturing process. Under the RtR framework, the gain adjustment problem of ESO is formulated as a Markov decision process. An efficient state space and reward function are wisely designed using the system’s observable information. Correspondingly, the gain of the ESO is adaptively adjusted to cope with changing environmental disturbances. Finally, a twin-delayed deep deterministic policy gradient algorithm is employed to implement the suggested scheme. The feasibility and superiority of the developed method are validated in a deep reactive ion etching process. Comparative results demonstrate that the presented scheme outperforms the ordinary ESO-RtR controller in terms of disturbance rejection.

A K-means Supported Reinforcement Learning Framework to Multi-dimensional Knapsack

In this paper, we address the difficulty of solving large-scale multi-dimensional knapsack instances (MKP), presenting a novel deep reinforcement learning (DRL) framework. In this DRL framework, we train different agents compatible with a discrete action space for sequential decision-making while still satisfying any resource constraint of the MKP. This novel framework incorporates the decision variable values in the 2D DRL where the agent is responsible for assigning a value of 1 or 0 to each of the variables. To the best of our knowledge, this is the first DRL model of its kind in which a 2D environment is formulated, and an element of the DRL solution matrix represents an item of the MKP. Our framework is configured to solve MKP instances of different dimensions and distributions. We propose a K-means approach to obtain an initial feasible solution that is used to train the DRL agent. We train four different agents in our framework and present the results comparing each of them with the CPLEX commercial solver. The results show that our agents can learn and generalize over instances with different sizes and distributions. Our DRL framework shows that it can solve medium-sized instances at least 45 times faster in CPU solution time and at least 10 times faster for large instances, with a maximum solution gap of 0.28% compared to the performance of CPLEX. Furthermore, at least 95% of the items are predicted in line with the CPLEX solution. Computations with DRL also provide a better optimality gap with respect to state-of-the-art approaches.

DeepLS: Local Search for Network Optimization Based on Lightweight Deep Reinforcement Learning

Deep Reinforcement Learning (DRL) is being investigated as a competitive alternative to traditional techniques for solving network optimization problems. A promising research direction lies in enhancing traditional optimization algorithms by offloading low-level decisions to a DRL agent. In this study, we consider how to effectively employ DRL to improve the performance of Local Search algorithms, i.e., algorithms that, starting from a candidate solution, explore the solution space by iteratively applying local changes (i.e., moves), yielding the best solution found in the process. We propose a Local Search algorithm based on lightweight Deep Reinforcement Learning (DeepLS) that, given a neighborhood, queries a DRL agent for choosing a move, with the goal of achieving the best objective value in the long term. Our DRL agent, based on permutation-equivariant neural networks, is composed by less than a hundred parameters, requiring only up to ten minutes of training and can evaluate problem instances of arbitrary size, generalizing to networks and traffic distributions unseen during training. We evaluate DeepLS on two illustrative NP-Hard network routing problems, namely OSPF Weight Setting and Routing and Wavelength Assignment, training on a single small network only and evaluating on instances 2x-10x larger than training. Experimental results show that DeepLS outperforms existing DRL-based approaches from literature and attains competitive results with state-of-the-art metaheuristics, with computing times up to 8x smaller than the strongest algorithmic baselines.

Deep reinforcement learning‐based active mass driver decoupled control framework considering control–structure interaction effects

AbstractControl–structure interaction (CSI) plays a significant role in active control systems. Popular methods incorporate actuator dynamics into an integrated control system to account for CSI, leading to a situation where existing structural control algorithms that ignore CSI cannot be applied directly. To address this issue, this study proposes a deep reinforcement learning (DRL) based active mass driver (AMD) decoupled control framework, in which a structural control algorithm is employed to generate the control force command without consideration of CSI, while a DRL agent is utilized to attenuate the CSI effects of AMD systems and achieve a desired control force. The DRL‐based AMD control framework is verified through a series of numerical experiments. Furthermore, the applicability of the control framework is confirmed in a wind‐excited 76‐story benchmark building. Comprehensive analysis indicates that the proposed control framework can effectively eliminate the CSI effects and apply accurate control force to the structure in various scenarios, which allows for an ideal structural response control.

A Collaborative Visual Sensing System for Precise Quality Inspection at Manufacturing Lines

Visual sensing has been widely adopted for quality inspection in production processes. This paper presents the design and implementation of a smart collaborative camera system, called BubCam , for automated quality inspection of manufactured ink bags in Hewlett-Packard (HP) Inc.’s factories. Specifically, BubCam estimates the volume of air bubbles in an ink bag, which may affect the printing quality. The design of BubCam faces challenges due to the dynamic ambient light reflection, motion blur effect, and data labeling difficulty. As a starting point, we design a single-camera system which leverages various deep learning (DL)-based image segmentation and depth fusion techniques. New data labeling and training approaches are proposed to utilize prior knowledge of the production system for training the segmentation model with a small dataset. Then, we design a multi-camera system which additionally deploys multiple wireless cameras to achieve better accuracy due to multi-view sensing. To save power of the wireless cameras, we formulate a configuration adaptation problem and develop the single-agent and multi-agent deep reinforcement learning (DRL)-based solutions to adjust each wireless camera’s operation mode and frame rate in response to the changes of presence of air bubbles and light reflection. The multi-agent DRL approach aims to reduce the retraining costs during the production line reconfiguration process by only retraining the DRL agents for the newly added cameras and the existing cameras with changed positions. Extensive evaluation on a lab testbed and real factory trial shows that BubCam outperforms six baseline solutions including the current manual inspection and existing bubble detection and camera configuration adaptation approaches. In particular, BubCam achieves 1.3x accuracy improvement and 300x latency reduction, compared with the manual inspection approach.

Deep reinforcement learning applied to statistical arbitrage investment strategy on cryptomarket

In the complex and dynamic nature of financial markets where increasingly sophisticated investment strategies are required, deep reinforcement learning (DRL) has proven successful in generating real-time investment strategies, outperforming classical models. Alongside this, statistical arbitrage strategies exploit temporary market inefficiencies to generate returns. In this regard, a novel DRL-based arbitrage method has been developed.This paper presents a unified framework that combines classical statistical arbitrage theory with DRL framework to generate investment strategies. The framework addresses the challenges of identifying similar asset portfolios, extracting signals indicating temporary price deviations, and determining optimal trading rules given market conditions. The proposed methodology involves constructing arbitrage portfolios based on cointegration relationships, removing signals from price series and portfolios, and using a DRL agent to make optimal decisions within a fixed time horizon.The empirical analysis focuses on the cryptocurrency market, known for its volatility and risk. Results demonstrate that DRL agents can generate strategies with positive returns in out-of-sample periods 79.52% to 112.82% with no transaction cost, outperforming market benchmark Bitcoin 32.51% return, the best performing over the period Litecoin with 57.11% return and the worst performing Solana with a 35.70% loss. Moreover, these strategies effectively reduce risk, achieving higher risk-adjusted returns on individual assets. The strategies maintain positive returns when considering transaction costs, with the DRL agent outperforming the standard arbitrage strategy. The best-performing strategy is based on a Deep Q-Network (DQN) agent with a return of 18.39%, annualized volatility of 12.22%, and an annualized Sharpe ratio of 2.43. At the same time, Bitcoin holds an annualized volatility of 44.13% and a Sharpe ratio of 1.08.Their randomness and coherence are studied to verify the robustness of the agents’ decisions. The actions generated by the agents are not random but based on well-founded policies, which align with the obtained results.

Deep reinforcement learning for microstructural optimisation of silica aerogels

Silica aerogels are being extensively studied for aerospace and transportation applications due to their diverse multifunctional properties. While their microstructural features dictate their thermal, mechanical, and acoustic properties, their accurate characterisation remains challenging due to their nanoporous morphology and the stochastic nature of gelation. In this work, a deep reinforcement learning (DRL) framework is presented to optimise silica aerogel microstructures modelled with the diffusion-limited cluster–cluster aggregation (DLCA) algorithm. For faster computations, two environments consisting of DLCA surrogate models are tested with the DRL framework for inverse microstructure design. The DRL framework is shown to effectively optimise the microstructure morphology, wherein the error of the material properties achieved is dependent upon the complexity of the environment. However, in all cases, with adequate training of the DRL agent, material microstructures with desired properties can be achieved by the framework. Thus, the methodology provides a resource-efficient means to design aerogels, offering computational advantages over experimental iterations or direct numerical solutions.

Coverage-guided fuzzing for deep reinforcement learning systems

While the past decade has witnessed a growing demand for employing deep reinforcement learning (DRL) in various domains to solve real-world problems, the reliability of DRL systems has become more of a concern. In particular, DRL agents are often trained on data from a potentially biased distribution over environmental settings, causing the trained agents to fail in certain cases despite high average-case performance. Hence, it is necessary and urgent to adequately test DRL agents to ensure the reliability of practical DRL systems. However, due to the fundamental difference in the programming paradigm and the development process, traditional software testing methodology cannot be applied directly to DRL systems. Given that, we introduce a novel testing framework for DRL systems, aiming to generate diverse test cases that can drive a DRL system to fail. Specifically, we design, implement and evaluate DRLFuzz, which is a coverage-guided fuzzing (CGF) framework for systematically testing DRL systems. Experimental results demonstrate that DRLFuzz can efficiently discover diverse failures in different DRL systems for various benchmark tasks. Compared with a random search baseline, DRLFuzz can generate 60% more failed cases in general. Additionally, the diversity of failed cases generated by DRLFuzz is increased by 4.6%∼14.1% in terms of mean pairwise distance (MPD). Furthermore, our experiments also indicate that the failed cases generated by DRLFuzz can be utilized to fine-tune the DRL agent to eliminate the failures resulting from inadequate exploration during training and thus improve the reliability of DRL systems.

MACNS: A generic graph neural network integrated deep reinforcement learning based multi-agent collaborative navigation system for dynamic trajectory planning

Multi-agent collaborative navigation is prevalent in modern transportation systems, including delivery logistics, warehouse automation, and personalised tourism, where multiple moving agents (e.g., robots and people) must meet at a common destination from different starting points. However, traditional methods face challenges in efficiently and accurately optimising routes for multiple moving agents (e.g., robots). These include dynamically adjusting the common destination in response to changing traffic conditions, encoding the real-world map, and the training speed. Therefore, we propose a generic Multi-Agent Collaborative Navigation System (MACNS) to address those challenges. First, we formalise the solution of the problem and challenges into a Markov Decision Process (MDP), which is further developed as a training environment where a Deep Reinforcement Learning (DRL) agent can learn patterns efficiently. Second, the proposed framework integrates a Graph Neural Network (GNN) into the policy network of Proximal Policy Optimisation for the homogeneous decision-making of each individual agent, showing excellent generalisation, extensibility and convergence speed. Finally, we demonstrate how MACNS can be applied and implemented in a real-world use case: tour groups gathering and picking up. Extensive simulations and real-world tests validate the effectiveness of the MACNS-based use case, showcasing its superiority over other state-of-the-art PPO-related methods.

Imitation learning enabled fast and adaptive task scheduling in cloud

Studies of resource provision in cloud computing have drawn extensive attention, since effective task scheduling solutions promise an energy-efficient way of utilizing resources while meeting diverse requirements of users. Deep reinforcement learning (DRL) has demonstrated its outstanding capability in tackling this issue with the ability of online self-learning, however, it is still prevented by the low sampling efficiency, poor sample validity, and slow convergence speed especially for deadline constrained applications. To address these challenges, an Imitation Learning Enabled Fast and Adaptive Task Scheduling (ILETS) framework based on DRL is proposed in this paper. First, we introduce behavior cloning to provide a well-behaved and robust model through Offline Initial Network Parameters Training (OINPT) so as to guarantee the initial decision-making quality of DRL. Next, we design a novel Online Asynchronous Imitation Learning (OAIL)-based method to assist the DRL agent to re-optimize its policy and to against the oscillations caused by the high dynamic of the cloud, which promises DRL agent moving towards the optimal policy with a fast and stable process. Extensive experiments on the real-world dataset have demonstrated that the proposed ILETS can consistently produce shorter response time, lower energy consumption and higher success rate than the baselines and other state-of-the-art methods at the accelerated convergence speed.

Employing Deep Reinforcement Learning to Maximize Lower Limb Blood Flow Using Intermittent Pneumatic Compression.

Intermittent pneumatic compression (IPC) systems apply external pressure to the lower limbs and enhance peripheral blood flow. We previously introduced a cardiac-gated compression system that enhanced arterial blood velocity (BV) in the lower limb compared to fixed compression timing (CT) for seated and standing subjects. However, these pilot studies found that the CT that maximized BV was not constant across individuals and could change over time. Current CT modelling methods for IPC are limited to predictions for a single day and one heartbeat ahead. However, IPC therapy for may span weeks or longer, the BV response to compression can vary with physiological state, and the best CT for eliciting the desired physiological outcome may change, even for the same individual. We propose that a deep reinforcement learning (DRL) algorithm can learn and adaptively modify CT to achieve a selected outcome using IPC. Herein, we target maximizing lower limb arterial BV as the desired outcome and build participant-specific simulated lower limb environments for 6 participants. We show that DRL can adaptively learn the CT for IPC that maximized arterial BV. Compared to previous work, the DRL agent achieves 98% ± 2 of the resultant blood flow and is faster at maximizing BV; the DRL agent can learn an "optimal" policy in 15 minutes ± 2 on average and can adapt on the fly. Given a desired objective, we posit that the proposed DRL agent can be implemented in IPC systems to rapidly learn the (potentially time-varying) "optimal" CT with a human-in-the-loop.

Toward Interpretable-AI Policies Using Evolutionary Nonlinear Decision Trees for Discrete-Action Systems.

Black-box artificial intelligence (AI) induction methods such as deep reinforcement learning (DRL) are increasingly being used to find optimal policies for a given control task. Although policies represented using a black-box AI are capable of efficiently executing the underlying control task and achieving optimal closed-loop performance-controlling the agent from the initial time step until the successful termination of an episode, the developed control rules are often complex and neither interpretable nor explainable. In this article, we use a recently proposed nonlinear decision-tree (NLDT) approach to find a hierarchical set of control rules in an attempt to maximize the open-loop performance for approximating and explaining the pretrained black-box DRL (oracle) agent using the labeled state-action dataset. Recent advances in nonlinear optimization approaches using evolutionary computation facilitate finding a hierarchical set of nonlinear control rules as a function of state variables using a computationally fast bilevel optimization procedure at each node of the proposed NLDT. In addition, we propose a reoptimization procedure for enhancing the closed-loop performance of an already derived NLDT. We evaluate our proposed methodologies (open- and closed-loop NLDTs) on different control problems having multiple discrete actions. In all these problems, our proposed approach is able to find relatively simple and interpretable rules involving one to four nonlinear terms per rule, while simultaneously achieving on par closed-loop performance when compared to a trained black-box DRL agent. A postprocessing approach for simplifying the NLDT is also suggested. The obtained results are inspiring as they suggest the replacement of complicated black-box DRL policies involving thousands of parameters (making them noninterpretable) with relatively simple interpretable policies. The results are encouraging and motivating to pursue further applications of proposed approach in solving more complex control tasks.

Structure-Enhanced DRL for Optimal Transmission Scheduling

Remote state estimation of large-scale distributed dynamic processes plays an important role in Industry 4.0 applications. In this paper, we focus on the transmission scheduling problem of a remote estimation system. First, we derive some structural properties of the optimal sensor scheduling policy over fading channels. Then, building on these theoretical guidelines, we develop a structure-enhanced deep reinforcement learning (DRL) framework for optimal scheduling of the system to achieve the minimum overall estimation mean-square error (MSE). In particular, we propose a structure-enhanced action selection method, which tends to select actions that obey the policy structure. This explores the action space more effectively and enhances the learning efficiency of DRL agents. Furthermore, we introduce a structure-enhanced loss function to add penalties to actions that do not follow the policy structure. The new loss function guides the DRL to converge to the optimal policy structure quickly. Our numerical experiments illustrate that the proposed structure-enhanced DRL algorithms can save the training time by 50% and reduce the remote estimation MSE by 10% to 25%, when compared to benchmark DRL algorithms. In addition, we show that the derived structural properties exist in a wide range of dynamic scheduling problems that go beyond remote state estimation.

Deep Generative Adversarial Reinforcement Learning for Semi-Supervised Segmentation of Low-Contrast and Small Objects in Medical Images.

Deep reinforcement learning (DRL) has demonstrated impressive performance in medical image segmentation, particularly for low-contrast and small medical objects. However, current DRL-based segmentation methods face limitations due to the optimization of error propagation in two separate stages and the need for a significant amount of labeled data. In this paper, we propose a novel deep generative adversarial reinforcement learning (DGARL) approach that, for the first time, enables end-to-end semi-supervised medical image segmentation in the DRL domain. DGARL ingeniously establishes a pipeline that integrates DRL and generative adversarial networks (GANs) to optimize both detection and segmentation tasks holistically while mutually enhancing each other. Specifically, DGARL introduces two innovative components to facilitate this integration in semi-supervised settings. First, a task-joint GAN with two discriminators links the detection results to the GAN's segmentation performance evaluation, allowing simultaneous joint evaluation and feedback. This ensures that DRL and GAN can be directly optimized based on each other's results. Second, a bidirectional exploration DRL integrates backward exploration and forward exploration to ensure the DRL agent explores the correct direction when forward exploration is disabled due to lack of explicit rewards. This mitigates the issue of unlabeled data being unable to provide rewards and rendering DRL unexplorable. Comprehensive experiments on three generalization datasets, comprising a total of 640 patients, demonstrate that our novel DGARL achieves 85.02% Dice and improves at least 1.91% for brain tumors, achieves 73.18% Dice and improves at least 4.28% for liver tumors, and achieves 70.85% Dice and improves at least 2.73% for pancreas compared to the ten most recent advanced methods, our results attest to the superiority of DGARL. Code is available at GitHub.