Deep Reinforcement Learning Approach for Traffic Light Control and Transit Priority

Here, the challenges of sample efficiency and navigation performance in deep reinforcement learning for visual navigation are focused and a deep imitation reinforcement learning approach is proposed. Our contributions are mainly three folds: first, a framework combining imitation learning with deep reinforcement learning is presented, which enables a robot to learn a stable navigation policy faster in the target‐driven navigation task. Second, the surrounding images is taken as the observation instead of sequential images, which can improve the navigation performance for more information. Moreover, a simple yet efficient template matching method is adopted to determine the stop action, making the system more practical. Simulation experiments in the AI‐THOR environment show that the proposed approach outperforms previous end‐to‐end deep reinforcement learning approaches, which demonstrate the effectiveness and efficiency of our approach.

Energy efficient task scheduling based on deep reinforcement learning in cloud environment: A specialized review

Reliable, deterministic real-time communication is fundamental to most industrial systems today. In many other domains Ethernet has become the most common platform for communication networks, but has been unsuitable to satisfy the requirements of industrial networks for a long time. This has changed with the introduction of Time-Sensitive-Networking (TSN), a set of standards utilizing Ethernet to implement deterministic real-time networks. This makes Ethernet a viable alternative to the expensive fieldbus systems commonly used in industrial environments. However, TSN is not a silver bullet. Industrial networks are a complex and highly dynamic environment and the configuration of TSN, especially with respect to latency, is a challenging but crucial task. Various approaches have been pursued for the configuration of TSN in dynamic industrial environments. Optimization techniques like Linear Programming (LP) are able to determine an optimal configuration for a given network, but the time consumption exponentially increases with the complexity of the environment. Machine Learning (ML) has become widely popular in the last years and is able to approximate a near-optimal TSN configuration for networks of different complexity. Yet, ML models are usually trained in a supervised manner which requires large amounts of data that have to be generated for the specific environment. Therefore, supervised methods are not scalable and do not adapt to changing dynamics of the network environment. To address these issues, this work proposes a Deep Reinforcement Learning (DRL) approach to the configuration of TSN in industrial networks. DRL combines two different disciplines, Deep Learning (DL) and Reinforcement Learning (RL), and has gained considerable traction in the last years due to breakthroughs in various domains. RL is supposed to autonomously learn a challenging task like the configuration of TSN without requiring any training data. The addition of DL allows to apply well-studied RL methods to a complex environment such as dynamic industrial networks. There are two major contributions made in this work. In the first step, an interactive environment is proposed which allows for the simulation and configuration of industrial networks using basic TSN mechanisms. The environment provides an interface that allows to apply various DRL methods to the problem of TSN configuration. The second contribution of this work is an in-depth study on the application of two fundamentally different DRL methods to the proposed environment. Both methods are evaluated on networks of different complexity and the results are compared to the ground truth and to the results of two supervised ML approaches. Ultimately, this work investigates if DRL can adapt to changing dynamics of the environment in a more scalable manner than supervised methods.

Learning-based approaches for solving large sequential decision making problems have become popular in recent years. The resulting agents perform differently and their characteristics depend on those of the underlying learning approach. Here, we consider a benchmark planning problem from the reinforcement learning domain, the Racetrack, to investigate the properties of agents derived from different deep (reinforcement) learning approaches. We compare the performance of deep supervised learning, in particular imitation learning, to reinforcement learning for the Racetrack model. We find that imitation learning yields agents that follow more risky paths. In contrast, the decisions of deep reinforcement learning are more foresighted, i.e., avoid states in which fatal decisions are more likely. Our evaluations show that for this sequential decision making problem, deep reinforcement learning performs best in many aspects even though for imitation learning optimal decisions are considered.KeywordsDeep reinforcement learningImitation learning

Along with the evolution of computer microarchitecture over the years, the number of dies, cores, and embedded multi-die interconnect bridges has grown. Optimizing the workload running on a central processing unit (CPU) to improve the computer performance has become a challenge. Matching workloads to systems with optimal system configurations to achieve the desired system performance is an open challenge in both academic and industrial research. In this paper, we propose two reinforcement learning (RL) approaches, a deep reinforcement learning (DRL) approach and an evolutionary deep reinforcement learning (EDRL) approach, to find an optimal system configuration for a given computer workload with a system performance objective. The experimental results demonstrate that both approaches can determine the optimal system configuration with the desired performance objective. The comparison studies illustrate that the DRL approach outperforms the standard RL approaches. In the future, these DRL approaches can be leveraged in system performance auto-tuning studies.

Sample effficient deep reinforcement learning for control

Intelligent fault diagnosis for rotating machinery using deep Q-network based health state classification: A deep reinforcement learning approach

Deep reinforcement learning (DRL) has demonstrated impressive performance in medical image segmentation, particularly for low-contrast and small medical objects. However, current DRL-based segmentation methods face limitations due to the optimization of error propagation in two separate stages and the need for a significant amount of labeled data. In this paper, we propose a novel deep generative adversarial reinforcement learning (DGARL) approach that, for the first time, enables end-to-end semi-supervised medical image segmentation in the DRL domain. DGARL ingeniously establishes a pipeline that integrates DRL and generative adversarial networks (GANs) to optimize both detection and segmentation tasks holistically while mutually enhancing each other. Specifically, DGARL introduces two innovative components to facilitate this integration in semi-supervised settings. First, a task-joint GAN with two discriminators links the detection results to the GAN's segmentation performance evaluation, allowing simultaneous joint evaluation and feedback. This ensures that DRL and GAN can be directly optimized based on each other's results. Second, a bidirectional exploration DRL integrates backward exploration and forward exploration to ensure the DRL agent explores the correct direction when forward exploration is disabled due to lack of explicit rewards. This mitigates the issue of unlabeled data being unable to provide rewards and rendering DRL unexplorable. Comprehensive experiments on three generalization datasets, comprising a total of 640 patients, demonstrate that our novel DGARL achieves 85.02% Dice and improves at least 1.91% for brain tumors, achieves 73.18% Dice and improves at least 4.28% for liver tumors, and achieves 70.85% Dice and improves at least 2.73% for pancreas compared to the ten most recent advanced methods, our results attest to the superiority of DGARL. Code is available at GitHub.

Computer vision has advanced so far that machines now can think and see as we humans do. Especially deep learning has raised the bar of excellence in computer vision. However, the recent emergence of deep reinforcement learning is threatening to soar even greater heights as it combines deep neural networks with reinforcement learning along with numerous added advantages over both. This, being a relatively recent technique, has not yet seen many works, and so its true potential is yet to be unveiled. Thus, this chapter focuses on shedding light on the fundamentals of deep reinforcement learning, starting with the preliminaries followed by the theory and basic algorithms and some of its variations, namely, attention aware deep reinforcement learning, deep progressive reinforcement learning, and multi-agent deep reinforcement learning. This chapter also discusses some existing deep reinforcement learning works regarding computer vision such as image processing and understanding, video captioning and summarization, visual search and tracking, action detection, recognition and prediction, and robotics. This work further aims to elucidate the existing challenges and research prospects of deep reinforcement learning in computer vision. This chapter might be considered a starting point for aspiring researchers looking to apply deep reinforcement learning in computer vision to reach the pinnacle of performance in the field by tapping into the immense potential that deep reinforcement learning is showing.

Multi-granularity coverage criteria for deep reinforcement learning systems

Intelligent traffic signal timing is critical to reduce traffic congestion and vehicle delay. Recent studies have shown promising results of deep reinforcement learning for traffic signal control. However, existing studies have only focused on selecting which direction (phase) to let vehicles go, not on phase duration. In this paper, we propose a deep reinforcement learning algorithm that automatically learns an optimal policy to adaptively determine phase duration. To improve algorithm performance and stability, we propose a phase sensitive neural network structure based on the deep deterministic policy gradient (DDPG) model, i.e. we design a deep neural network controller for each specific traffic signal phase with DDPG; we develop some interesting training techniques to improve training efficiency, i.e. dividing the training process into three stages and introducing the episode-break mechanism. We test the proposed methods on an isolated intersection under diverse traffic demands. Experiments show that our method is more effective.

Traffic congestion has recently become a real issue especially within crowded cities and urban areas. Intelligent transportation systems (ITS) leveraged various advanced techniques aiming to optimize the traffic flow and subsequently alleviate the traffic congestion. In particular, traffic signal control TSC is one of the essential ITS techniques for controlling the traffic flow at intersections. Many research works have been proposed to develop algorithms and techniques which optimize TSC behavior. Recent works leverage Deep Learning (DL) and Reinforcement Learning (RL) techniques to optimize TSCs. However, most of Deep RL proposals are based on complex definitions of state and reward in the RL framework. In this work, we propose to use an alternative way of formulating the state and reward definitions. Basically, The basic idea is to define both state and reward in a simplified and straightforward manner rather than the complex design. We hypothesize that such a design approach simplifies the learning of the RL agent and hence provides a rapid convergence to optimal policies. For the agent architecture, we employ the double deep Q-Network (DDQN) along with prioritized experience replay (PER). We conduct the experiments using the Simulation of Urban MObility (SUMO) simulator interfaced with Python framework and we compare the performance of our proposal to traditional and learning-based techniques.

Deep reinforcement learning for stock portfolio optimization by connecting with modern portfolio theory

To develop driving automation technologies for humans, a human-centered methodology should be adopted for safety and satisfactory user experience. Automated lane change decision in dense highway traffic is challenging, especially when considering different driver preferences. This paper proposes a personalized lane change decision algorithm based on deep reinforcement learning. Firstly, driving experiments are carried out on a moving-base simulator. Based on the analysis of the experiment data, three personalization indicators are selected to describe the driver preferences in lane-change decisions. Then, a deep reinforcement learning (RL) approach is applied to design human-like agents for automated lane change decisions to capture the driver preferences, with refined rewards using the three personalization indicators. Finally, the trained RL agents and benchmark agents are tested in a two-lane highway driving scenario. Results show that the proposed algorithm can achieve higher consistency of lane change decision preferences than the comparison algorithm.

Optimal scheduling of island integrated energy systems considering multi-uncertainties and hydrothermal simultaneous transmission: A deep reinforcement learning approach