Deep Reinforcement Learning-Based Routing and Spectrum Assignment of EONs by Exploiting GCN and RNN for Feature Extraction

Abstract

Deep reinforcement learning (DRL) has been introduced to the routing and spectrum assignment (RSA) of elastic optical networks (EONs) where the RSA policies are learnt during the interaction of a DRL agent with the EON environment. Upon each new traffic arrival, the DRL agent senses the EON state, and makes the RSA decision accordingly. Therefore, the EON state feature extraction is essential for the performance. However, current approaches, mainly based on fully connected neural networks or convolutional neural networks, are unsatisfactory in EON state feature extraction due to the following two reasons. (1) The optical network topology information is not well considered. (2) The path-level features are highly related to the RSA decision, due to the spectrum continuity constraint, while path-level feature extraction has not been well exploited. To overcome the above shortcomings, in this paper, we propose to utilize the Graph Convolutional Neural Network (GCN) and the Recurrent Neural Network (RNN) for the feature extraction of the network topology, and the aggregation of link-level features to path-level features. By doing this, critical RSA-related information can be sensed by the DRL agent to make better actions. Simulation results demonstrate that our proposed method outperforms previous approaches.