Multi-layer double deep Q network for active distribution network equivalent modeling with internal identification for EV loads

Abstract

With more and more distributed renewable energy generations and diversified electric vehicle (EV) loads connected to active distribution networks (ADNs), the power interaction between the ADNs and backbone network becomes more complicated. Furthermore, it is difficult to exactly simulate the power flows in the ADN due to large amounts of branches without power measurement units. Consequently, this paper proposes an equivalent modeling for the ADN, consisting of a motor and a synthetic constant impedance (Z), constant current (I), constant power (P) load (ZIP) with an internal identification for EV loads. The equivalent modeling utilizes a multi-layer double deep Q network (MLDDQN) to track the dynamics of the original ADN using few power measurements in the boundary with high precision. In the MLDDQN, the types of the ZIP load and motor are selected from the load pool to obtain the dynamic equivalent modeling of the ADN in the first layer, and the weight of the model is determined to obtain the active power (P) and reactive power (Q) (PQ component) of motor and ZIP load in the second layer through the powerful feature extraction ability of deep learning and the optimization decision-making ability of reinforcement learning. Another layer is adopted to optimize the parameters of the equivalent EV to independently trace the characteristics of the original EV loads. The simulation results show that the proposed MLDDQN improves the sample selection and behavior value function evaluation in deep reinforcement learning through prior experience playback, huber loss function strategy and dueling network. Additionally, the precision and applicability of the equivalent modeling is tested with comparisons with traditional DDQN and DQN.