Multiobjective multihydropower reservoir operation optimization with transformer-based deep reinforcement learning

Abstract

The paper introduces a transformer-based deep reinforcement learning (T-DRL) approach designed to address the multiobjective multihydropower reservoir operation optimization (MMROO) problem. Unlike existing literature that primarily focuses on maximizing power generation from individual reservoirs, the MMROO model in this study considers the broader context of multiple reservoirs, encompassing total power generation, ecological protection, and residential area water supply. The computational challenges posed by the numerous constraints and nonlinearities of multiple reservoirs render conventional multiobjective evolutionary algorithms both expensive and lacking in generalization capabilities for solving the MMROO problem. To overcome these challenges, the paper proposes a T-DRL approach that leverages the multihead attention mechanism within the encoder module to adeptly extract complex information from reservoirs and residential areas. The two-stage encoder effectively processes diverse information separately. The multireservoir network of the decoder then generates optimal decisions based on contextual information. The case study focusing on Lake Mead and Lake Powell in the Colorado River Basin demonstrates the efficacy of the T-DRL approach, producing operation strategies that outperform a state-of-the-art method. Specifically, the proposed approach yields a 10.11% increase in electricity generation, a 39.69% reduction in amended annual proportional flow deviation, and a 4.10% rise in water supply revenue. Overall, the T-DRL approach emerges as an effective method for the multiobjective operation of multihydropower reservoir systems.