A Reinforcement Learning framework for Wake Steering of Wind Turbines

Abstract

Ideally, optimum power for a single turbine is obtained when the wind-turbine is aligned with the wind direction. However in multi-turbine wind-farm set-up, wake effects lead to decreased power production from downstream turbine [1, 2, 3, 4, 5]. Hence, a control strategy based on wake steering involves misalignment of upstream turbines with the wind direction causing their wakes to deflect away from downstream wind turbines needs to be investigated. A great deal of work has been put into dynamically controlling the orientation of the individual wind turbines to maximize the power output of the farm [6, 7, 8, 9]. In the wake-steering based control, the misaligned wind turbines produce less power, while the performance of downstream turbines gets enhanced which increases overall net power gain for the wind power plant. Traditionally, the benefits of wake steering have been demonstrated assuming fixed wind directions (e.g., using high-fidelity modeling). Amongst the most recent techniques, particularly promising is the use of Reinforcement learning (RL), which is a branch of machine learning where models are trained to make decisions based on observations of their environment. It is a flexible framework for devising strategies for solving optimal control problems in a broad range of applications across the sciences. Early attempts at using Reinforcement learning for wake steering have been carried out [7, 8, 9], and show promising results. In practice, however, wake-steering controllers must operate in dynamic wind environments in which the wind conditions are estimated from imperfect measurements. Hence, a reinforcement learning framework is developed in this work for dynamic wind conditions. The results show that the framework is promising, and we compare the deep reinforcement learning approach against a considerably more expensive traditional optimization approach which serves as a good baseline. Future work could include looking at more realistic wake models, steering in the presence of noisy observations, and incorporating weather predictions.