Deep Reinforcement Learning Based on Proximal Policy Optimization for the Maintenance of a Wind Farm with Multiple Crews

Luca Pinciroli,Michele Compare,Enrico Zio,Guido Ballabio,Piero Baraldi

doi:10.3390/en14206743

Luca Pinciroli, Michele Compare + Show 3 more

Open Access

https://doi.org/10.3390/en14206743

Copy DOI

Abstract

The life cycle of wind turbines depends on the operation and maintenance policies adopted. With the critical components of wind turbines being equipped with condition monitoring and Prognostics and Health Management (PHM) capabilities, it is feasible to significantly optimize operation and maintenance (O&M) by combining the (uncertain) information provided by PHM with the other factors influencing O&M activities, including the limited availability of maintenance crews, the variability of energy demand and corresponding production requests, and the long-time horizons of energy systems operation. In this work, we consider the operation and maintenance optimization of wind turbines in wind farms woth multiple crews. A new formulation of the problem as a sequential decision problem over a long-time horizon is proposed and solved by deep reinforcement learning based on proximal policy optimization. The proposed method is applied to a wind farm of 50 turbines, considering the availability of multiple maintenance crews. The optimal O&M policy found outperforms other state-of-the-art strategies, regardless of the number of available maintenance crews.

Highlights

Wind power generation is one of the most affordable ways of providing clean energy to the market
The scheduled and predictive maintenance strategies require the setting of the time interval between two consecutive maintenance interventions and the Remaining Useful Life (RUL) threshold, respectively, This is done by optimizing the wind farm profit over 250 episodes using the Tree-structured Parzen Estimator (TPE) algorithm [43]
Notice that the predictive and Reinforcement Learning (RL) policies provide better performances than the corrective and scheduled maintenance policies, which are the maintenance strategies most commonly applied to wind farms [44,45,46,47,48], with a 40% increment of the profit when the RUL predictions provided by the Prognostics and Health Management (PHM) systems are used

Summary

Introduction

Wind power generation is one of the most affordable ways of providing clean energy to the market. Future competitiveness of wind power generation will depend on the possibility of further reducing wind turbines (WTs) operation and maintenance (O&M) costs, which currently reach 20–25% of the total energy production cost [1,2]. For this reason, efforts are being devoted to the development and implementation of cost-efficient O&M policies for maximizing energy production while reducing maintenance costs [3,4]. WTs are equipped with Prognostics and Health Management (PHM) capabilities to assess the current health state of critical components and predict their Remaining Useful Life (RUL) based on condition monitoring data collected by sensors. RUL gives, in principle, the information needed for PdM, the implementation of PdM in real-world businesses is challenged by practical issues related to:

Objectives

Methods

Results

Conclusion