Cloud Segmentation and Motion Tracking in Sky Images

Abstract

Cloud cover significantly affects the solar irradiance incident on a photovoltaic (PV) module, so identifying and predicting cloud motion is useful for PV applications, such as informing tracker movements and predicting short-term power. This work presents two algorithms to aid in real-time weather predictions, i.e., a convolutional autoencoder (CAE) to identify clouds and a particle tracker to predict cloud movement. The CAE model integrates information from multiple cloud segmentation approaches, and then utilizes transfer learning on these unreliable, automatically generated masks to bootstrap model performance. The presented model improves upon the state-of-the art metrics with a resultant pixelwise accuracy greater than 90% while remaining lightweight in number of samples used. For tracking and prediction of cloud movements, particle tracking is useful in areas where cloud coverage is transient and clouds move in smaller fragments. By combining neural networks and more classical image processing techniques, the system becomes more robust and explainable than image processing or pure neural network technologies alone, while also demonstrating the power of transfer learning techniques in application.