Developing a deep learning-based layer-3 solution for thermal infrared large-scale photovoltaic module inspection from orthorectified big UAV imagery data

Abstract

The increasing adoption of photovoltaic(PV) technology highlights the need for efficient and large-scale deployment-ready inspection solutions. In the thermal infrared imagery-based inspection framework, we develop a robust and versatile deep learning model for the classification of defect-related patterns on PV modules. The model is developed from big UAV imagery data, and designed as a layer-3 building block that can be implemented on top of any two-stage PV inspection workflow comprising: (1)An aerial Structure from Motion– MultiView Stereo (SfM-MVS) photogrammetric acquisition/processing stage, at which a georeferenced thermal orthomosaic of an inspected PV site is generated, and which enables to locate precisely defective modules on field; then (2)an instance segmentation stage that extracts the images of modules. Orthomosaics from 28 different PV sites were produced, comprising 93220 modules with various types, layouts and thermal patterns. Modules were extracted through a developed semi-automatic workflow, then labeled into six classes. Data augmentation and balancing techniques were used to prepare a highly representative and balanced deep learning-ready dataset. The dataset was used to train, cross-validate and test the developed classifier, as well as benchmarking with the VGG16 architecture. The developed model achieves the state-of-art performance and versatility on the addressed classification problem, with a mean F1-score of94.52%. The proposed three-layer solution resolves the issues of conventional imagery-based workflows. It ensures highly accurate and versatile defect detection, and can be efficiently deployed to real-world large-scale applications.