Prompt fine-tuning-based efficient federated foreign object detection of electric transmission line

Abstract

Foreign objects such as bird’s nests, honeycombs, kites, and plastics threaten the safety of electric transmission lines. Training a foreign object detection model with high generalization and accuracy needs to collect numerous data, which is usually unavailable because of data privacy and security. Federated learning is a promising solution but the total calculation amount is huge, and large number of gradient data will be transmitted in federated learning model updating. And because the data amount, training efficiency, and model accuracy are different at edge clients, it is not appropriate to aggregate clients’ edge models equally. To address the above mentioned problems, a prompt fine-tuning based efficient federated foreign object detection method of electric transmission line, which includes prompt federated fine-tuning mechanism and self-adaptive dynamic weighting aggregation strategy, is proposed in this paper. Prompt federated finetuning freezes the model backbone in edge client model training, and only the unfrozen parts of the model are transmitted and updated in the model aggregation phase. Self-adaptive dynamic weighting aggregation mechanism takes local data amount, edge model’s accuracy, and aggregation rounds into account when aggregating local models. Comprehensive experimental results demonstrate that the proposed method has not only better model accuracy but also faster training speed compared with other federated learning baselines on transmission line foreign object detection.