Transformer for Nonintrusive Load Monitoring: Complexity Reduction and Transferability

Abstract

Nonintrusive load monitoring (NILM) is to obtain individual appliance’s electricity consumption from aggregated smart meter data. In this article, we propose a middle window transformer model, termed <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Midformer</i> , for NILM. Existing models are limited by high computational complexity, dependency on data, and poor transferability. In Midformer, we first exploit patchwise embedding to shorten the input length, and then reduce the size of queries in the attention layer by only using global attention on a few selected input locations at the center of the window to capture the global context. The cyclically shifted window technique is used to preserve connection across patches. We also follow the pretraining and fine-tuning paradigm to relieve the dependency on data, reduce the computation in modeling training, and enhance transferability of the model to unknown tasks and domains. Our experimental study using two real-world data sets demonstrates the superior performance and transferability of Midformer over three baseline models.