Evaluation of selected technologies for the implementation of meter data management system

Abstract

Implementation of an efficient meter data management system presents a challenging task since it has to process and store a large number of incoming time-series data entries in almost real-time. A number of solutions for efficient processing and storage of big data streams are today available as open source or commercial software. However, the choice of the most applicable solution highly depends on the requirements of a specific use case scenario since the performance of the aforementioned solutions often vary depending on the specific use-case parameters (e.g. incoming data frequency, average size of a single data entry, etc.). Thus, in this paper we examine different platforms adequate for the implementation of a smart metering data acquisition system, to identify the most efficient ones among the considered candidates. The most important requirement of our meter data management system is to offer a stable solution that processes and stores high volumes of continuously incoming data readings with minimal loss-rate. For this purpose we propose a modular solution where components communicate over a message-queuing system, while the ultimate data repository is a NoSQL database. After carrying out all the specifically designed performance tests, we identify the following platforms as the most promising ones to implement our smart metering solution: Kafka as the messaging broker and time-series database InfluxDB. Finally, we verified that our MDMS successfully processes and stores 2.5 M data entries in a time period under eight minutes which confirms its targeted performance efficiency.