Research on real-time data processing technology for Internet of things

Abstract

In the age of internet of things, the storage and real-time processing requirements of massive amounts of real-time data generated by sensors, so that the traditional database or data processing architecture has been unable to deal with. Analyzing the processing characteristics of the real-time data of the things and studying the data processing technology of the Internet of things. A distributed storage and computing architecture for data partition and multi node distribution is proposed. The architecture can achieve linear expansion of the processing performance and storage size of the real-time data. For different real-time data processing scene, the design of different data partitions in different formats, on the one hand to ensure data replication redundancy, on the one hand to provide the optimal data query efficiency for different scenarios.