Workload-optimized sensor data store for industrial IoT gateways

Abstract

In industrial Internet of Things (IoT) environments, sensor devices continue to generate a stream of sensor data, and the management of an ever-growing amount of data is a vital feature for IoT gateways. However, our preliminary analysis of some popular key–value stores used in IoT gateways revealed that none of the systems exploits the distinctive characteristics of IoT workloads – append-only and immutable – thus the systems show limitations in managing sensor data. To address this issue, in this study, we propose Indexing-of-Indexes (IoI) and LogFlush-and-Append (LFA) to exploit such characteristics. IoI is an indexing and data organization scheme designed to eliminate the notorious compaction-induced write amplification observed in legacy key–value stores, and LFA is a data ingestion scheme intended to remove double data write issues in legacy write-ahead logging implementations. We implement a prototype key–value store, Sen-Store, which incorporates our proposals, and we evaluate its performance using synthetic workloads and the TPCx-IoT benchmark. The evaluation results show that Sen-Store achieves up to 17.6× and 2.1× higher IoTps than industry-leading RocksDB and state-of-the-art IoTDB, respectively.