Time-tired compaction: An elastic compaction scheme for LSM-tree based time-series database

Abstract

Time-series DBMSs based on the LSM-tree have been widely applied in numerous scenarios ranging from daily life to industrial production. Compared to the traditional key–value data, the time-series data workload has significant features of writing and querying in chronological order. While simultaneously, such features bring new challenges to efficient queries, especially data compaction. Namely, an effective time-series data compaction algorithm is crucial for efficient storage and query of massive time-series data. However, the current data compaction method based on traditional LSM-tree cannot solve the problems such as high delay and inaccurate range of time sequence data query. Therefore, we propose a novel compaction algorithm, Time-Tiered Compaction, to customize for time-series scenarios. Time-Tiered Compaction leverages the characteristics of time-series workloads to estimate query loads to select optimum SSTables for merging during every compaction process, reducing unnecessary expenses. Time-Tiered Compaction is implemented on Apache IoTDB to evaluate algorithm performance using TPC-xIoT. Results show that the presented Time-Tiered Compaction significantly reduces (30%) the latency of the range queries with only a slight increase (5%) in point queries, compared with traditional strategies.