The Performance Survey of in Memory Database

Abstract

To satisfy the ever-increasing performance demand of Big Data and critical applications the data management needs to offer the flexible schema, high availability, light weight replica, high volume and high scalability features so as to facilitate the transaction. The in memory database (IMDB) eliminates the I/O bottleneck by storing data in main memory. We give a deeper analysis of current main-stream IMDB systems performance which focuses on the data structure, architecture, volume, concurrency, availability and scalability. The V3 performance model is proposed to evaluate the Velocity, Volume and Varity of the 19 IMDB systems, in order to highlight the candidates with realtime transaction and high volume processing capacity coordinately. Test results clearly demonstrate that NewSQL is better at dealing with high-frequency trading models. To fully utilize the advantages of the multi-core and many-core processors capability improvements, a three-level optimization design strategy, which includes the memory-access level, the kernel-speedup level and the data-partition level also be proposed using the hardware parallelism for achieving task-level and data-level parallelism of IMDB programs, guarantees the IMDB could accelerate the real-time transaction in an efficient way. We believe that IMDB should become a compulsive option for enterprise users.