Scalable database logging for multicores

Abstract

Modern databases, guaranteeing atomicity and durability, store transaction logs in a volatile, central log buffer and then flush the log buffer to non-volatile storage by the write-ahead logging principle. Buffering logs in central log store has recently faced a severe multicore scalability problem, and log flushing has been challenged by synchronous I/O delay. We have designed and implemented a fast and scalable logging method, E leda , that can migrate a surge of transaction logs from volatile memory to stable storage without risking durable transaction atomicity. Our efficient implementation of E leda is enabled by a highly concurrent data structure, G rasshopper , that eliminates a multicore scalability problem of centralized logging and enhances system utilization in the presence of synchronous I/O delay. We implemented E leda and plugged it to WiredTiger and Shore-MT by replacing their log managers. Our evaluation showed that E leda -based transaction systems improve performance up to 71 x, thus showing the applicability of E leda.