Abstract
It is well known that in-place update index, unordered log structured index and ordered log structured index are three typical data organizations which are designed to meet different workload requirements respectively and wildly used in big data storage systems. Differentiated workload requirements in different phase of the data lifecycle, e.g. various types of data are injected into the big data storage systems in the write optimized manner, then they are needed to be read in the read optimized manner for analysis, lead to data organization transformation(data transformation for short). However, the simple mixture of foreground data injection and background data transformation causes serious disk contention. Frequent disk head seeks result in low disk throughput, and not only prolong the data transformation process, but also increase foreground data injection latency. In this paper, we propose \emph{Workload Shifting}, a novel log- structured design that shifts background data transformation away from the foreground data injection. Compared with conventional RAID0 disk array, \emph{Workload Shifting} effectively isolates background data transformation and foreground data injections, avoids the disk contention between them to boost their performance. We have implemented \emph{Workload Shifting} prototype on one multiple disks based disk array. Extensive experimental evaluation results show that compared with conventional RAID0 disk arrays, \emph{Workload Shifting} can avoid disk contention and speed up both data injection and data transformation significantly.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.