Inducing Suffix and LCP Arrays in External Memory

Abstract

Previous chapter Next chapter Full AccessProceedings 2013 Proceedings of the Meeting on Algorithm Engineering and Experiments (ALENEX)Inducing Suffix and LCP Arrays in External MemoryTimo Bingmann, Johannes Fischer, and Vitaly OsipovTimo Bingmann, Johannes Fischer, and Vitaly Osipovpp.88 - 102Chapter DOI:https://doi.org/10.1137/1.9781611972931.8PDFBibTexSections ToolsAdd to favoritesExport CitationTrack CitationsEmail SectionsAboutAbstract We consider text index construction in external memory (EM). Our first contribution is an inducing algorithm for suffix arrays in external memory. Practical tests show that this outperforms the previous best EM suffix sorter [Dementiev et al., ALENEX 2005] by a factor of about two in time and I/O-volume. Our second contribution is to augment the first algorithm to also construct the array of longest common prefixes (LCPs). This yields the first EM construction algorithm for LCP arrays. The overhead in time and I/O volume for this extended algorithm over plain suffix array construction is roughly two. Our algorithms scale far beyond problem sizes previously considered in the literature (text size of 80 GiB using only 4 GiB of RAM in our experiments). Previous chapter Next chapter RelatedDetails Published:2013ISBN:978-1-61197-253-5eISBN:978-1-61197-293-1 https://doi.org/10.1137/1.9781611972931Book Series Name:ProceedingsBook Code:PRAL13Book Pages:1-182