Proceedings
2013 Proceedings of the Fifteenth Workshop on Algorithm Engineering and Experiments (ALENEX)
Inducing Suffix and LCP Arrays in External Memory
Pages 88 - 102
Abstract
We consider text index construction in external memory (EM). Our first contribution is an inducing algorithm for suffix arrays in external memory. Practical tests show that this outperforms the previous best EM suffix sorter [Dementiev et al., ALENEX 2005] by a factor of about two in time and I/O-volume. Our second contribution is to augment the first algorithm to also construct the array of longest common prefixes (LCPs). This yields the first EM construction algorithm for LCP arrays. The overhead in time and I/O volume for this extended algorithm over plain suffix array construction is roughly two. Our algorithms scale far beyond problem sizes previously considered in the literature (text size of 80 GiB using only 4 GiB of RAM in our experiments).
Formats available
You can view the full content in the following formats:
Information & Authors
Information
Published In
2013 Proceedings of the Fifteenth Workshop on Algorithm Engineering and Experiments (ALENEX)
Pages: 88 - 102
Editors: Peter Sanders, Karlsruhe Institute of Technology, Karlsruhe, Germany and Norbert Zeh, Dalhousie University, Halifax, Nova Scotia, Canada
ISBN (Print): 978-1-611972-53-5
ISBN (Online): 978-1-61197-293-1
Copyright
© 2013 by the Society for Industrial and Applied Mathematics.
History
Published online: 18 December 2013
Authors
Metrics & Citations
Metrics
Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited By
- Building and Checking Suffix Array Simultaneously by Induced Sorting MethodIEEE Transactions on Computers, Vol. 71, No. 4 | 1 Apr 2022
- Scalable Suffix Sorting on a Multicore MachineIEEE Transactions on Computers, Vol. 69, No. 9 | 1 Sep 2020
- An Improved Algorithm for Building Suffix Array in External MemoryParallel Architectures, Algorithms and Programming | 26 January 2020
- Scalable Construction of Text Indexes with Thrill2018 IEEE International Conference on Big Data (Big Data) | 1 Dec 2018
- Engineering a Lightweight External Memory Suffix Array Construction AlgorithmMathematics in Computer Science, Vol. 11, No. 2 | 11 February 2017
- Scalable K-Order LCP Array Construction for Massive DataParallel Architecture, Algorithm and Programming | 6 October 2017
- TextsCompact Data Structures | 8 Sep 2016
- Induced Suffix Sorting for String Collections2016 Data Compression Conference (DCC) | 1 Mar 2016
- Low Space External Memory Construction of the Succinct Permuted Longest Common Prefix ArrayString Processing and Information Retrieval | 21 September 2016
- LCP Array Construction Using O(sort(n)) (or Less) I/OsString Processing and Information Retrieval | 21 September 2016
- Indexing Arbitrary-Length k-Mers in Sequencing ReadsPLOS ONE, Vol. 10, No. 7 | 16 July 2015
- Parallel External Memory Suffix SortingCombinatorial Pattern Matching | 16 June 2015
- A Bulk-Parallel Priority Queue in External Memory with STXXLExperimental Algorithms | 20 June 2015
- Longest Common Prefix with MismatchesString Processing and Information Retrieval | 5 September 2015
- Induced Sorting Suffixes in External Memory with Better Design and Less SpaceString Processing and Information Retrieval | 5 September 2015
- Fast Parallel Computation of Longest Common PrefixesSC14: International Conference for High Performance Computing, Networking, Storage and Analysis | 1 Nov 2014
- Large-Scale Pattern Search Using Reduced-Space On-Disk Suffix ArraysIEEE Transactions on Knowledge and Data Engineering, Vol. 26, No. 8 | 1 Aug 2014
- Lempel-Ziv Parsing in External Memory2014 Data Compression Conference | 1 Mar 2014
- LCP Array Construction in External MemoryExperimental Algorithms | 1 Jan 2014
- Space-Efficient Construction of the Burrows-Wheeler TransformString Processing and Information Retrieval | 1 Jan 2013
- External Memory Generalized Suffix and LCP Arrays ConstructionCombinatorial Pattern Matching | 1 Jan 2013
View Options
View options
- Access via your Institution
- Questions about how to access this content? Contact SIAM at [email protected].