Free access
Proceedings
2017 Proceedings of the Ninteenth Workshop on Algorithm Engineering and Experiments (ALENEX)

CSA++: Fast Pattern Search for Large Alphabets

Abstract

Indexed pattern search in text has been studied for many decades. For small alphabets, the FM-Index provides unmatched performance for Count operations, in terms of both space required and search speed. For large alphabets – for example, when the tokens are words – the situation is more complex, and FM-Index representations are compact, but potentially slow. In this paper we apply recent innovations from the field of inverted indexing and document retrieval to compressed pattern search, including for alphabets into the millions. Commencing with the practical compressed suffix array structure developed by Sadakane, we show that the Elias-Fano code-based approach to document indexing can be adapted to provide new trade-off options in indexed pattern search, and offers significantly faster pattern processing compared to previous implementations, as well as reduced space requirements. We report a detailed experimental evaluation that demonstrates the relative advantages of the new approach, using the standard Pizza&Chili methodology and files, as well as applied use-cases derived from large-scale data compression, and from natural language processing. For large alphabets, the new structure gives rise to space requirements that are close to those of the most highly-compressed FM-Index variants, in conjunction with unparalleled Count throughput rates.

Formats available

You can view the full content in the following formats:

Information & Authors

Information

Published In

cover image Proceedings
2017 Proceedings of the Ninteenth Workshop on Algorithm Engineering and Experiments (ALENEX)
Pages: 73 - 82
Editors: Sándor Fekete, TU, Braunschwieg, Germany and Vijaya Ramachandran, UT, Austin, Texas, USA
ISBN (Online): 978-1-61197-476-8

History

Published online: 4 January 2017

Authors

Affiliations

Notes

*
This work was supported under the Australian Research Council's Discovery Projects funding scheme (project number DP140103256).

Metrics & Citations

Metrics

Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

Cited By

There are no citations for this item

View Options

View options

PDF

View PDF

Get Access

Media

Figures

Other

Tables

Share

Share

Copy the content Link

Share with email

Email a colleague

Share on social media

On May 28, 2024, our site will enter Read Only mode for a limited time in order to complete a platform upgrade. As a result, the following functions will be temporarily unavailable: registering new user accounts, any updates to existing user accounts, access token activations, and shopping cart transactions. Contact [email protected] with any questions.