A new approach for multi-pattern string matching in large text corpora

Abstract

Multi-pattern string matching with large set of patterns is nowadays a key issue in various text retrieval applications. Filtering undesirable URLs, Finding quotes from famous holy books texts, extracting specific patterns from DNA sequences, Antivirus scanning, intrusion detection or even music retrieval are some applications of multi-pattern string matching. As the size of corpora and the number of patterns increase, the necessity for finding efficiently multiple patterns also increases. In this paper, a new approach for multiple pattern string matching is introduced. The proposed approach employs filtering useless parts of the text and indexing techniques based on finite state machines in order to achieve better performance. The proposed approach beside its efficiency has the advantage of being scalable and accurate even in noisy text corpora. Multiple experiments have been conducted on three real-life datasets, in order to evaluate the efficiency, flexibility and scalability of the proposed approach.