Discovering Patterns from Large and Dynamic Sequential Data

Abstract

Most daily and scientific data are sequential in nature. Discovering important patterns from such data can benefit the user and scientist by predicting coming activities, interpreting recurring phenomena, extracting outstanding similarities and differences for close attention, compressing data, and detecting intrusion. We consider the following incremental discovery problem for large and dynamic sequential data. Suppose that patterns were previously discovered and materialized. An update is made to the sequential database. An incremental discovery will take advantage of discovered patterns and compute only the change by accessing the affected part of the database and data structures. In addition to patterns, the statistics and position information of patterns need to be updated to allow further analysis and processing on patterns. We present an efficient algorithm for the incremental discovery problem. The algorithm is applied to sequential data that honors several sequential patterns modeling weather changes in Singapore. The algorithm finds what it is supposed to find. Experiments show that for small updates and large databases, the incremental discovery algorithm runs in time independent of the data size.