Knowledge Discovery Tasks Research Articles

Mop: An Efficient Algorithm for Mining Frequent Pattern with Subtree Traversing

Mining frequent patterns in database has emerged as an important task in knowledge discovery and data mining. In this paper, we present an efficient algorithm called Mop for fast frequent pattern discovery. Mop utilizes a new kind of data structure called OP_tree (ordered pattern tree) and some particular properties of frequent patterns to facilitate the process of mining frequent patterns. An OP_tree is a special frequent pattern tree, where the children of any node are sorted according to the supports of corresponding items. Efficiency of Mop is achieved with three techniques: (1) it adopts OP_tree to store a large database to avoid repetitive database scans, (2) it finds all frequent 2-patterns in the construction of OP_tree to avoid the costly generation of a large number of candidate 2-patterns, (3) the supports of candidate k-patterns (k>2) can be obtained by traversing a few of specific subtrees of the OP_tree, which greatly reduces the search space and avoid multi-scans of a database. We experimentally compare our algorithm with the Apriori algorithm and the FP-growth algorithm on one real database and one synthetical database. The experimental results show that Mop is about an order of magnitude faster than the Apriori algorithm. Mop also outperforms the FP-growth algorithm, especially when support threshold is very low and databases are quite large.

Mining interesting subgraphs by output space sampling

Lack of scalability of the mining process and the enormous size of the output set are two significant bottlenecks of Frequent Subgraph Mining (FSM). The first restricts the applicability of FSM to large datasets. The second makes it difficult for the user to analyze the frequent patterns for subsequent usage in typical knowledge discovery tasks, such as classification, clustering, outlier detection, etc. In this thesis, I propose output space sampling (OSS) to alleviate the above two problems. In this paradigm, the objective is to sample frequent patterns instead of complete enumeration. The sampling process automatically performs the interestingness based selection by embedding the interestingness score of the patterns in the desired target distribution. This obviates a two-step mechanism since the sampling automatically prefers the patters that are interesting. Another important point to note is that OSS is a generic method that applies for any kind of patterns such as a set, a sequence, a tree and of-course a graph. OSS is based on Markov Chain Monte Carlo (MCMC) sampling. It performs a random walk on the candidate subgraph partial order and returns subgraph samples when the walk converges to a desired stationary distribution. The transition probability matrix of the random walk is computed locally to avoid a complete enumeration of the candidate frequent patterns, which makes the sampling paradigm scalable to large real life graph datasets. Output space sampling is an entire paradigm shift in frequent pattern mining (FPM) that holds enormous promise. While traditional FPM strives for completeness, OSS targets to obtain a few interesting samples. The definition of interestingness can be very generic, so user can sample patterns from different target distributions by choosing different interestingness functions. This is very beneficial as mined patterns are subject to subsequent use in various knowledge discovery tasks. Output space sampling is a general idea that has various applications. In this thesis, we utilize this general idea to solve specific problems. We consider two different problems: (1) frequent pattern summarization (2) sampling discriminatory patterns. For both the above problems, we assume that the direct mining task is infeasible, so that the user wants to adopt sampling to find few interesting patterns in a reasonable amount of time. We also use the concept of OSS to find representative patterns that are very different from each other. OSS naturally supports this requirement as it obtains random samples which are very different from each other. In this thesis, two different algorithms are proposed for representative pattern mining, which are introduced in the next two paragraphs. The first algorithm for the representative pattern mining that is proposed in this thesis is called MUSK. It is based on a uniform sampling of the output space. It obtains representative patterns by sampling uniformly from the pool of all frequent maximal patterns; uniformity is achieved by a variant of Markov Chain Monte Carlo (MCMC) algorithm. MUSK follows the concept of OSS by sampling from a target distribution where the maximal patterns have uniform value for the interestingness score. The second algorithm that we propose for this task is ORIGAMI . It defines the representative pattern-set (R) in a novel manner that attempts to reduce structural similarities among patterns in R while extending the coverage of frequent pattern space as much as possible. Intuitively, two patterns are α-orthogonal if their similarity is bounded above by α. Each α-orthogonal pattern is also a representative for those patterns that are at least β similar to it. Given user defined α, β ∈ [0, 1], the goal of O RIGAMI is to mine an a-orthogonal, β-representative set that minimizes the set of unrepresented patterns. Similar to OSS paradigm, O RIGAMI uses a randomized algorithm to randomly traverse the pattern space, seeking previously unexplored regions, to return a set of maximal patterns. But, uncharacteristic to OSS, ORIGAMI employs a second-step to extract an α-orthogonal, β-representative set from the mined maximal patterns using a local optimal algorithm. The second step is essential to provide the α-orthogonal, β-representative guarantee. (Abstract shortened by UMI.)

Efficient algorithms for the mining of constrained frequent patterns from uncertain data

Mining of frequent patterns is one of the popular knowledge discovery and data mining (KDD) tasks. It also plays an essential role in the mining of many other patterns such as correlation, sequences, and association rules. Hence, it has been the subject of numerous studies since its introduction. Most of these studies find all the frequent patterns from collection of precise data, in which the items within each datum or transaction are definitely known and precise. However, there are many real-life situations in which the user is interested in only some tiny portions of these frequent patterns. Finding all frequent patterns would then be redundant and waste lots of computation. This calls for constrained mining , which aims to find only those frequent patterns that are interesting to the user. Moreover, there are also many real-life situations in which the data are uncertain. This calls for uncertain data mining . In this article, we propose algorithms to efficiently find constrained frequent patterns from collections of uncertain data .

FpVAT

As frequent pattern mining plays an essential role in many knowledge discovery and data mining (KDD) tasks, numerous algorithms for finding frequent patterns have been proposed over the past 15 years. However, most of these algorithms return the mining results in the form of textual lists containing frequent patterns showing those frequently occurring sets of items. It is well known that "a picture is worth a thousand words". The use of visual representation can enhance the user's understanding of the inherent relations in a collection of frequent patterns. In this paper, we develop a simple yet useful visual analytic tool for supporting frequent pattern mining called FpVAT . Such a visual analytic tool consists of two modules: One module gives users an overview so that they can derive insight from a massive amount of raw data; another module enables users to perform analytical reasoning on the mining results via interactive visual interfaces so that users can detect the expected frequent patterns and discover the unexpected frequent patterns. As a visual analytic tool, our FpVAT is equipped with several interactive features for effective visual support in the data analysis and KDD process for various real-life applications.

Event-based lossy compression for effective and efficient OLAP over data streams

An innovative event-based lossy compression model for effective and efficient OLAP over data streams, called ECM-DS, is presented and experimentally assessed in this paper. The main novelty of our compression approach with respect to traditional data stream compression techniques relies on exploiting the semantics of the reference application scenario in order to drive the compression process by means of the “degree of interestingness” of events occurring in the target stream. This finally improves the quality of retrieved approximate answers to OLAP queries over data streams, and, in turn, the quality of complex knowledge discovery tasks over data streams developed on top of ECM-DS, and implemented via ad-hoc data stream mining algorithms. Overall, the compression strategy we propose in this research puts the basis for a novel class of intelligent applications over data streams where the knowledge on actual streams is integrated-with and correlated-to the knowledge related to expired events that are considered critical for the target OLAP analysis scenario. Finally, a comprehensive experimental evaluation over several classes of data stream sets clearly confirms the benefits deriving from the event-based data stream compression approach proposed in ECM-DS.

Development of traditional Chinese medicine clinical data warehouse for medical knowledge discovery and decision support

Traditional Chinese medicine (TCM) is a scientific discipline, which develops the related theories from the long-term clinical practices. The large-scale clinical data are the core empirical knowledge source for TCM research. This paper introduces a clinical data warehouse (CDW) system, which incorporates the structured electronic medical record (SEMR) data for medical knowledge discovery and TCM clinical decision support (CDS). We have developed the clinical reference information model (RIM) and physical data model to manage the various information entities and their relationships in TCM clinical data. An extraction-transformation-loading (ETL) tool is implemented to integrate and normalize the clinical data from different operational data sources. The CDW includes online analytical processing (OLAP) and complex network analysis (CNA) components to explore the various clinical relationships. Furthermore, the data mining and CNA methods are used to discover the valuable clinical knowledge from the data. The CDW has integrated 20,000 TCM inpatient data and 20,000 outpatient data, which contains manifestations (e.g. symptoms, physical examinations and laboratory test results), diagnoses and prescriptions as the main information components. We propose a practical solution to accomplish the large-scale clinical data integration and preprocessing tasks. Meanwhile, we have developed over 400 OLAP reports to enable the multidimensional analysis of clinical data and the case-based CDS. We have successfully conducted several interesting data mining applications. Particularly, we use various classification methods, namely support vector machine, decision tree and Bayesian network, to discover the knowledge of syndrome differentiation. Furthermore, we have applied association rule and CNA to extract the useful acupuncture point and herb combination patterns from the clinical prescriptions. A CDW system consisting of TCM clinical RIM, ETL, OLAP and data mining as the core components has been developed to facilitate the tasks of TCM knowledge discovery and CDS. We have conducted several OLAP and data mining tasks to explore the empirical knowledge from the TCM clinical data. The CDW platform would be a promising infrastructure to make full use of the TCM clinical data for scientific hypothesis generation, and promote the development of TCM from individualized empirical knowledge to large-scale evidence-based medicine.

Cluster identification and separation in the growing self-organizing map: application in protein sequence classification

Growing self-organizing map (GSOM) has been introduced as an improvement to the self-organizing map (SOM) algorithm in clustering and knowledge discovery. Unlike the traditional SOM, GSOM has a dynamic structure which allows nodes to grow reflecting the knowledge discovered from the input data as learning progresses. The spread factor parameter (SF) in GSOM can be utilized to control the spread of the map, thus giving an analyst a flexibility to examine the clusters at different granularities. Although GSOM has been applied in various areas and has been proven effective in knowledge discovery tasks, no comprehensive study has been done on the effect of the spread factor parameter value to the cluster formation and separation. Therefore, the aim of this paper is to investigate the effect of the spread factor value towards cluster separation in the GSOM. We used simple k-means algorithm as a method to identify clusters in the GSOM. By using Davies–Bouldin index, clusters formed by different values of spread factor are obtained and the resulting clusters are analyzed. In this work, we show that clusters can be more separated when the spread factor value is increased. Hierarchical clusters can then be constructed by mapping the GSOM clusters at different spread factor values.

On subgroup discovery in numerical domains

Subgroup discovery is a Knowledge Discovery task that aims at finding subgroups of a population with high generality and distributional unusualness. While several subgroup discovery algorithms have been presented in the past, they focus on databases with nominal attributes or make use of discretization to get rid of the numerical attributes. In this paper, we illustrate why the replacement of numerical attributes by nominal attributes can result in suboptimal results. Thereafter, we present a new subgroup discovery algorithm that prunes large parts of the search space by exploiting bounds between related numerical subgroup descriptions. The same algorithm can also be applied to ordinal attributes. In an experimental section, we show that the use of our new pruning scheme results in a huge performance gain when more that just a few split-points are considered for the numerical attributes.

MaSDA: A system for analyzing mass spectrometry data

Mass spectrometry (MS) approaches have been recently coupled with advanced data analysis techniques in order to enable clinicians to discover useful knowledge from MS data. However, effectively and efficiently handling and analyzing MS data requires to take into account a number of issues. In particular, the huge dimensionality and the variety of noisy factors present in MS data require careful preprocessing and modeling phases in order to make them amenable to the further analysis. In this paper we present MaSDA, a system performing advanced analysis on MS data. MaSDA has the following main features: (i) it implements an approach of MS data representation that exploits a model based on low dimensional, dense time series; (ii) it provides a wide set of MS preprocessing operations which are accomplished by means of a user-friendly graphical tool; (iii) it embeds a number of tools implementing various tasks of data mining and knowledge discovery, in order to assist the user in taking critical clinical decisions. Our system has been experimentally tested on several publicly available datasets, showing effectiveness and efficiency in supporting advanced analysis of MS data.

A hybrid coevolutionary algorithm for designing fuzzy classifiers

Rule learning is one of the most common tasks in knowledge discovery. In this paper, we investigate the induction of fuzzy classification rules for data mining purposes, and propose a hybrid genetic algorithm for learning approximate fuzzy rules. A novel niching method is employed to promote coevolution within the population, which enables the algorithm to discover multiple rules by means of a coevolutionary scheme in a single run. In order to improve the quality of the learned rules, a local search method was devised to perform fine-tuning on the offspring generated by genetic operators in each generation. After the GA terminates, a fuzzy classifier is built by extracting a rule set from the final population. The proposed algorithm was tested on datasets from the UCI repository, and the experimental results verify its validity in learning rule sets and comparative advantage over conventional methods.

Band selection for hyperspectral imagery using affinity propagation

Hyperspectral imagery generally contains enormous amounts of data because of hundreds of spectral bands. Band selection is often adopted to reduce computational cost and accelerate knowledge discovery and other tasks such as subsequent classification. An exemplar-based clustering algorithm termed affinity propagation for band selection is proposed. Affinity propagation is derived from factor graph, and operates by initially considering all data points as potential cluster centres (exemplars) and then exchanging messages between data points until a good set of exemplars and clusters emerges. Affinity propagation has been applied to computer vision and bioinformatics, and shown to be much faster than other clustering methods for large data. By combining the information about the discriminative capability of each individual band and the correlation/similarity between bands, the exemplars generated by affine propagation have higher importance and less correlation/similarity. The performance of band selection is evaluated through a pixel image classification task. Experimental results demonstrate that, compared with some popular band selection methods, the bands selected by affinity propagation best characterise the hyperspectral imagery from the pixel classification standpoint.

Bridging Data Management and Knowledge Discovery in the Life Sciences

In this work we present an application for integrating and analyzing life science data using a biomedical data warehouse system and tools developed in-house enabling knowledge discovery tasks. Knowledge discovery is known as a process where different steps have to be coupled in order to solve a specified question. In order to create such a combination of steps, a data miner using our in-house developed knowledge discovery tool KD3 is able to assemble functional objects to a data mining workflow. The generated workflows can easily be used for ulterior purposes by only adding new data and parameterizing the functional objects in the process. Workflows guide the performance of data integration and aggregation tasks, which were defined and implemented using a public available open source tool. To prove the concept of our application, intelligent query models were designed and tested for the identification of genotype-phenotype correlations in Marfan Syndrome. It could be shown that by using our application, a data miner can easily develop new knowledge discovery algorithms that may later be used to retrieve medical relevant information by clinical researchers.

Sentence recognition using artificial neural networks

The paper describes an application of artificial neural networks (ANN) for natural language text reasoning. The task of knowledge discovery in text from a database, represented with a database file consisting of sentences with similar meanings but different lexico-grammatical patterns, was solved with ANNs which recognize the meaning of the text using training files with limited dictionary. The paper features recognition algorithms of text meaning from a selected source using 3-layer ANNs. Tests of the new method have also been described.

CSMC: A combination strategy for multi-class classification based on multiple association rules

Constructing accurate classifier based on association rules is an important and challenging task in data mining and knowledge discovery. In this paper, a novel combination strategy for multi-class classification (CSMC) based on multiple rules is proposed. In CSMC, rules are regarded as classification experts, after the calculation of the basic probability assignments (bpa) and evidence weights, Yang’s rule of combination is employed to combine the distinct evidence bodies to realize an aggregate classification. A numerical example is shown to highlight the procedure of the proposed method at the end of this paper. The comparison with popular methods like CBA, C4.5, RIPPER and MCAR indicates that CSMC is a competitive method for classification based on association rule.

Layered critical values: a powerful direct-adjustment approach to discovering significant patterns

Standard pattern discovery techniques, such as association rules, suffer an extreme risk of finding very large numbers of spurious patterns for many knowledge discovery tasks. The direct-adjustment approach to controlling this risk applies a statistical test during the discovery process, using a critical value adjusted to take account of the size of the search space. However, a problem with the direct-adjustment strategy is that it may discard numerous true patterns. This paper investigates the assignment of different critical values to different areas of the search space as an approach to alleviating this problem, using a variant of a technique originally developed for other purposes. This approach is shown to be effective at increasing the number of discoveries while still maintaining strict control over the risk of false discoveries.

Extraction of activity patterns on large video recordings

Extracting the hidden and useful knowledge embedded within video sequences and thereby discovering relations between the various elements to help an efficient decision-making process is a challenging task. The task of knowledge discovery and information analysis is possible because of recent advancements in object detection and tracking. The authors present how video information is processed with the ultimate aim to achieve knowledge discovery of people activity and also extract the relationship between the people and contextual objects in the scene. First, the object of interest and its semantic characteristics are derived in real-time. The semantic information related to the objects is represented in a suitable format for knowledge discovery. Next, two clustering processes are applied to derive the knowledge from the video data. Agglomerative hierarchical clustering is used to find the main trajectory patterns of people and relational analysis clustering is employed to extract the relationship between people, contextual objects and events. Finally, the authors evaluate the proposed activity extraction model using real video sequences from underground metro networks (CARETAKER) and a building hall (CAVIAR).

Support-Less Association Rule Mining Using Tuple Count Cube

Association rule mining is one of the important tasks in data mining and knowledge discovery (KDD). The traditional task of association rule mining is to find all the rules with high support and high confidence. In some applications, we are interested in finding high confidence rules even though the support may be low. This type of problem differs from the traditional association rule mining problem; hence, it is called support-less association rule mining. Existing algorithms for association rule mining, such as the Apriori algorithm, cannot be used efficiently for support-less association rule mining since those algorithms mostly rely on identifying frequent item-sets with high support. In this paper, we propose a new model to perform support-less association rule mining, i.e., to derive high confidence rules regardless of their support level. A vertical data structure, the Peano Count Tree (P-tree), is used in our model to represent all the information we need. Based on the P-tree structure, we build a special data cube, called the Tuple Count Cube (T-cube), to derive high confidence rules. Data cube operations, such as roll-up, on T-cube, provide efficient ways to calculate the count information needed for support-less association rule mining.

Managing discoveries in the visual analytics process

Visualization systems traditionally focus on graphical representation of information. They tend not to provide integrated analytical services that could aid users in tackling complex knowledge discovery tasks. Users' exploration in such environments is usually impeded due to several problems: 1) valuable information is hard to discover when too much data is visualized on the screen; 2) Users have to manage and organize their discoveries off line, because no systematic discovery management mechanism exists; 3) their discoveries based on visual exploration alone may lack accuracy; and 4)they have no convenient access to the important knowledge learned by other users. To tackle these problems, it has been recognized that analytical tools must be introduced into visualization systems. In this paper, we present a novel analysis-guided exploration system, called the Nugget Management System (NMS). It leverages the collaborative effort of human comprehensibility and machine computations to facilitate users' visual exploration processes. Specifically, NMS first helps users extract the valuable information (nuggets) hidden in datasets based on their interests. Given that similar nuggets may be rediscovered by different users, NMS consolidates the nugget candidate set by clustering based on their semantic similarity. To solve the problem of inaccurate discoveries, localized data mining techniques are applied to refine the nuggets to best represent the captured patterns in datasets. Visualization techniques are then employed to present our collected nugget pool and thus create the nugget view. Based on the nugget view, interaction techniques are designed to help users observe and organize the nuggets in a more intuitive manner and eventually faciliate their sense-making process. We integrated NMS into XmdvTool, a freeware multivariate visualization system. User studies were performed to compare the users' efficiency and accuracy in finishing tasks on real datasets, with and without the help of NMS. Our user studies confirmed the effectiveness of NMS.

Towards mining past content of Web pages

While much attention has recently focused on preserving the past content of the Web, there is still a lack of efficient tools for utilizing data stored in Web archives. Web archives constitute large data sources that could be extensively analysed and mined for knowledge discovery. In this paper, we describe the issues involved with mining Web archive data. We discuss several concepts related to collecting and analysing historical content of Web pages and briefly describe two knowledge discovery tasks—temporal summarization and object history detection.

Discovering Both Positive and Negative Fuzzy Association Rules in Large Transaction Databases

Mining association rules is an important task of dara mining and knowledge discovery. Traditional association rules mining is built on transaction databases, which has some limitations. Two of these limitations are 1) each transaction merely contains binary items, meaning that an item either occurs in a transaction or not; 2) only positive association rules are discovered, while negative associations are ignored. Mining fuzzy association rules has been proposed to address the first limitation, while mining algorithms for negative association rules have been developed to resolve the second limitation. In this paper, we combine these two approaches to propose a novel approach for mining both positive and negative fuzzy association rules. The interestingness measure for both positive and negative fuzzy association rule is proposed, the algorithm for mining these rules is described, and an illustrative example is presented to demonstrate how the measure and the algorithm work.