Data Mining Environment Research Articles

A Supporting Framework for Real-Time Data Mining

Computers widely used in the field of real time process control, and it's very necessary to pay more attention in the field. In this article, real-time data model, real-time data mining environment and the computing characteristics are analyzed .Based on investigating the development of a general purpose methodology for real-time data mining, we propose a novel supporting framework. The framework adopts a novel dynamic data mining process model.

Systematic and integrative analysis of large gene lists using DAVID bioinformatics resources

DAVID bioinformatics resources consists of an integrated biological knowledgebase and analytic tools aimed at systematically extracting biological meaning from large gene/protein lists. This protocol explains how to use DAVID, a high-throughput and integrated data-mining environment, to analyze gene lists derived from high-throughput genomic experiments. The procedure first requires uploading a gene list containing any number of common gene identifiers followed by analysis using one or more text and pathway-mining tools such as gene functional classification, functional annotation chart or clustering and functional annotation table. By following this protocol, investigators are able to gain an in-depth understanding of the biological themes in lists of genes that are enriched in genome-scale studies.

The European Glaucoma Society Glaucocard project: improved digital documentation of medical data for glaucoma patients based on standardized structured international datasets

To evaluate existing international IT-based ophthalmological medical data projects, and to define a glaucoma data set based on existing international standards of medical and ophthalmological documentation. To develop the technical environment for easy data mining and data exchange in different countries in Europe. Existing clinical and IT-based projects for documentation of medical data in general medicine and ophthalmology were analyzed to create new data sets for medical documentation in glaucoma patients. Different types of data transfer methods were evaluated to find the best method of data exchange between ophthalmologists in different European countries. Data sets from existing IT projects showed a wide variability in specifications, use of codes, terms and graphical data (perimetry, optic nerve analysis etc.) in glaucoma patients. New standardized digital datasets for glaucoma patients were defined, based on existing standards, which can be used by general ophthalmologists for follow-up examinations and for glaucoma specialists to perform teleconsultation, also across country borders. Datasets are available in different languages. Different types of data exchange methods using secure medical data transfer by internet, USB stick and smartcard were tested for different countries with regard to legal acceptance, practicability and technical realization (e.g. compatibility with EMR systems). By creating new standardized glaucoma specific cross-national datasets, it is now possible to develop an electronic glaucoma patient record system for data storage and transfer based on internet, smartcard or USB stick. The digital data can be used for referrals and for teleconsultation of glaucoma specialists in order to optimize glaucoma treatment. This should lead to an increase of quality in glaucoma care, and prevent expenses in health care costs by unnecessary repeated examinations.

BioMart and Bioconductor: a powerful link between biological databases and microarray data analysis

biomaRt is a new Bioconductor package that integrates BioMart data resources with data analysis software in Bioconductor. It can annotate a wide range of gene or gene product identifiers (e.g. Entrez-Gene and Affymetrix probe identifiers) with information such as gene symbol, chromosomal coordinates, Gene Ontology and OMIM annotation. Furthermore biomaRt enables retrieval of genomic sequences and single nucleotide polymorphism information, which can be used in data analysis. Fast and up-to-date data retrieval is possible as the package executes direct SQL queries to the BioMart databases (e.g. Ensembl). The biomaRt package provides a tight integration of large, public or locally installed BioMart databases with data analysis in Bioconductor creating a powerful environment for biological data mining.

Improving Classification Accuracy of Decision Trees for Different Abstraction Levels of Data

Classification is an important problem in data mining. Given a database of records, each tagged with a class label, a classifier generates a concise and meaningful description for each class that can be used to classify subsequent records. Since the data is collected from disparate sources in many actual data mining environments, it is common to have data values in different abstraction levels. This article introduces the multiple abstraction level problem in decision tree classification, and proposes a method to deal with it. The proposed method adopts the notion of fuzzy relation for solving the multiple abstraction level problem. The experimental results show that the proposed method reduces classification error rates significantly when multiple abstraction levels of data are involved.

GeneKeyDB: A lightweight, gene-centric, relational database to support data mining environments

BackgroundThe analysis of biological data is greatly enhanced by existing or emerging databases. Most existing databases, with few exceptions are not designed to easily support large scale computational analysis, but rather offer exclusively a web interface to the resource. We have recognized the growing need for a database which can be used successfully as a backend to computational analysis tools and pipelines. Such database should be sufficiently versatile to allow easy system integration.ResultsGeneKeyDB is a gene-centered relational database developed to enhance data mining in biological data sets. The system provides an underlying data layer for computational analysis tools and visualization tools. GeneKeyDB relies primarily on existing database identifiers derived from community databases (NCBI, GO, Ensembl, et al.) as well as the known relationships among those identifiers. It is a lightweight, portable, and extensible platform for integration with computational tools and analysis environments.ConclusionGeneKeyDB can enable analysis tools and users to manipulate the intersections, unions, and differences among different data sets.

A Visual Tool For Mining Macroeconomics Data

Data mining environments need tools capable of aiding results comprehension, in particular by resorting to the power of visual perception. This paper presents a tool designed and implemented in MatLabTM to facilitate visual mining of a large set of macroeconomics data on the world import and export activity of seven countries, in order to extend an analysis already performed with the aim of building models of national specialization and identifying possible market outlets. A model of the “market” is built, by using a Multilayer Perceptron; then it is fed to a graphical interface which supports queries such as: can country X expand towards country Y in one particular sector? Answer messages point out, for the selected countries, sectors in which the market could be expanded, and for what specific products. As an aid to making sense of a given suggestion, a detailed market analysis can be carried out by resorting to a set of tabular and graphical views that are able to map the productive structure of each country and also the dynamics of its evolution through the years. The advantages of MatLabTM as a development environment are pointed out. It is argued that the additional effort for developing ad-hoc task oriented interfaces is especially justified in the case of frequent and interactive mining to be performed by decision makers.

Stability and scalability in decision trees

Tree-based methods are statistical procedures for automatic learning from data, whose main applications are integrated into a data-mining environment for decision support systems. Here, we focus on two problems of decision trees: the stability of the rules obtained and their applicability to huge data sets. Since the tree-growing process is highly dependent on data, i.e. small fluctuations in data can cause big changes in the tree-growing process, we focused instead on the stability of the trees themselves. To this end we propose a series of data diagnostics to prevent internal instability in the tree-growing process before a particular split is made. Indeed, to be effective in actual managerial problems they must be applicable to massive amounts of stored data with maximum efficiency. For this reason we studied the theoretical complexity of such an algorithm. Finally, we present an algorithm that can cope with such problems, with linear cost upon the individuals, which can use a robust impurity measure as a splitting criterion.

An environment for knowledge discovery in biology

This paper describes a data mining environment for knowledge discovery in bioinformatics applications. The system has a generic kernel that implements the mining functions to be applied to input primary databases, with a warehouse architecture, of biomedical information. Both supervised and unsupervised classification can be implemented within the kernel and applied to data extracted from the primary database, with the results being suitably stored in a complex object database for knowledge discovery. The kernel also includes a specific high-performance library that allows designing and applying the mining functions in parallel machines. The experimental results obtained by the application of the kernel functions are reported.

다중 추상화 수준의 데이터를 위한 결정 트리 분류기

Since the data is collected from disparate sources in many actual data mining environments, it is common to have data values in different abstraction levels. This paper shows that such multiple abstraction levels of data can cause undesirable effects in decision tree classification. After explaining that equalizing abstraction levels by force cannot provide satisfactory solutions of this problem, it presents a method to utilize the data as it is. The proposed method accommodates the generalization/specialization relationship between data values in both of the construction and the class assignment phase of decision tree classification. The experimental results show that the proposed method reduces classification error rates significantly when multiple abstraction levels of data are involved.

協調ビジュアルデータマイニングのためのテレイマ-ジョン環境の構築

To effectively visualize huge volume datasets, we should consider the following view points; 1) A collaborative analysis of the datasets is required. 2) The datasets may not be stored on hand.3) The datasets may overflow into physical memory. These can cause a problem on interactive visualization. To solve the problem, we will initiate a project in which we plan to develop a teleimmersive environment for collaborative visual data mining.

Distributed Data Integration and Mining Using ADMIRE Technology

In this paper we present our work on the engine for integration of environmental data. We present a suite of selected environmental scenarios, which are integrated into a novel data mining and integration environment, being developed in the project ADMIRE . The scenarios have been chosen for their suitability for data mining by environmental experts. They deal with meteorological and hydrological problems, and apply the chosen solutions to pilot areas within Slovakia. The main challenge is that the environmental data required by scenarios are maintained and provided by different organizations and are often in different formats. We present our approach to the specification and execution of data integration tasks, which deals with the distributed nature and heterogeneity of required data resources.

Data Mining and Machine Oriented Modeling: A Granular Computing Approach

From the processing point of view, data mining is machine derivation of interesting properties (to human) from the stored data. Hence, the notion of machine oriented data modeling is explored: An attribute value, in a relational model, is a meaningful label (a property) of a set of entities (granule). A model using these granules themselves as attribute values (their bit patterns or lists of members) is called a machine oriented data model. The model provides a good database compaction and data mining environment. For moderate size databases, finding association rules, decision rules, and etc., can be reduced to easy computation of i>set theoretical operations of granules. In the second part, these notions are extended to real world objects, where the universe is granulated (clustered) into granules by binary relations. Data modeling and mining with such additional semantics are formulated and investigated. In such models, data mining is essentially a machine “calculus” of granules-granular computing.

Data analysis and mining environment: a distributed intelligent agent technology application

There is a tremendous explosion in the amount of data that organizations generate, collect and store. Managers are beginning to recognize the value of this asset, and are increasingly relying on intelligent systems to access, analyze, summarize, and interpret information from large and multiple data sources. These systems help them make critical business decisions faster or with a greater degree of confidence. Data mining is a promising new technology that helps bring business intelligence into these systems. While there is a plethora of data mining techniques and tools available, they present inherent problems for end‐users such as complexity, required technical expertise, lack of flexibility and interoperability, etc. These problems can be mitigated by deploying software agents to assist end‐users in their problem solving endeavors. This paper presents the design and development of an intelligent software agent based data analysis and mining environment called IDM, which is utilized in decision making activities.

Application of intelligent agent technology for managerial data analysis and mining

Data analysis and mining technologies help bring business intelligence into organizational decision support systems (DSS). While a myriad of data analysis and mining technologies are commercially available today, organizations are seeing a growing gap between powerful storage (data warehouse) systems and the business users' ability to analyze and act effectively on the information they contain. We contend that to narrow this gap effectively, a data analysis and mining environment is needed that can bring together and make available for use many of these technologies, that can support business users with different backgrounds, and with which the users can work comfortably.This paper illustrates the design and construction of such an environment, called the Intelligent Data Miner. IDM is Web-based and it is intended to provide an organization-wide decision support capability for business users. Intelligent agent technology is used as the basis for IDM design. IDM provides several types of data access capabilities to access and analyze the data contained in a data warehouse to obtain the critical information needed by business decision-makers. It supports both predefined and ad hoc data access, data analysis, data presentation, and data mining requests from non-technical users.An operational prototype of IDM, implemented using Java and JATLite (Java Agent Template, Lite from Stanford University), allowed us to examine the feasibility of having the "agents" automatically control and coordinate activities and tasks on the business users' behalf. These agents proved to hide the complexity of data analysis and mining activities, techniques, and methods from the business users, for effective use of the warehouse data.

Towards on-line analytical mining in large databases

Great efforts have been paid in the Intelligent Database Systems Research Lab for the research and development of efficient data mining methods and construction of on-line analytical data mining systems.Our work has been focused on the integration of data mining and OLAP technologies and the development of scalable, integrated, and multiple data mining functions. A data mining system, DBMiner, has been developed for interactive mining of multiple-level knowledge in large relational databases and data warehouses. The system implements a wide spectrum of data mining functions, including characterization, comparison, association, classification, prediction, and clustering. It also builds up a user-friendly, interactive data mining environment and a set of knowledge visualization tools. In-depth research has been performed on the efficiency and scalability of data mining methods. Moreover, the research has been extended to spatial data mining, multimedia data mining, text mining, and Web mining with several new data mining system prototypes constructed or under construction, including GeoMiner, MultiMediaMiner, and WebLogMiner.This article summarizes our research and development activities in the last several years and shares our experiences and lessons with the readers.

DBMiner

Based on our years-of-research, a data mining system, DB-Miner, has been developed for interactive mining of multiple-level knowledge in large relational databases. The system implements a wide spectrum of data mining functions, including generalization, characterization, association, classification, and prediction. By incorporation of several interesting data mining techniques, including attribute-oriented induction, progressive deepening for mining multiple-level rules, and meta-rule guided knowledge mining, the system provides a user-friendly, interactive data mining environment with good performance.