Data Mining Environment Research Articles

NFC-1 Total Environment for Text Data Mining(TETDM)(Challenge for Realizing Early Profits, The 29th Annual Conference of the Japanese Society for Artificial Intelligence)

NFC-1 Total Environment for Text Data Mining(TETDM)(Challenge for Realizing Early Profits, The 29th Annual Conference of the Japanese Society for Artificial Intelligence)

Digging deep into weighted patient data through multiple-level patterns

Large data volumes have been collected by healthcare organizations at an unprecedented rate. Today both physicians and healthcare system managers are very interested in extracting value from such data. Nevertheless, the increasing data complexity and heterogeneity prompts the need for new efficient and effective data mining approaches to analyzing large patient datasets. Generalized association rule mining algorithms can be exploited to automatically extract hidden multiple-level associations among patient data items (e.g., examinations, drugs) from large datasets equipped with taxonomies. However, in current approaches all data items are assumed to be equally relevant within each transaction, even if this assumption is rarely true.This paper presents a new data mining environment targeted to patient data analysis. It tackles the issue of extracting generalized rules from weighted patient data, where items may weight differently according to their importance within each transaction. To this aim, it proposes a novel type of association rule, namely the Weighted Generalized Association Rule (W-GAR). The usefulness of the proposed pattern has been evaluated on real patient datasets equipped with a taxonomy built over examinations and drugs. The achieved results demonstrate the effectiveness of the proposed approach in mining interesting and actionable knowledge in a real medical care scenario.

IJARCCE - Computer and Communication Engineering

Driven by the need to control the content chaos that pervades local drives, file shares, email systems and document stores, organizations large and small are looking to impose order through Content Management. There are two types of content - structured and unstructured. Unstructured content includes: email, PowerPoint presentations, images, videos, audio recordings, documents, records and other files. ECM focuses on unstructured content to make it structured. Data mining is a tool used to extract important information from existing data and enable better decision- making throughout the banking and retail industries. They use data warehousing to combine various data from databases into an acceptable format so that the data can be mined. The data is then analysed and the information that is captured is used throughout the organization to support decision-making. Data Mining techniques can be very helpful to the banks for better targeting and acquiring new customers, fraud detection in real time, providing segment based products, analysis of the customers purchase patterns over time for better retention and relationship. Those banks that have realized the usefulness of data mining and are in the process of building a data mining environment for their decision-making process will obtain huge benefit and derive considerable competitive advantage in future.

The Data Mining OPtimization Ontology

The Data Mining OPtimization Ontology (DMOP) has been developed to support informed decision-making at various choice points of the data mining process. The ontology can be used by data miners and deployed in ontology-driven information systems. The primary purpose for which DMOP has been developed is the automation of algorithm and model selection through semantic meta-mining that makes use of an ontology-based meta-analysis of complete data mining processes in view of extracting patterns associated with mining performance. To this end, DMOP contains detailed descriptions of data mining tasks (e.g., learning, feature selection), data, algorithms, hypotheses such as mined models or patterns, and workflows. A development methodology was used for DMOP, including items such as competency questions and foundational ontology reuse. Several non-trivial modeling problems were encountered and due to the complexity of the data mining details, the ontology requires the use of the OWL 2 DL profile. DMOP was successfully evaluated for semantic meta-mining and used in constructing the Intelligent Discovery Assistant, deployed at the popular data mining environment RapidMiner.

The Data Mining OPtimization Ontology

The Data Mining OPtimization Ontology (DMOP) has been developed to support informed decision-making at various choice points of the data mining process. The ontology can be used by data miners and deployed in ontology-driven information systems. The primary purpose for which DMOP has been developed is the automation of algorithm and model selection through semantic meta-mining that makes use of an ontology-based meta-analysis of complete data mining processes in view of extracting patterns associated with mining performance. To this end, DMOP contains detailed descriptions of data mining tasks (e.g., learning, feature selection), data, algorithms, hypotheses such as mined models or patterns, and workflows. A development methodology was used for DMOP, including items such as competency questions and foundational ontology reuse. Several non-trivial modeling problems were encountered and due to the complexity of the data mining details, the ontology requires the use of the OWL 2 DL profile. DMOP was successfully evaluated for semantic meta-mining and used in constructing the Intelligent Discovery Assistant, deployed at the popular data mining environment RapidMiner.

범죄예측에서의 데이터마이닝 적용 가능성 연구 : 절도범죄를 중심으로

최근, 범죄가 증가함에 따라 범죄를 예측하고 예방하는 것은 사회의 중요한 이슈이며 정부 및 지자체는 다양한 방법론을 활용하여 범죄를 사전에 막고자 노력하고 있다. 데이터마이닝은 범죄예측 및 예방에 활용되는 대표적인 방법론이며, 범죄 패턴 분석, 범죄 발생 예측 등 다양한 분야에서 연구되고 있다. 그러나 데이터마이닝의 결과가 범죄학에서의 범죄 환경요소와 어떤 관련이 있는지 혹은, 사건해결에 어떤 도움을 줄 수 있는지에 대한 연구는 이루어지고 있지 않다. 따라서 본 논문에서는, 범죄학에서 범죄의 발생과 범죄 환경요소들의 상호 관련성을 보이고 범죄 발생과 관련된 환경요소와 데이터마이닝에 활용되는 변수 간의 관계를 정의하고자 하였다. 또한, 국내 보호관찰소에서 보관되고 있는 절도범죄 데이터를 사용하여 실제로 데이터마이닝의 결과가 범죄 환경요소와 어떤 관련이 있는지를 보이기 위해 군집분석을 적용하였다. 그 결과 각 군집별로 범죄가 발생하는 환경에 차이가 있었으며, 이를 활용하여 데이터마이닝이 범죄학관점에서 범죄 예측 및 예방 활용에 유의미함을 보였다. Recently, crime prediction and prevention are the most important social issues, and global and local governments have tried to prevent crime using various methodologies. One of the methodologies, data mining can be applied at various crime fields such as crime pattern analysis, crime prediction, etc. However, there is few researches to find the relationships between the results of data mining and crime components in terms of criminology. In this study, we introduced environmental criminology, and identified relationships between environment factors related with crime and variables using at data mining. Then, using real burglary data occurred in South Korea, we applied clustering to show relations of results of data mining and crime environment factors. As a result, there were differences in the crime environment caused by each cluster. Finally, we showed the meaning of data mining use at crime prediction and prevention area in terms of criminology.

MyChEMBL: A Virtual Platform for Distributing Cheminformatics Tools and Open Data

MyChEMBL is an open virtual platform which provides a free, secure, standardised and easy to use chemoinformatics environment for bioactivity data mining, machine learning, application development, learning and teaching. The main technical features of myChEMBL along with its applications and future plans are discussed here.

DAMEWARE: A Web Cyberinfrastructure for Astrophysical Data Mining

Astronomy is undergoing a methodological revolution triggered by an unprecedented wealth of complex and accurate data. The new panchromatic, synoptic sky surveys require advanced tools for discovering patterns and trends hidden behind data which are both complex and of high dimensionality. We present DAMEWARE (DAta Mining & Exploration Web Application REsource): a general purpose, web-based, distributed data mining environment developed for the exploration of large data sets, and finely tuned for astronomical applications. By means of graphical user interfaces, it allows the user to perform classification, regression, or clustering tasks with machine learning methods. Salient features of DAMEWARE include its ability to work on large datasets with minimal human intervention, and to deal with a wide variety of real problems such as the classification of globular clusters in the galaxy NGC1399; the evaluation of photometric redshifts; and, finally, the identification of candidate Active Galactic Nuclei in multiband photometric surveys. In all these applications, DAMEWARE allowed us to achieve better results than those attained with more traditional methods. With the aim of providing potential users with all needed information, in this paper we briefly describe the technological background of DAMEWARE, give a short introduction to some relevant aspects of data mining, followed by a summary of some science cases and, finally, provide a detailed description of a template use case.

E-learning and educational data mining in cloud computing: an overview

E-learning is related to virtualised distance learning by means of electronic communication mechanisms, using its functionality as a support in the process of teaching-learning. When the learning process becomes computerised, educational data mining employs the information generated from the electronic sources to enrich the learning model for academic purposes. To provide support to e-learning systems, cloud computing is set as a natural platform, as it can be dynamically adapted by presenting a scalable system for the changing necessities of the computer resources over time. It also eases the implementation of data mining techniques to work in a distributed scenario, regarding the large databases generated from e-learning. We give an overview of the current state of the structure of cloud computing, and we provide details of the most common infrastructures that have been developed for such a system. We also present some examples of e-learning approaches for cloud computing, and finally, we discuss the suitability of this environment for educational data mining, suggesting the migration of this approach to this computational scenario.

Comparative Analysis of Different Imputation Methods to Treat Missing Values in Data Mining Environment

Data cleaning is one of the important step of KDD (Knowledge discovery in database) process. One critical problem in data cleaning is the presence of missing values. Various approaches have proposed to find & replace such missing data including use of mean value, use of global constant, replace by more probable value etc. Imputation is one of the important procedures in statistics that is used to replace the missing values in a data set. One advantage of this approach is that the missing data treatment is independent of the learning algorithms that are used. This allows the user to select the most suitable and appropriate imputation method for each situation. This paper analyze the six different imputation methods proposed in the field of statistics and implement them in Data mining environment. An artificial data set of 1000 records is used to analyze the performance of these methods. For testing the significance of these methods Z-test approach were used. Exhaustive experiments show the effectiveness of the proposed methods. It is assumed that all the attributes of input data are of numeric data type.

State of the Art of Artificial Intelligence and Predictive Analytics in the E&P Industry: A Technology Survey

Summary Artificial intelligence (AI) has been used for more than 2 decades as a development tool for solutions in several areas of the exploration and production (E&P) industry: virtual sensing, production control and optimization, forecasting, and simulation, among many others. Nevertheless, AI applications have not been consolidated as standard solutions in the industry, and most common applications of AI still are case studies and pilot projects. In this work, an analysis of a survey conducted on a broad group of professionals related to several E&P operations and service companies is presented. This survey captures the level of AI knowledge in the industry, the most common application areas, and the expectations of the users from AI-based solutions. It also includes a literature review of technical papers related to AI applications and trends in the market and in research and development. The survey helped to verify that (a) data mining and neural networks are by far the most popular AI technologies used in the industry; (b) approximately 50% of respondents declared they were somehow engaged in applying workflow automation, automatic process control, rule-based case reasoning, data mining, proxy models, and virtual environments; (c) production is the area most affected by the applications of AI technologies; (d) the perceived level of available literature and public knowledge of AI technologies is generally low; and (e) although availability of information is generally low, it is not perceived equally among different roles. This work aims to be a guide for personnel responsible for production and asset management on how AI-based applications can add more value and improve their decision making. The results of the survey offer a guideline on which tools to consider for each particular oil and gas challenge. It also illustrates how AI techniques will play an important role in future developments of information-technology (IT) solutions in the E&P industry.

Towards an environment for data mining based analysis processes in bioinformatics and personalized medicine

Medicine is undergoing a revolution that is even transforming the nature of health care from reactive to proactive. To serve these new and diverse needs, bioinformatics and data mining are teaming up to generate tools and procedures for prediction of disease recurrence and progression, response to treatment, as well as new insights into various oncogenic pathways by taking into account the user needs and their heterogeneity. The p-medicine consortium ( http://www.p-medicine.eu ) is creating a biomedical platform to facilitate the translation from current practice to a predictive, personalized, preventive, participatory and psycho-cognitive medicine by integrating VPH models, clinical practice, imaging and omics data. In this paper, we present the challenges for data mining based analysis in bio- and medical informatics and our approach towards a data mining environment addressing these requirements in the p-medicine platform.

Bioinformatics resource manager v2.3: an integrated software environment for systems biology with microRNA and cross-species analysis tools

BackgroundMicroRNAs (miRNAs) are noncoding RNAs that direct post-transcriptional regulation of protein coding genes. Recent studies have shown miRNAs are important for controlling many biological processes, including nervous system development, and are highly conserved across species. Given their importance, computational tools are necessary for analysis, interpretation and integration of high-throughput (HTP) miRNA data in an increasing number of model species. The Bioinformatics Resource Manager (BRM) v2.3 is a software environment for data management, mining, integration and functional annotation of HTP biological data. In this study, we report recent updates to BRM for miRNA data analysis and cross-species comparisons across datasets.ResultsBRM v2.3 has the capability to query predicted miRNA targets from multiple databases, retrieve potential regulatory miRNAs for known genes, integrate experimentally derived miRNA and mRNA datasets, perform ortholog mapping across species, and retrieve annotation and cross-reference identifiers for an expanded number of species. Here we use BRM to show that developmental exposure of zebrafish to 30 uM nicotine from 6–48 hours post fertilization (hpf) results in behavioral hyperactivity in larval zebrafish and alteration of putative miRNA gene targets in whole embryos at developmental stages that encompass early neurogenesis. We show typical workflows for using BRM to integrate experimental zebrafish miRNA and mRNA microarray datasets with example retrievals for zebrafish, including pathway annotation and mapping to human ortholog. Functional analysis of differentially regulated (p<0.05) gene targets in BRM indicates that nicotine exposure disrupts genes involved in neurogenesis, possibly through misregulation of nicotine-sensitive miRNAs.ConclusionsBRM provides the ability to mine complex data for identification of candidate miRNAs or pathways that drive phenotypic outcome and, therefore, is a useful hypothesis generation tool for systems biology. The miRNA workflow in BRM allows for efficient processing of multiple miRNA and mRNA datasets in a single software environment with the added capability to interact with public data sources and visual analytic tools for HTP data analysis at a systems level. BRM is developed using Java™ and other open-source technologies for free distribution (http://www.sysbio.org/dataresources/brm.stm).

Privacy Preserving Combinatorial Function for Multi-Partitioned Data Sets

preserve the electronic fitness records of data mining applications, Data perturbation is used. It is a form of privacy- preserving data mining. An improved amount and range of information stored in databases has direct to an enhancement in the desire for ranked and best match queries. Such queries are mainly applicable when dealing with privacy- sensitive information. To facilitate privacy preservation in data mining or machine learning algorithms over horizontally partitioned or vertically partitioned data, many protocols have been proposed using SMC and various secure building blocks. In this work, we plan to present an effective and efficient cluster based privacy preserving data perturbation technique to mine Multi-partitioned data sets that comprises of both vertical and horizontal data sets which is current demand of e- business and e-commerce data mining environment. To evaluate a trade of between data privacy and transparency of individual's data, data perturbation technique is presented with validation and authentication. In multi-partitioned data distribution, data perturbation raised ambiguity between vertical and horizontal partitions of the data. To overcome the ambiguity, we plan to introduce divisive k-neighbor clusters for multi-partitioned data sets. The performance is evaluated with bench data sets obtained from popular e-business / e- commerce sites. (Amazon, e-bay etc.,). Keywords-partitioned data sets, Privacy preservation, Data perturbation technique.

Classification of savanna tree species, in the Greater Kruger National Park region, by integrating hyperspectral and LiDAR data in a Random Forest data mining environment

The accurate classification and mapping of individual trees at species level in the savanna ecosystem can provide numerous benefits for the managerial authorities. Such benefits include the mapping of economically useful tree species, which are a key source of food production and fuel wood for the local communities, and of problematic alien invasive and bush encroaching species, which can threaten the integrity of the environment and livelihoods of the local communities. Species level mapping is particularly challenging in African savannas which are complex, heterogeneous, and open environments with high intra-species spectral variability due to differences in geology, topography, rainfall, herbivory and human impacts within relatively short distances. Savanna vegetation are also highly irregular in canopy and crown shape, height and other structural dimensions with a combination of open grassland patches and dense woody thicket – a stark contrast to the more homogeneous forest vegetation. This study classified eight common savanna tree species in the Greater Kruger National Park region, South Africa, using a combination of hyperspectral and Light Detection and Ranging (LiDAR)-derived structural parameters, in the form of seven predictor datasets, in an automated Random Forest modelling approach. The most important predictors, which were found to play an important role in the different classification models and contributed to the success of the hybrid dataset model when combined, were species tree height; NDVI; the chlorophyll b wavelength (466nm) and a selection of raw, continuum removed and Spectral Angle Mapper (SAM) bands. It was also concluded that the hybrid predictor dataset Random Forest model yielded the highest classification accuracy and prediction success for the eight savanna tree species with an overall classification accuracy of 87.68% and KHAT value of 0.843.

Development of KaPPA-View4 for omics studies on Jatropha and a database system KaPPA-Loader for construction of local omics databases

The publication of the whole genome sequence of Jatropha curcas L. has contributed to the study of gene functions of this plant, especially in data-driven investigations such as transcriptome and proteome analyses. Metabolomics analyses of Jatropha have also been reported in recent years. However, the analytical tools for omics data from Jatropha are limited. We prepared a set of pathway maps where the predicted genes of Jatropha were assigned based on KEGG pathway maps, and developed an omics viewer named KaPPA-View4-Jatropha where the pathway maps were implemented. Out of 40,929 predicted genes, 8085 genes were mapped on the KEGG Metabolism maps, other KEGG maps, or gene category maps that were generated from gene classification data of KEGG BRITE. Two transcriptome datasets, four metabolome datasets and one gene co-expression dataset were registered in the viewer. To facilitate data sharing of unpublished omics data among research collaborators, we developed a local database system, KaPPA-Loader. These data mining environments and the supporting database system will help Jatropha researchers to discover key genes such as those involved in oil production, biosynthesis of toxic compounds, and stress resistance. KaPPA-View4-Jatropha and KaPPA-Loader are available from the KaPPA-View portal site (http://kpv.kazusa.or.jp/).

Orange4WS Environment for Service-Oriented Data Mining

Novel data-mining tasks in e-science involve mining of distributed, highly heterogeneous data and knowledge sources. However, standard data mining platforms, such as Weka and Orange, involve only their own data mining algorithms in the process of knowledge discovery from local data sources. In contrast, next generation data mining technologies should enable processing of distributed data sources, the use of data mining algorithms implemented as web services, as well as the use of formal descriptions of data sources and knowledge discovery tools in the form of ontologies, enabling automated composition of complex knowledge discovery workflows for a given data mining task. This paper proposes a novel Service-oriented Knowledge Discovery framework and its implementation in a service-oriented data mining environment Orange4WS (Orange for Web Services), based on the existing Orange data mining toolbox and its visual programming environment, which enables manual composition of data mining workflows. The new service-oriented data mining environment Orange4WS includes the following new features: simple use of web services as remote components that can be included into a data mining workflow; simple incorporation of relational data mining algorithms; a knowledge discovery ontology to describe workflow components (data, knowledge and data mining services) in an abstract and machine-interpretable way, and its use by a planner that enables automated composition of data mining workflows. These new features are showcased in three real-world scenarios.

Algal Functional Annotation Tool: a web-based analysis suite to functionally interpret large gene lists using integrated annotation and expression data

BackgroundProgress in genome sequencing is proceeding at an exponential pace, and several new algal genomes are becoming available every year. One of the challenges facing the community is the association of protein sequences encoded in the genomes with biological function. While most genome assembly projects generate annotations for predicted protein sequences, they are usually limited and integrate functional terms from a limited number of databases. Another challenge is the use of annotations to interpret large lists of 'interesting' genes generated by genome-scale datasets. Previously, these gene lists had to be analyzed across several independent biological databases, often on a gene-by-gene basis. In contrast, several annotation databases, such as DAVID, integrate data from multiple functional databases and reveal underlying biological themes of large gene lists. While several such databases have been constructed for animals, none is currently available for the study of algae. Due to renewed interest in algae as potential sources of biofuels and the emergence of multiple algal genome sequences, a significant need has arisen for such a database to process the growing compendiums of algal genomic data.DescriptionThe Algal Functional Annotation Tool is a web-based comprehensive analysis suite integrating annotation data from several pathway, ontology, and protein family databases. The current version provides annotation for the model alga Chlamydomonas reinhardtii, and in the future will include additional genomes. The site allows users to interpret large gene lists by identifying associated functional terms, and their enrichment. Additionally, expression data for several experimental conditions were compiled and analyzed to provide an expression-based enrichment search. A tool to search for functionally-related genes based on gene expression across these conditions is also provided. Other features include dynamic visualization of genes on KEGG pathway maps and batch gene identifier conversion.ConclusionsThe Algal Functional Annotation Tool aims to provide an integrated data-mining environment for algal genomics by combining data from multiple annotation databases into a centralized tool. This site is designed to expedite the process of functional annotation and the interpretation of gene lists, such as those derived from high-throughput RNA-seq experiments. The tool is publicly available at http://pathways.mcdb.ucla.edu.

A Technical Approach on Large Data Distributed Over a Network

Data mining is nontrivial extraction of implicit, previously unknown and potential useful information from the data. For a database with number of records and for a set of classes such that each record belongs to one of the given classes, the problem of classification is to decide the class to which the given record belongs. The classification problem is also to generate a model for each class from given data set. We are going to make use of supervised classification in which we have training dataset of record, and for each record the class to which it belongs is known. There are many approaches to supervised classification. Decision tree is attractive in data mining environment as they represent rules. Rules can readily expressed in natural languages and they can be even mapped o database access languages. Now a days classification based on decision trees is one of the important problems in data mining which has applications in many areas. Now a days database system have become highly distributed, and we are using many paradigms. we consider the problem of inducing decision trees in a large distributed network of highly distributed databases. The classification based on decision tree can be done on the existence of distributed databases in healthcare and in bioinformatics, human computer interaction and by the view that these databases are soon to contain large amounts of data, characterized by its high dimensionality. Current decision tree algorithms would require high communication bandwidth, memory, and they are less efficient and scalability reduces when executed on such large volume of data. So there are some approaches being developed to improve the scalability and even approaches to analyse the data distributed over a network.[keywords: Data mining, Decision tree, decision tree induction, distributed data, classification]

Exploiting Need of Heterogeneous Data Mining Environment Based on Mobile Computing Environments

the concept of Data Mining services is not alone sufficient. Data mining services play an important role in the field of Communication industry. Data mining is also called knowledge discovery in several database including mobile databases and for heterogeneous environment. In this paper, we discuss and analyze the consumptive behavior based on data mining technology. We discuss and analyze different aspects of data mining techniques and their behavior in mobile devices. We also analyze the better method or rule of data mining services which is more suitable for mobile devices. In this paper, we survey several aspects of open service framework based on grid structure which provides the heterogeneous environment for data mining on mobile computing environments. discovering frequent user's behavior patterns in the mobile agent system have been studied extensively in recent years. The key feature in most of these algorithms is that they use a dataset and frequent Item-Sets visited by the customers. In this case, some problems occur because they do not consider that mobile user's behavior patterns are dynamically variable as time passes. In this paper we discuss some of the data mining service which are use in different areas and then apply those services to mobile devices and then apply those DMS services in mobile computing and exploiting the need of DMS in mobile computing environments using CLDC and MIDP components. The Connected, Limited Device Configuration (CLDC) and the Mobile Information Device Profile (MIDP) have emerged as J2ME standards for mobile phone applications development which are used with DMS services. The role of CLDC and MIDP component is to apply Data Mining Services in mobile. Data Mining Services are useful in several sectors including Mobile, WWW, HealthCare scenario etc. We discuss several aspects step-by-step in this paper and analyze those aspects and approaches sequentially. Discuss their advantages and disadvantages and conclude with new concept.