Knowledge Discovery From Data Research Articles

A visual-based toolkit to support mobility data analytics

The Knowledge Discovery from Data (KDD) process is widely used across various domains to get valuable insights from data. Many platforms, like KNIME or RapidMiner, offer effective tools for KDD analysts, allowing them to perform data analytics tasks in a visual fashion, without writing code. In recent years, the increasing availability of mobility data has led to a surge in KDD-based initiatives from both industry and academia in the Intelligent Transportation Systems (ITS) domain. Still, KDD platforms lack comprehensive support for some typical mobility data manipulation tasks. As a result, mobility data analysis still requires a significant coding phase, with reduced productivity and hindered replicability of results.To address this gap, this paper presents a novel solution aimed at supporting ITS data analysts in defining KDD processes more efficiently. More in detail, we extended the KNIME platform by introducing a collection of new components explicitly tailored to facilitate some peculiar KDD tasks from mobility data. These components encompass critical functionalities such as map coverage analysis, trajectory partitioning and map-matching.To showcase the effectiveness of the proposed solution, we used it to replicate a study published in the ITS data analytics domain. Thanks to our proposal, such replication can be accomplished in a few minutes and with just a few clicks, without any manual coding, resulting in a pipeline that is easier to understand, distribute and re-execute, also for domain experts with no programming experience.Our solution is open-source and freely downloadable from the Knime Hub. In this way, we aim to foster data-driven research and practice in the ITS field, by providing researchers and practitioners with more effective analytics tools to handle mobility data.

Inductive database to support iterative data mining: Application to biomarker analysis on patient data in the Fight-HF project

Machine learning is now an essential part of any biomedical study but its integration into real effective Learning Health Systems, including the whole process of Knowledge Discovery from Data (KDD), is not yet realised. We propose an original extension of the KDD process model that involves an inductive database. We designed for the first time a generic model of Inductive Clinical DataBase (ICDB) aimed at hosting both patient data and learned models. We report experiments conducted on patient data in the frame of a project dedicated to fight heart failure. The results show how the ICDB approach allows to identify biomarker combinations, specific and predictive of heart fibrosis phenotype, that put forward hypotheses relative to underlying mechanisms. Two main scenarios were considered, a local-to-global KDD scenario and a trans-cohort alignment scenario. This promising proof of concept enables us to draw the contours of a next-generation Knowledge Discovery Environment (KDE).

COMPARATIVE ANALYSIS OF CLUSTER VALIDITY INDICES IN TERMS OF CONSISTENCY

Cluster analysis is one of the key issues in Data Mining, the most important stage of the Knowledge Discovery from Data (KDD) process, and is widely used. There are 3 main tasks of cluster analysis: determining the optimal number of clusters, clustering algorithms and evaluating the quality of clustering. One of the most important steps in cluster analysis is to evaluate the quality of clustering. A number of indices have been proposed to assess the outcome of clustering. The analysis shows that these indices, which are used to assess the quality of clustering, often show inconsistent results. Therefore, extensive research has recently been conducted on the study of indices and new indices are proposed. The article examines a number of internal and external evaluation indices. Different size data sets are taken and k-means, k-medoids, agglomerative hierarchical, BIRCH and OPTICS algorithms are applied to them. A number of internal and external evaluation indices are used to assess the results of the experiment, and the results are analyzed comparatively. Experiments show that Ac, Pr, Rc and F-m indices show similar results in in group determining in a given clustering structure.

The Reproducible Data Reuse (ReDaR) Framework to Capture and Assess Multiple Data Streams

AbstractMuch of the literature in knowledge discovery from data (KDD) focuses on algorithms that are faster and more accurate at capturing patterns in a given data set. However, answering a research question is fundamentally connected with how well the data is aligned with the questions being asked. Thus, data selection is one of the most important steps to ensure that models produced from the KDD process are useful in practice. A lack of documentation about the data selection rationale and the transformations needed to semantically align the data streams prevents others from reproducing the research and obfuscates development of best practices in data integration. Our goal in this paper is to provide KDD practitioners with a framework that brings together theories in provenance, information quality, and contextual reasoning, to enable researchers to achieve a semantically aligned dataset with data selection, description, and documentation based on an application‐focused assessment.

A Novel Approach For Frequent Itemset Mining Using Geometric Progression Number Labeling

The task of finding Frequent Item Sets is a very important and time-consuming process in the field of data mining. In the process of Knowledge discovery from data (KDD), the analysis of Frequent Itemset Mining (FIM) produces very use full information like how the items or data are correlated with each other and how frequently it is being found in the database especially in the transaction data, its report plays a vital role for the decision-makers to procure items and to stock it. Since the stock is the asset of the business and the profit of the business depends on the liquidity of the asset, more importance was given to Frequent Itemset mining. In this present Big data era, the variety and volume of data are growing tremendously; it is a challenging situation to mine the valuable information in a faster manner. Even though many researchers proposed many different kinds of algorithms for Frequent Item Set Mining it suffers defeat against either the size of frequent itemset or the time taken to finish the task. From this point of view, this paper proposes a new approach to mine the frequent items by using the Geometric progression series number as a label for each item and making its sum of items in the subset as a single value to find its frequency.

The Review on Data Mining Techniques and its Applications

Data mining is a field of research which is escalating gradually. Data mining is the method of analyze data from unique views and summarization it interested in useful data. Data mining, also in general referred to like knowledge discovery from data (KDD), is the computerize or suitable extraction of patterns in place of knowledge completely stored or captured in large databases, data warehouse, the web, other vast information repositories or information streams. Data mining machine works with data warehouse and the whole process is divided into action plan to be performed on data: selection, transformation, mining and results elucidation. In this term paper, we have reviewed equal types of obedience in data mining, also make clear different areas somewhere used data mining feeling and used of it

Data summarization: a survey

Summarization has been proven to be a useful and effective technique supporting data analysis of large amounts of data. Knowledge discovery from data (KDD) is time consuming, and summarization is an important step to expedite KDD tasks by intelligently reducing the size of processed data. In this paper, different summarization techniques for structured and unstructured data are discussed. The key finding of this survey is that not all summarization techniques create a summary suitable for further analysis. It is highlighted that sampling techniques are a viable way of creating a summary for further knowledge discovery such as anomaly detection from summary. Also different summary evaluation metrics are discussed.

A comprehensive overview on the foundations of formal concept analysis

The immersion of voluminous collection of data is inevitable almost everywhere. The invention of mathematical models to analyse the patterns and trends of the data is an emerging necessity to extract and predict useful information in any Knowledge Discovery from Data (KDD) process. The Formal Concept Analysis (FCA) is an efficient mathematical model used in the process of KDD which is specially designed to portray the structure of the data in a context and depict the underlying patterns and hierarchies in it. Due to the huge increase in the application of FCA in various fields, the number of research and review articles on FCA has raised to a large extent. This review differs from the existing ones in presenting the comprehensive survey on the fundamentals of FCA in a compact and crisp manner to benefit the beginners and its focuses on the scalability issues in FCA. Further, we present the generic anatomy of FCA apart from its origin and growth at a primary level.

Identifying ecosystem patterns from time series of anchovy (Engraulis ringens) and sardine (Sardinops sagax) landings in northern Chile

ABSTRACTUseful knowledge acquisition from known and systematized information (data) is a big challenge for researchers, users and finally, decision makers. In this sense, knowledge discovery from data (KDD) process represents a valuable tool for information analysis. Moreover, this work presents an approach through KDD in time series pattern identification for anchovy and sardine fisheries and environmental data, in northern Chile. Time series, multivariate analysis and data mining techniques, along with technical literature review for results validation. The KDD approach and the data mining techniques implemented achieved an integration between these variables, identifying relevant patterns associated with fisheries abundance fluctuations and strong association with environmental changes such as El Niño and long-term cold–warm regimes between them, establishing anchovy and sardine pre-dominant time-periods, associated with environmental conditions are identified. The latter establishes groundwork for studying underlying functional relationships that could reduce gaps in the national fisheries management policies for those fisheries.

Particle Swarm Optimization based Feature Selection

Selection is a pre-processing step in knowledge discovery from data (KDD) which aims at retrieving relevant data from the database beforehand. It imparts quality to the results of data mining tasks by selecting optimal feature set from larger set of features. Various feature selection techniques have been proposed in past which, unfortunately, suffer from unavoidable problems such as high computational cost and getting stuck into the local optima. Evolutionary algorithms such as Particle Swarm Optimization (PSO) possess immense abilities to explore a large search space and rarely fall into local optima thus making them a nice choice for feature selection. In this paper, we have explored pros and cons of traditional and PSO based feature selection techniques and suggested some effective changes in existing approaches. KeywordsSwarm Optimization (PSO), Evolutionary Algorithm

Ethiopic maternal care data mining: discovering the factors that affect postnatal care visit in Ethiopia

BackgroundImproving maternal health and reducing maternal mortality rate are key concerns. One of the eight millennium development goals adopted at the millennium summit, was to improve maternal health in Ethiopia. This leads towards discovering the factors that hinder postnatal care visit in Ethiopia.MethodsIn this research, knowledge discovery from data (KDD) was applied to identify the factors that hinder postnatal care visits in Ethiopia. Decision tree (using J48 algorithm) and rule induction (using JRip algorithm) techniques were applied on 6558 records of Ethiopian demographic and health survey data. To construct essential target dataset attributes exploratory data analysis with frequency diagram is performed, missing value was filled and noisy value was corrected. Also the data are preprocessed using business and data understanding with detail statistical summary.ResultJ48 (93.97 % accuracy) and JRip (93.93 % accuracy) identifies places of delivery, assistance of health delivery professional, prenatal care health professional and age are the determinant factors. However, residence places also taken into consideration.ConclusionsIn this study, encouraging results were obtained by employing both decision tree and rule induction techniques. The rules generated by J48 and JRip algorithms are much understandable to explain the outcome easily. Thus, the result obtained highly supportive to construct, evaluate and update advertising and promotional maternal health policies. It is better to create a generic model with more coverage in terms of economic, demographic, social and genetic factors so as to integrate the result with knowledge based system.

A large-scale dataset of solar event reports from automated feature recognition modules

The massive repository of images of the Sun captured by the Solar Dynamics Observatory (SDO) mission has ushered in the era of Big Data for Solar Physics. In this work, we investigate the entire public collection of events reported to the Heliophysics Event Knowledgebase (HEK) from automated solar feature recognition modules operated by the SDO Feature Finding Team (FFT). With the SDO mission recently surpassing five years of operations, and over 280,000 event reports for seven types of solar phenomena, we present the broadest and most comprehensive large-scale dataset of the SDO FFT modules to date. We also present numerous statistics on these modules, providing valuable contextual information for better understanding and validating of the individual event reports and the entire dataset as a whole. After extensive data cleaning through exploratory data analysis, we highlight several opportunities for knowledge discovery from data (KDD). Through these important prerequisite analyses presented here, the results of KDD from Solar Big Data will be overall more reliable and better understood. As the SDO mission remains operational over the coming years, these datasets will continue to grow in size and value. Future versions of this dataset will be analyzed in the general framework established in this work and maintained publicly online for easy access by the community.

Démarche centrée utilisateur pour la conception de SIAD basés sur un processus d'ECD, application dans le domaine de la santé

This article concerns the design of Decision Support Systems (DSS) based on a Knowledge Discovery from Data (KDD) process. This process aims at generating a set of software modules; it is iterative and interactive For this reason, it is essential to us to take Human-Computer Interaction principles and models into account in the development of such systems. The interactive decisional system development is currently approached according to two antagonistic approaches. For the first approach, technology is fundamental; the second is "user-centered", placing the human actors in a central position. Although the first approach is still very present in companies, the current tendency "is user-centered ". In this context, we propose an approach which aims at integrating the stages of KDD process in a development model enriched under the human-computer interaction point of view, which is the U-model. Our application context is the fight against the nosocomial infections in the healthcare domain.

Research on Database and Information Technology in the Decision Tree of Communication Construction Scheme

With the development of database technology as well as the widespread application of Database management system, our capabilities of both generating and collecting data have been increasing rapidly. In addition, popular use of the World Wide Web as a global information system has flooded us with a tremendous amount of data and information. This explosive growth in stored or transient data has generated an urgent need for new techniques and automated tools that can intelligently assist us in transforming the vast amounts of data into useful information and knowledge. The Data Mining technology brought forward. Data mining, also popularly referred to knowledge discovery from data (KDD), is the automated or convenient extraction of patterns representing knowledge implicitly stored or captured in large databases, data warehouses, the Web, other massive information repositories, or data streams. Data mining is a multidisciplinary field, drawing work from areas including database technology, machine learning, statistics, pattern recognition, information retrieval, neural networks, knowledge-based systems, artificial intelligence, high-performance computing, and data visualization. Database and Information Technology in the Decision Tree of is very important for the military. The Data Mining had been applied and studied in these years, and it has been applied in many domains successfully, such as business, finance and medical treatment. However, little is applied in communication construction scheme.

A COMPREHENSIVE SURVEY ON DATA MINING

Now a day’s internet is a significant place for interchanging of data like text, images, audio, and video and for share-out information preferably in digital form. The usage of internet leads to accessing the immense amount of data. Data may be unstructured data, structure data, and semi-structured data. So we are storing and processing such vast amount of data having gigantic complexity. Researchers from the University of Berkeley estimate that every year about one Exabyte (= 1 Million Terabyte) of data brought forth, of which a large portion is in digital form. For ex., 100 hours of videos are uploaded on YouTube every minute (statistics from YouTube). The query is how to analyze such ample data effectively and efficiently? Answer is data mining. Data mining pertains to the process of analyzing, studying such declamatory quantity of data for witnessing useful patterns and knowledge. Today we have accumulation of large amount of data but deficiency of knowledge. In this paper our focusing on surveillance of a nimble arising field data mining which is also known as knowledge discovery from data (KDD). We are constituting fundamentals of data mining, also several strategies for analyzing data like classification, estimation, prediction, association rules, clustering etc., data mining process, its types like web, content and structure mining. After understating basics, we are presenting the different data mining models like decision tree, neural network with their bedrocks. Also presents real world applications and future scope of the dynamic and extraordinary discipline.

In the realm of Big Data ...

In 2008, Chris Anderson (2008), at that time the Editor–in–Chief of Wired, proposed that in the age of the petabyte, there was no longer any need for the scientific method, nor for models or theories. Although it might be contended that this was more provocation and journalistic hubris than formal or substantiated claim, the issue was taken up and has gathered momentum ever since. Indeed within a year or so of Anderson’s article, and a series of rejoinders published on the Edge Web site, ‘The Age of Big Data’ was being heralded, and the measure had increased from petabytes to exabytes, zettabytes, and yottabytes. Diebold (2012) usefully distinguishes between Big Data ‘the phenomenon’, ‘the term’, and ‘the discipline’; arguing that the phenomenon ‘continues unabated’, the term is ‘firmly entrenched’, and the discipline is ‘emerging’. In what follows we focus initially on the term and the phenomenon, but our main objective is to argue that it is critical that there is general understanding of the emerging discipline. In particular we aim to justify the assertion that in the age of Big Data the ability to be able to develop abstractions and concepts is at least as important as it was previously; perhaps even more so. Moreover that these skills and techniques need to be understood and available to all of us in an era where we are all analysts and researchers at least to the extent of our use of the internet and its potential for affording search and investigation of online resources. We seek to offer some critical insights into these activities — modeling, conceptualizing, and theorizing — by comparing and contrasting Knowledge Discovery from Data (KDD) with the Grounded Theory Method (GTM). The former a technical orientation, that although predating Big Data, lies at the heart of the emerging tools and techniques. The latter a widely used approach to qualitative research aimed at developing conceptual models ‘grounded in the data’.

An Analysis of students’ performance using classification algorithms

In recent years, the analysis and evaluation of students performance and retaining the standard of education is a very important problem in all the educational institutions. The most important goal of the paper is to analyze and evaluate the school students performance by applying data mining classification algorithms in weka tool. The data mining tool has been generally accepted as a decision making tool to facilitate better resource utilization in terms of students performance. The various classification algorithms could be specifically mentioned as J48, Random Forest, Multilayer Perceptron, IB1 and Decision Table are used. The results of such classification model deals with accuracy level, confusion matrices and also the execution time. Therefore conclusion could be reached that the Random Forest performance is better than that of different algorithms. I. Introduction Data Mining could be a promising and flourishing frontier in analysis of data and additionally the result of analysis has many applications. Data Mining can also be referred as Knowledge Discovery from Data (KDD).This system functions as the machine-driven or convenient extraction of patterns representing knowledge implicitly keep or captured in huge databases, data warehouses, the Web, data repositories, and information streams. Data Mining is a multidisciplinary field, encompassing areas like information technology, machine learning, statistics, pattern recognition, data retrieval, neural networks, information based systems, artificial intelligence and data visualization. The application of data mining is widely prevalent in education system. Educational data mining is an emerging field which can be effectively applied in the field of education. The educational data mining uses several ideas and concepts such as Association rule mining, classification and clustering. The knowledge that emerges can be used to better understand students' promotion rate, students' retention rate, students' transition rate and the students' success. The data mining system is pivotal and crucial to measure the students' performance improvement. The classification algorithms can be used to classify and analyze the students' data set in accurate manner. The students' academic performance is influenced by various factors like parents' education, locality, economic status, attendance, gender and result. The main objective of the paper is to use data mining methodologies to study and analyze the school students' performance. Data mining provides many tasks that could be used to study the students' performance. In this paper, the classification task is employed to gauge students' performance and deals with the accuracy, confusion matrices and the execution time taken by the various classification data mining algorithms. This paper is catalogued as follows. Section 2 enumerates a related work. Section 3 presents the idea of Classification and discusses the aspects of classification algorithm. Section 4 elaborates a Data Preprocessing. Section 5 explains the Implementation of model construction. Section 6 describes the results and discussions. Section 7 provides the conclusion. II. Related Work Alaael-Halees 2009 suggested that Data Mining is an emerging methodology used in educational field to enhance the understanding of learning process. The application of Data mining is widely spread in Higher Education system. In Education domain many researchers and authors have explored and discussed various applications of data mining in higher education. The authors had gone through the survey of the literature to understand the importance of data mining applications. In the year 2001 Luan al. suggested a powerful decision support tool called data mining. Data Mining is a powerful tool for academic purposes Alumni, Institutional effectiveness, marketing and enrollment can benefit from the use of data mining Data Mining is the most suited technology that can be used by lecturer, student, alumnus, manager and other educational staff and is a useful tool for decision making on their educational activities

Knowledge discovery about quality of life changes of spinal cord injury patients: clustering based on rules by states.

In this paper, an integral Knowledge Discovery Methodology, named Clustering based on rules by States, which incorporates artificial intelligence (AI) and statistical methods as well as interpretation-oriented tools, is used for extracting knowledge patterns about the evolution over time of the Quality of Life (QoL) of patients with Spinal Cord Injury. The methodology incorporates the interaction with experts as a crucial element with the clustering methodology to guarantee usefulness of the results. Four typical patterns are discovered by taking into account prior expert knowledge. Several hypotheses are elaborated about the reasons for psychological distress or decreases in QoL of patients over time. The knowledge discovery from data (KDD) approach turns out, once again, to be a suitable formal framework for handling multidimensional complexity of the health domains.

Knowledge discovery from data as a framework to decision support in medical domains

Introduction: Knowledge discovery from data (KDD) is a multidisciplinary discipline which appeared in 1996 for “non trivial identifying of valid, novel, potentially useful, ultimately understandable patterns in data”. Pre-treatment of data and post-processing is as important as the data exploitation (Data Mining) itself. Different analysis techniques can be properly combined to produce explicit knowledge from data. Methods: Hybrid KDD methodologies combining Artificial Intelligence with Statistics and visualization have been used to identify patterns in complex medical phenomena: experts provide prior knowledge (pK); it biases the search of distinguishable groups of homogeneous objects; support-interpretation tools (CPG) assisted experts in conceptualization and labelling of discovered patterns, consistently with pK. Results: Patterns of dependency in mental disabilities supported decision-making on legislation of the Spanish Dependency Law in Catalonia. Relationships between type of neurorehabilitation treatment and patterns of response for brain damage are assessed. Patterns of the perceived QOL along time are used in spinal cord lesion to improve social inclusion. Conclusion: Reality is more and more complex and classical data analyses are not powerful enough to model it. New methodologies are required including multidisciplinarity and stressing on production of understandable models. Interaction with the experts is critical to generate meaningful results which can really support decision-making, particularly convenient transferring the pK to the system, as well as interpreting results in close interaction with experts. KDD is a valuable paradigm, particularly when facing very complex domains, not well understood yet, like many medical phenomena.

Bridging experience and evidence in mental health care reform

Bridging experience and evidence in mental health care reform