Efficient Data Mining Techniques Research Articles

64 Digital Technologies and Machine Learning: a New Way to Look at Novel Traits at Spatial and Temporal Dimensions

Abstract Accurate measurement of target traits in individual animals or groups of animals under different environmental conditions, such as varying diets, disease treatments, and management practices, is crucial for the progress of livestock research. While experimental studies on research farms ensure internal validity of results, observational studies on commercial farms provide external validity, i.e. how such results generalize to commercial settings. Both types of studies require longitudinal follow-up of animals and individual identification for monitoring their well-being, as well as their performance in terms of production, reproduction, and disease traits. Advances in digital technologies, including wearables and remote sensors, coupled with efficient data management and data mining techniques, now allow for accurate monitoring of animals and their environment in both spatial and temporal dimensions. These technologies enable investigations of novel traits such as traits related to product quality, animal behavior and welfare, individual feed intake, and gas emissions. This presentation will highlight our research efforts in developing and adapting tools and methods for livestock applications, including specific research projects that implement such tools. For instance, we will discuss a case study aiming to predict dairy cows’ future performance based on their growth and mammary gland development during calf-heifer stages.

A data mining technique for detecting malignant mesothelioma cancer using multiple regression analysis.

Lung cancer is a substantial health issue globally, and it is one of the main causes of mortality. Malignant mesothelioma (MM) is a common kind of lung cancer. The majority of patients with MM have no symptoms. In the diagnosis of any disease, etiology is crucial. MM risk factor detection procedures include positron emission tomography, magnetic resonance imaging, biopsies, X-rays, and blood tests, which are all necessary but costly and intrusive. Researchers primarily concentrated on the investigation of MM risk variables in the study. Mesothelioma symptoms were detected with the help of data from mesothelioma patients. The dataset, however, included both healthy and mesothelioma patients. Classification algorithms for MM illness diagnosis were carried out using computationally efficient data mining techniques. The support vector machine outperformed the multilayer perceptron ensembles (MLPE) neural network (NN) technique, yielding promising findings. With 99.87% classification accuracy achieved using 10-fold cross-validation over 5 runs, SVM is the best classification when contrasted to the MLPE NN, which achieves 99.56% classification accuracy. In addition, SPSS analysis is carried out for this study to collect pertinent and experimental data.

Improvement of Association Algorithm and Its Application in Audit Data Mining

Data mining has become a relatively modern platform for information retrieval. The efficient data mining techniques can increase the reliability and accuracy of internal auditing for the various community even while lowering audit risk. Existing audit data mining approaches lack significant identification of hidden connections and interactions in bid data platforms. Hence, this study extends the literature survey on the signification of audit data mining in multiple applications. This survey identifies the scope of improved association algorithms in audit data mining, a rule-based machine learning approach to determine the exciting relationship among variables in large audit datasets. Therefore, a Conceptual Framework of Improved Association Algorithm (CFiAA) and its application in audit data mining is proposed. This study examines the strengths and weaknesses of the proposed CFiAA in audit mining. The proposed model has been trained using an audit data set and validates with various audit datasets. Finally, this paper presents the comparative analysis of the proposal to show its highest performance related to existing models. Thus, CFiAA scores the performance ratio of 94.5%, accuracy ratio of 92.4%, an efficiency ratio of 92.5%, F1 measure of 91.8%, error rate 32.5%, prediction ratio of 93.7%, and the precision ratio of 92.5% compared to existing models.

A Review of Satellite Remote Sensing Techniques of River Delta Morphology Change

River deltas are important coastal depositional systems that are home to almost half a billion people worldwide. Understanding morphology changes in deltas is important in identifying vulnerabilities to natural disasters and improving sustainable planning and management. In this paper, we critically review literature on satellite remote sensing techniques that were used to study delta morphology changes. We identify and categorize these techniques into 3 major classes: (1) one-step change detection, 2) two-step change detection, and (3) ensemble classifications. In total, we offer a review of 18 techniques with example studies, and strengths and caveats of each. Synthesis of literature reveals that sub-pixel-based algorithms perform better than pixel-based ones. Machine learning techniques rank second to sub-pixel techniques, although an ensemble of techniques can be used just as effectively to achieve high feature detection accuracies. We also evaluate the performance of the 7 most commonly used techniques in literature on a sample of global deltas. Findings show the unsupervised classification significantly outperforms the others, and is recommended as a first-order delta morphological feature extraction technique in previously unknown, or, data sparse deltaic territories. We propose four pathways for future advancement delta morphological remote sensing: (1) utilizing high-resolution imagery and development of more efficient data mining techniques, (2) moving toward universal applicability of algorithms and their transferability across satellite platforms, (3) use of ancillary data in image processing algorithms, and (4) development of a global-scale repository of deltaic data for the sharing of scientific knowledge across disciplines.

WITHDRAWN: An efficient self-tuning spectral clustering algorithm for chronic kidney disease prediction

Abstract Recently, Chronic Kidney Disease (CKD) is an increasingly severe problem which considered as a global health problem. To control CKD disease, early detection and characterization are considered to be important factors. In literature, many techniques were developed to identify CKD at an early stage but it fails to accurate identification because of redundancy and high dimension data. To solve the abovementioned problems, the efficient data mining technique must be utilized which increases accuracy for the identification of CKD. Hence, in this paper, self-tuning spectral clustering is introduced to reduce the high dimension and redundancy problem in data. The self-tuning spectral clustering is shown to extract and reveal required information from laboratory and clinical patient data which most helpful to assist physicians in increasing the accuracy of CKD identification before reach a severe stage. The clustering results are applying to machine learning techniques such as Deep Neural Network (DNN), Artificial Neural Network (ANN), Support Vector Machine (SVM), Random Forest (RF) and K-Nearest Neighbour (KNN). The proposed technique is implemented in the Phython and performance is evaluated. The performance of the proposed technique is analyzed with statistical measurements such as accuracy, specificity, precision, recall, sensitivity and F_score. In comparison analysis, our finding shows that SVM and KNN provide better identification of CKD with clustering technique and maximum 0.96% accuracy.

Gui Based Performance Comparison of WBC Segmentation and Its Classification

The earlier method for diagnosing or detecting some diseases was through Manual Examination of Blood Smear Image and had a lot of human errors and tedious work. Overcome these issues modern methods like segmentation with efficient data mining techniques and classify leukocytes (WBC) based on machine learning algorithms are implemented. With the aid of the image processing technique, automatically diagnoses the diseases using the features of WBCs. For accurate diagnosing of disease, correctly classified leukocytes, and its subclass are required. Geometrical, textural, and statistical features of different images are extracted and applied in classification algorithms. Hence, multi level-based classification developed using the different classifiers like LibSVM, Naive Bayes, J48, Zero R, PART and Random Forest classifiers are considered and select the best classifier which is used to efficiently classify each category. From the performance indices, the best technique can suit for the identification of disease like Leukaemia. The proposed work in this paper demonstrated using MATLAB in GUI Environment. Classification is developed using WEKA software. The segmentation of different testing and training images have done Using 5 fold Cross Validation techniques performance indices measured and classification accuracy was compared in this paper.

Configuring differential evolution adaptively via path search in a directed acyclic graph for data clustering

Abstract As an efficient data mining technique, data clustering has been widely-used for data analysis and extracting valuable hidden information. Leveraging the simplicity and effectiveness, the evolutionary optimization-driven clustering algorithms have exhibited promising performance and attracted tremendous attention. Up to the present, how to enable these algorithms to escape from local optima and accelerate convergence rates is an ongoing challenge. In this paper, we propose a novel adaptive Differential Evolution (DE) variant to deal with the above challenge when clustering data. In the improved DE algorithm, the four interdependent components, including mutation strategy, crossover strategy, scaling factor value, and crossover rate, are adaptively configured in an integrated manner via ant colony optimization (ACO) during the problem-solving process. To be specific, the relationships of four components in the DE algorithm are modeled as a directed acyclic graph, and a path in the graph exactly corresponds to a configuration for DE. During the optimization process, ant colony optimization is employed to search for a reasonable path for each individual of DE in terms of pheromones on arcs. In this manner, the configuration of the four interdependent components of DE will be generated dynamically, which is then used to guide the successive search behaviors of individuals in DE. Each individual has a path, representing a configuration for each component. After each iteration, individuals that generate promising solutions are allowed to deposit pheromone on the paths, resulting in more pheromones on the arcs appearing in better algorithm configurations (paths) more frequently. Through this manner, the search strategies and parameters of DE are comprehensively adapted by ACO. The proposed algorithm is named ACODE for short. To verify its effectiveness, the proposed ACODE is compared with four representative data clustering algorithms on eight widely-used benchmark datasets. The experimental results demonstrate the advantages of ACODE over half of the datasets.

Performance of Naïve Bayes, C4.5 and KNN using Breast Cancer, Iris and Hypothyroid Datasets

Data mining usually specifies the discovery of specific pattern or analysis of data from a large dataset. Classification is one of an efficient data mining technique, in which class the data are classified are already predefined using the existing datasets. The classification of medical records in terms of its symptoms using computerized method and storing the predicted information in the digital format is of great importance in the diagnosis of various diseases in the medical field. In this paper, finding the algorithm with highest accuracy range is concentrated so that a cost-effective algorithm can be found. Here the data mining classification algorithms are compared with their accuracy of finding exact data according to the diagnosis report and their execution rate to identify how fast the records are classified. The classification technique based algorithms used in this study are the Naive Bayes Classifier, the C4.5 tree classifier and the K-Nearest Neighbor (KNN) to predict which algorithm is the best suited for classifying any kind of medical dataset. Here the datasets such as Breast Cancer, Iris and Hypothyroid are used to predict which of the three algorithms is suitable for classifying the datasets with highest accuracy of finding the records of patients with the particular health problems. The experimental results represented in the form of table and graph shows the performance and the importance of Naïve Bayes, C4.5 and K-Nearest Neighbor algorithms. From the performance outcome of the three algorithms the C4.5 algorithm is a lot better than the Naïve Bayes and the K-Nearest Neighbor algorithm.

A Novel Association Rule-Based Data Mining Approach for Internet of Things Based Wireless Sensor Networks

Wireless Sensor Network (WSN) is one of the fundamental technologies used in the Internet of Things (IoT) which is deployed for diverse applications to carry out precise real-time observations. The limited resources of WSN with massive volume of fast-flowing IoT data make the aggregation and analytics of data more challenging. Recently, data mining-based solutions have been proposed to effectively handle the data being generated by the sensors and to analyze the data patterns for deducing the required information from it. The increasing need of these techniques motivated us to propose a distributed and efficient data mining technique that not only handles the massive and rapidly generated data by the nodes, but also increases the life span of the network. In this paper, we propose a novel scheme for the IoT based WSN that mines the sensor data using association rule without moving it to any Cluster Head (CH) or Base Station (BS). The new proposed scheme enables sensors to perform computations locally and only the minimum higher-level statistical summaries of the data at Cluster Members (CMs) are exchanged with their CH. This considerably reduces the communication overhead which ultimately prolongs the network lifetime. The proposed scheme is evaluated via extensive simulations and the results obtained demonstrate that the integration of the proposed scheme in the existing protocols significantly reduces the communication overhead which ultimately prolongs the network lifetime and stability.

CBA_Optimized: An Optimized Association based Classification Model with Enhanced Classification Accuracy Rate

Association classification is now evolving as an efficient data mining technique for building the classification models with enhanced accuracy rate. Many studies proposed different pruning methods to obtain highly efficient classifier rules in order to enhance the accuracy of the classifier. The pruning strategy must lead to reduced number of high quality rules that contribute to the accurate classification of the data. This study presents the improved version of association rule based classification algorithm CBA (Classification based on Association ) called CBA_Optimized that attempts to optimize the rules discovered by rule generation process used in CBA. The CBA_Optimized uses a unique ranking process to order the occurrence of the rules based on high confidence and rule class frequency to select the classifier rule to classify the data. This paper presents the efficient pruning method and extends the unique single rule classification strategy of CBA being more simple and accessible AC algorithms. With reduced size and slightly faster processing speed than other AC algorithms, the study attempts to enhance the classification accuracy rate.

Prediction of kidney disease stages using data mining algorithms

Abstract Early detection and characterization are considered to be critical factors in the management and control of chronic kidney disease. Herein, use of efficient data mining techniques is shown to reveal and extract hidden information from clinical and laboratory patient data, which can be helpful to assist physicians in maximizing accuracy for identification of disease severity stage. The results of applying Probabilistic Neural Networks (PNN), Multilayer Perceptron (MLP), Support Vector Machine (SVM) and Radial Basis Function (RBF) algorithms have been compared, and our findings show that the PNN algorithm provides better classification and prediction performance for determining severity stage in chronic kidney disease.

A Pattern-Based Method For Handling Confidence Measures While Mining Satellite Displacement Field Time Series: Application to Greenland Ice Sheet and Alpine Glaciers

For more than 40 years, Earth observation satellites have been regularly providing images of glaciers that can be used to derive surface displacement fields and study their dynamics. In the context of global warming, the analysis of displacement field time series (DFTS) can provide useful information. Efficient data mining techniques are, thus, required to extract meaningful displacement evolutions from such large and complex datasets. In this paper, a pattern-based data mining approach, which handles confidence measures, is proposed to analyze DFTS. In order to focus on the most reliable measurements, a displacement evolution reliability measure is defined. It is aimed at assessing the quality of each evolution and pruning the search space. Experiments on two different DFTS (annual displacement fields derived from optical data over Greenland ice sheet and 11-day displacement fields derived from synthetic aperture radar data over Alpine glaciers) show the potential of the proposed approach.

Malignant Mesothelioma Disease Diagnosis using Data Mining Techniques

ABSTRACTMalignant mesothelioma (MM) is very aggressive progress tumors of the pleura. MM in humans results from exposure to asbestos and asbestiform fibers. The incidence of MM is extremely high in some Turkish villages. Under computationally efficient data mining (DM) techniques, classification procedures were performed for MM disease diagnosis. The support vector machine (SVM) achieved promising results, outperforming the multilayer perceptron ensembles (MLPE) neural network method. It was observed that SVM is the best classification with 99.87% accuracy obtained via 10-fold crossvalidation in 5 runs when compare to MLPE neural network, which gives 99.56% classification accuracy. Sensitivity analysis is performed to find the important inputs for MM disease diagnosis under SVM model. Alkaline phosphatase (ALP) ranging from 300 to 500 gives the maximum possibility of having the MM disease. The MM disease dataset was prepared from a faculty of medicine’s database using new patient’s hospital reports from the south east region of Turkey.

Programming models and systems for Big Data analysis

ABSTRACTBig Data analysis refers to advanced and efficient data mining and machine learning techniques applied to large amount of data. Research work and results in the area of Big Data analysis are continuously rising, and more and more new and efficient architectures, programming models, systems, and data mining algorithms are proposed. Taking into account the most popular programming models for Big Data analysis (MapReduce, Directed Acyclic Graph, Message Passing, Bulk Synchronous Parallel, Workflow and SQL-like), we analysed the features of the main systems implementing them. Such systems are compared using four classification criteria (i.e. level of abstraction, type of parallelism, infrastructure scale and classes of applications) for helping developers and users to identify and select the best solution according to their skills, hardware availability, productivity and application needs.

Mobile health architecture for obesity management using sensory and social data

Abstract One of the principal causes of several chronic diseases (e.g., diabetes, high cholesterol, and hypertension) is the obesity epidemic in high and middle income countries. Obesity also leads to an increasingly negative effect on public health resources. Therefore, obesity and overweight have to be monitored to mitigate and prevent the potential risks generated from the threat of related diseases and from reducing productivity experienced by businesses. A mobile-health monitoring system includes sensing, transmitting, storing, processing, and analyzing intensive, continuous, and heterogeneous medical data. However, current approaches are standalone mobile applications, augmented mobile applications, or mobile health systems. These approaches only consider simple activities (assess, detect, or control obesity) and rely on a mobile phone to perform complex processing operations on the collected data. Such complex operations need (1) efficient data mining techniques, (2) more memory consumption and processing time, and (3) long life mobile battery. In this work, we develop a new comprehensive mobile architecture for tackling the challenging issues of obesity control, monitoring, and prevention. We introduce a set of business requirements considering stakeholders, sensor devices, and architecture requirements to meet our architecture's objectives. Our architecture system can also help individuals track food intake, lifestyle, calories intake, calories consumption, and exercise activities. We analyze the data collected from continuous monitoring using non-invasive sensors, in addition to the data collected from social communities created to propagate awareness and share appropriate information about the obesity problem and its solution. We develop data mining algorithms and sentiment analysis algorithms and generate intelligent suggestions, warnings, and recommendations to control and mitigate the risk of obesity and its related diseases. We develop schemes for reducing data and saving energy, which minimize the amount of network traffic within the community of sensors. Moreover, we totally implement our architecture system as a collection of Web services organized by the model–view–controller design pattern to write, retrieve, and access data to and from the cloud storage firebase. We finally evaluate the efficacy and scalability of the implemented system using a comprehensive cloud database including entered data, calculated data, sensory data, and social data of 50 underweight, overweight, normal, and obese volunteer subjects. The obtained results show our architecture's objectives are fulfilled.

A Comprehensive Survey of Data Mining Techniques on Time Series Data for Rainfall Prediction

Time series data available in huge amounts can be used in decision-making. Such time series data can be converted into information to be used for forecasting. Various techniques are available for prediction and forecasting on the basis of time series data. Presently, the use of data mining techniques for this purpose is increasing day by day. In the present study, a comprehensive survey of data mining approaches and statistical techniques for rainfall prediction on time series data was conducted. A detailed comparison of different relevant techniques was also conducted and some plausible solutions are suggested for efficient time series data mining techniques for future algorithms.Â

Real-Time Recognition of Calling Pattern and Behaviour of Mobile Phone Users through Anomaly Detection and Dynamically-Evolving Clustering

In the competitive telecommunications market, the information that the mobile telecom operators can obtain by regularly analysing their massive stored call logs, is of great interest. Although the data that can be extracted nowadays from mobile phones have been enriched with much information, the data solely from the call logs can give us vital information about the customers. This information is usually related with the calling behaviour of their customers and it can be used to manage them. However, the analysis of these data is normally very complex because of the vast data stream to analyse. Thus, efficient data mining techniques need to be used for this purpose. In this paper, a novel approach to analyse call detail records (CDR) is proposed, with the main goal to extract and cluster different calling patterns or behaviours, and to detect outliers. The main novelty of this approach is that it works in real-time using an evolving and recursive framework.

Rough Set Theory and Fuzzy Logic Based Warehousing of Heterogeneous Clinical Databases

Large amounts of data about the patients with their medical conditions are presented in the Medical databases. Analyzing all these databases is one of the difficult tasks in the medical environment. In order to warehouse all these databases and to analyze the patient’s condition, we need an efficient data mining technique. In this paper, an efficient data mining technique for warehousing clinical databases using Rough Set Theory (RST) and Fuzzy Logic is proposed. Our proposed methodology contains two phases – (i) Clustering and (ii) Classification. In the first phase, Rough Set Theory is used for clustering. Clustering is one of the data mining techniques for warehousing the heterogeneous data bases. Clustering technique is used to group data that have similar characteristics in the same cluster and also to group the data that have dissimilar characteristics with other clusters. After clustering the data, similar objects will be clustered in one cluster and the dissimilar objects will be clustered under another cluster. The RST can be reduced the complexity. Then in the second phase, these clusters are classified using Fuzzy Logic. Normally, Classification with Fuzzy Logic is generated more number of rules. Since the RST is utilized in our work, the classification using Fuzzy can be done with less amount of complexity. The proposed approach is evaluated using various clinical related databases from heart disease datasets – Cleveland, Switzerland and Hungarian. The performance analysis is based on Sensitivity, Specificity and Accuracy with different cluster numbers. The experimentation results show that our proposed methodology provides better accuracy result.

Multidimensional Analysis and Mining of Call Detail Records Using Pattern Cube Algorithm

In the emerging and keenly competitive telecommunications market, it is imperative for mobile telecom operators to carry out regular analysis of their massive stored call logs to maintain and manage their numerous subscribers. Such sequential stream data analysis requires an efficient data mining algorithm and techniques bearing in mind the challenges posed by its massive size. Many data mining applications have been adapted for similar purposes. However, there has not been much emphasis on an in-depth mining of call detail records (CDR) as a multidimensional sequential stream data with its attending storage overhead. This paper proposes a novel algorithm for the multidimensional analysis of call records. Pattern Cube Algorithm (PCA) was implemented with a computer program and established empirically that: a massive CDR can be meaningfully summarized into a handy record as data mart with a gain of about 90% reduction in size and the possibility of processing a huge data from any server irrespective of the size of the target data. A quantitative exploration of the various gains in IT resources is conducted through an extensive experimental study on a sample of CDR adapted from MTN Communications Nigeria Limited.

Big Data Classification using Fuzzy K-Nearest Neighbor

becomes troublesome to perform effective analysis using the current traditional techniques. Big data put forward a lot of challenges due to its several characteristics like volume, velocity, variety, variability, value and complexity. Today there is not only a necessity for efficient data mining techniques to process large volume of data but in addition a need for a means to meet the computational requirements to process such huge volume of data. The objective of this paper is to classify big data using Fuzzy K-Nearest Neighbor classifier, and to provide a comparative study between the results of the proposed systems and the method reviewed in the literature. In this paper we implemented the Fuzzy KNearest Neighbor method using the MapReduce paradigm to process on big data. Results on different data sets show that the proposed Fuzzy K-Nearest Neighbor method outperforms a better performance than the method reviewed in the literature.