Data Mining Tasks Research Articles

Efficient Feature Selection Algorithm Based on Particle Swarm Optimization With Learning Memory

Feature selection is an important pre-processing step in machine learning and data mining tasks, which improves the performance of the learning models by removing redundant and irrelevant features. Many feature selection algorithms have been widely studied, including greedy and random search approaches, to find a subset of the most important features for fulfilling a particular task (i.e., classification and regression). As a powerful swarm-based meta-heuristic method, particle swarm optimization (PSO) is reported to be suitable for optimization problems with continuous search space. However, the traditional PSO has rarely been applied to feature selection as a discrete space search problem. In this paper, a novel feature selection algorithm based on PSO with learning memory (PSO-LM) is proposed. The goal of the learning memory strategy is designed to inherit much more useful knowledge from those individuals who have higher fitness and offer faster progress, and the genetic operation is used to balance the local exploitation and the global exploration of the algorithm. Moreover, the $k$ -nearest neighbor method is used as a classifier to evaluate the classification accuracy of a particle. The proposed method has been evaluated on some international standard data sets, and the results demonstrated its superiority compared with those wrapper-based feature selection methods.

Regression Analysis With Differential Privacy Preserving

In the field of data mining, protecting sensitive data from being leaked is part of the focuses of current research. As a strict and provable definition of privacy model, differential privacy provides an excellent solution to the problem of privacy leakage. Numerous methods have been suggested to enforce differential privacy in various data mining tasks, such as regression analysis. However, existing solutions for regression analysis is less than satisfactory since the amount of noise added is excessive. What's worse, the adversary can launch model inversion attacks to infer sensitive information with the published regression model. Motivated by this, we propose a differential privacy budget allocation model. We optimize the regression model by adjusting the privacy budget allocation within the objective function. Extensive evaluation results show the superiority of the proposed model in terms of noise reduction, model inversion attack proof, and the trade-off between privacy protection and data utility.

Tuning parameters via a new rapid, accurate and parameter-less method using meta-learning

Dealing with a large parameter space in data mining tasks is extremely time-consuming, and the tuning method itself needs to be tuned since methods themselves have at least one parameter. Here, a new rapid and parameter-less method is presented to tune algorithms on diverse datasets to achieve high quality results in a short consumed time. The method presented here uses a pre-knowledge by using meta-features to guess closer point to optimal point in parameter space of target algorithms (here, support vector machine algorithm is used). For preparing the pre-knowledge, 282 meta-features are introduced and then genetic algorithm is applied to determine best meta-features for the target algorithm. Then the best meta-features are used to tune the target algorithm on unseen datasets. The results show in less than 0.19 minute in average, the method obtains approximately the same classification rates in comparison with others, but the consumed time is dramatically declined.

Two-Stage Topic Extraction Model for Bibliometric Data Analysis Based on Word Embeddings and Clustering

Topic extraction is an essential task in bibliometric data analysis, data mining and knowledge discovery, which seeks to identify significant topics from text collections. The conventional topic extraction schemes require human intervention and involve also comprehensive pre-processing tasks to represent text collections in an appropriate way. In this paper, we present a two-stage framework for topic extraction from scientific literature. The presented scheme employs a two-staged procedure, where word embedding schemes have been utilized in conjunction with cluster analysis. To extract significant topics from text collections, we propose an improved word embedding scheme, which incorporates word vectors obtained by word2vec, POS2vec, word-position2vec and LDA2vec schemes. In the clustering phase, an improved clustering ensemble framework, which incorporates conventional clustering methods (i.e., k-means, k-modes, k-means++, self-organizing maps and DIANA algorithm) by means of the iterative voting consensus, has been presented. In the empirical analysis, we analyze a corpus containing 160,424 abstracts of articles from various disciplines, including agricultural engineering, economics, engineering and computer science. In the experimental analysis, performance of the proposed scheme has been compared to conventional baseline clustering methods (such as, k-means, k-modes, and k-means++), LDA-based topic modelling and conventional word embedding schemes. The empirical analysis reveals that ensemble word embedding scheme yields better predictive performance compared to the baseline word vectors for topic extraction. Ensemble clustering framework outperforms the baseline clustering methods. The results obtained by the proposed framework show an improvement in Jaccard coefficient, Folkes & Mallows measure and F1 score.

Speed Up Similarity Search of Time Series Under Dynamic Time Warping

Similarity search is a foundational task in time series data mining. Although there are many ways to measure the similarity of time series, a lot of evidence indicates that dynamic time warping (DTW) has the best robustness in many applications. Unfortunately, the expensive computational cost limits its application in large-scale databases. To speed up similarity search under DTW, we design a framework of two-stage similarity search for time series. In the first stage, we propose an improved lower bounding distance, which can be used to discard plenty of dissimilar series to get a set of candidate sequences. In the second stage, to efficiently get the retrieval result from the set of candidate sequences, we explore early abandoning strategy to avoid the full calculation of DTW. Extensive experiments are conducted on real-world data sets. The experimental results indicate that the proposed method can improve the retrieval efficiency of similarity search under DTW and guarantee no false dismissals.

A survey of clustering algorithms for an industrial context

Across a wide variety of fields and especially for industrial companies, data are being collected and accumulated at a dramatic pace from many different resources and services. Hence, there is an urgent need for a new generation of computational theories and tools to assist humans in extracting useful information from the rapidly growing volumes of digital data. A well-known fundamental task of data mining to extract information is clustering. However, with the modified applications for various domains, several researchers have developed and have provided many clustering algorithms. This complexity makes it difficult for researchers and practitioners to keep up with clustering algorithms development. As a result, finding appropriate algorithms helps significantly to organize information and extract the correct answer from different queries of the databases. In this respect, the aim of this paper is to find the appropriate clustering algorithm for sparse industrial dataset. To achieve this goal, we first present related work that focus on comparing different clustering algorithms over the past twenty years. After that, we provide a categorization of different clustering algorithms found in the literature by matching their properties to the 4V’s challenges of Big data which allow us to select the candidate clustering algorithm. Finally, using internal validity indices, K-means, agglomerative hierarchical, DBSCAN and SOM have been implemented and compared on four datasets. In addition, we highlighted the best performing clustering algorithm that gives us the efficient clusters for each dataset.

Multi-Tasks Discovery Method Based on the Concept Network for Data Mining

Business and data understanding, that aims to identify multiple mining tasks, is the most primary phase in planning a practical data mining project. However, traditional tasks determination problem could only be solved by experienced analysts, which suffers from high communication cost and low efficiency. In this paper, we study the automatic task discovery method following the problem solving theory. First, we establish the concept network (CN) model to represent human knowledge and experience in the problem solving process. Then, we propose and demonstrate the structure of two major mining tasks (clustering and classification) in a CN. Finally, a data mining tasks discovery method (DMTD) is put forward, followed by two analysis subject evaluation algorithms. Experiment results illustrate that the DMTD is able to discover all the potential mining tasks from a predefined concept network, filtered by the important or interesting analysis subjects. Moreover, these tasks defined by the DMTD are proven to be available and valuable by fifty published papers.

Imputation-based Ensemble Techniques for Class Imbalance Learning

Correct classification of rare samples is a vital data mining task and of paramount importance in many research domains. This article mainly focuses on the development of the novel class-imbalance learning techniques, which make use of oversampling methods integrated with bagging and boosting ensembles. Two novel oversampling strategies based on the single and the multiple imputation methods are proposed. The proposed techniques aim to create useful synthetic minority class samples, similar to the original minority class samples, by estimation of missing values that are already induced in the minority class samples. The re-balanced datasets are then used to train base-learners of the ensemble algorithms. In addition, the proposed techniques are compared with the commonly used class imbalance learning methods in terms of three performance metrics including AUC, F-measure, and G-mean over several synthetic binary class datasets. The empirical results show that the proposed multiple imputation-based oversampling combined with bagging significantly outperforms other competitors.

Incremental Mining of High Utility Patterns in One Phase by Absence and Legacy-Based Pruning

Mining high utility patterns in dynamic databases is an important data mining task. While a naive approach is to mine a newly updated database in its entirety, the state-of-the-art mining algorithms all take an incremental approach. However, the existing incremental algorithms either take a two-phase paradigm that generates a large number of candidates that causes scalability issues or employ a vertical data structure that incurs a large number of join operations that leads to efficiency issues. To address the challenges with the existing incremental algorithms, this paper proposes a new algorithm incremental direct discovery of high utility patterns (Id2HUP+). Id2HUP+ adapts a one-phase paradigm by improving the relevance-based pruning and upper-bound-based pruning proposes a novel data structure for a quick update of dynamic databases and proposes the absence-based pruning and legacy-based pruning dedicated to incremental mining. The extensive experiments show that our algorithm is up to 1-3 orders of magnitude more efficient than the state-of-the-art algorithms, and is the most scalable algorithm.

A Novel Segmentation and Representation Approach for Streaming Time Series

Along with the coming of Internet of Everything era, massive numbers of pervasive connected devices in various fields are continuously producing oceans of time series stream data. In order to carry out different kinds of data mining tasks (similarity search, classification, clustering, and prediction) based on streaming time series efficiently and effectively, segmentation and representation which segment a streaming time series into several subsequences and provide approximative representation for the raw data, should be done as the first step. With the virtue of solid theoretical foundations, piecewise linear representation (PLR) has been gained success in yielding more compact representation and fewer segments. However, the current state of art PLR methods have their own flaws: For one thing, most of current PLR methods focus on the guaranteed error bound instead of the holistic approximation error, which may lead to excessive fitting errors of segments and loss of factual research significance. For another, most of current PLR methods process streaming time series with some fixed criteria, which cannot provide a more flexible way to represent streaming time series. Motivated by the above analysis, we propose a novel continuous segmentation and multi-resolution representation approach based on turning points, which subdivides the streaming time series by a set of temporal feature points and represents the time series flexibly. Our method can not only generate more accurate approximation than the state-of-the-art of PLR algorithm, but also represent the streaming time series in a more flexible way to meet different needs of users. Extensive experiments on different kinds of typical time series datasets have been conducted to demonstrate the superiorities of our method.

A Generalized Approach for Anomaly Detection From the Internet of Moving Things

Internet of Moving Things are connected to a variety of different types of sensors to form a world of moving things, including people, animals, vehicles, drones, and boats, etc. As the data of collectible moving things continue to increase, anomaly detection of moving things has become an increasingly popular data mining task. Traditional trajectory outlier detection algorithms can detect common anomalies effectively, but it is hard to detect generalized anomalies, such as viewable direction anomalies, gravity anomalies, and magnetic field anomalies which can be collected by the accelerometer, gyroscope, magnetometer, and RPM sensor, etc. For this, we proposed a generalized approach for anomaly detection from the Internet of Moving Things, called the moving things outlier detection algorithm (MTOD). We propose the distance of moving things, which is equal to the weighted sum of the location distance and the multi-sensor distance, and then use the multi-sensor data generalization and moving things partitioning and anomaly detection three-step framework to detect the generalized anomaly. The experimental results show that our MTOD algorithm can detect moving things anomaly efficiency and accurately.

Time Series Data Classification Based on Dual Path CNN-RNN Cascade Network

Time series data classification is a significant topic as its application can be found in a various domain. Recent studies have shown that data-driven approach based on deep learning is powerful for data mining tasks. A typical deep learning method, Artificial Neural Network (ANN), has been proven to be capable for match complicated functions thus leading to the popularity. Convolutional neural network (CNN) is a special kind of ANN that has been widely used in the area of image processing tasks as its ability for extracting spatial features. However, it remains a challenge for implementing CNN in time series data classification. Recurrent Neural Network (RNN) is popular for tackling time series data as it can effectively utilize temporal information. But it is time-consuming to train RNN. This paper proposes a Dual Path CNN-RNN Cascade Network (DPCRCN) that achieves an end-to-end learning for classification. We use a dual path CNN to achieve a multi-size receptive field for better feature extraction, then using RNN and the following fully-connected layers to learn the map between the given features and the output. We also use Region of Interest (RoI) pooling to make our model capable for a flexible shape of data. We evaluate our model on Activity Recognition system based on Multisensor data fusion (AReM) dataset and we compare with many popular algorithms. We also evaluate our model using different shape of data. The results show that our model outperforms the alternatives. In addition, we provide the details of training our model.

Cost-Sensitive Topical Data Acquisition from the Web

The cost of acquiring training data instances for induction of data mining models is one of the main concerns in real-world problems. The web is a comprehensive source for many types of data which can be used for data mining tasks. But the distributed and dynamic nature of web dictates the use of solutions which can handle these characteristics. In this paper, we introduce an automatic method for topical data acquisition from the web. We propose a new type of topical crawlers that use a hybrid link context extraction method for topical crawling to acquire on-topic web pages with minimum bandwidth usage and with the lowest cost. The new link context extraction method which is called Block Text Window (BTW), combines a text window method with a block-based method and overcomes challenges of each of these methods using the advantages of the other one. Experimental results show the predominance of BTW in comparison with state of the art automatic topical web data acquisition methods based on standard metrics.

Impact of Learners’ Quality and Diversity in Collaborative Clustering

Abstract Collaborative Clustering is a data mining task the aim of which is to use several clustering algorithms to analyze different aspects of the same data. The aim of collaborative clustering is to reveal the common underlying structure of data spread across multiple data sites by applying clustering techniques. The idea of collaborative clustering is that each collaborator shares some information about the segmentation (structure) of its local data and improve its own clustering with the information provided by the other learners. This paper analyses the impact of the quality and the diversity of the potential learners to the quality of the collaboration for topological collaborative clustering algorithms based on the learning of a Self-Organizing Map (SOM). Experimental analysis on real data-sets showed that the diversity between learners impact the quality of the collaboration. We also showed that some internal indexes of quality are a good estimator of the increase of quality due to the collaboration.

IBike: Intelligent public bicycle services assisted by data analytics

Bicycle sharing systems are becoming increasingly prevalent in urban environments. These systems provide an environmentally friendly transportation alternative in cities. The management of these systems faces many optimization problems. The most important of these problems are the individual maintenance of bicycles, rebalancing and shared facilities, and the use of systems by creating requirements in asymmetrical patterns. A series of data mining tasks based on real data sets is performed to solve the problem of unbalanced bicycle use.By analyzing the characteristics of each station, the stations are modeled from the perspective of individuals and clusters by means of different models. The evaluation indicators used to address the accuracy of the results provide an effective method for shared bicycle predictions.

Data Mining Techniques and its Uses in different fields: A Review Paper

Abstract: The paper introduced the Data Mining and issues related to it. Data mining is a technique by which we can extract useful knowledge from urge set of data. Data mining tasks used to perform various operations and used to solve various problems related to data mining. Data warehouse is the collection of different method and techniques used to extract useful information from raw data. Genetic Algorithm is based upon the Darwin’s Theory in which low standard chromosomes are removed from the population because of their inability to survive the process of selection. The high standard chromosomes survive and are mixed by recombination to form more appropriate individuals. In this urge amount of data is used to predict future result by following several steps.

Particle swarm optimization for network-based data classification

Complex networks provide a powerful tool for data representation due to its ability to describe the interplay between topological, functional, and dynamical properties of the input data. A fundamental process in network-based (graph-based) data analysis techniques is the network construction from original data usually in vector form. Here, a natural question is: How to construct an “optimal” network regarding a given processing goal? This paper investigates structural optimization in the context of network-based data classification tasks. To be specific, we propose a particle swarm optimization framework which is responsible for building a network from vector-based data set while optimizing a quality function driven by the classification accuracy. The classification process considers both topological and physical features of the training and test data and employing PageRank measure for classification according to the importance concept of a test instance to each class. Results on artificial and real-world problems reveal that data network generated using structural optimization provides better results in general than those generated by classical network formation methods. Moreover, this investigation suggests that other kinds of network-based machine learning and data mining tasks, such as dimensionality reduction and data clustering, can benefit from the proposed structural optimization method.

A new integrated workflow for improving permeability estimation in a highly heterogeneous reservoir of Sawan Gas Field from well logs data

The Sawan Gas Field is one of the most promising gas fields in Pakistan with a cumulative production of 850 BCF. The repetition of coarse sandstone, medium sandstone, and sandstone-shale intercalation in the production zone cause extreme heterogeneity. Consequently permeability varies enormously (from 0.01 mD to more than 1000 mD). Nevertheless, verifiable and accurate estimation of permeability in the production zone with no previous laboratory-derived data is considered a challenging task. In this study, we explore a methodology for improving permeability estimation based on the combination of neural network (NN), multiple variable regression, and classification of data mining using conventional well logs (GR, LLD, RHOB, DT, and NPHI). The approach works in two-steps. First, we compute permeability using empirical, statistical, and virtual techniques on a fully cored well in order to select the specialized regression model that will be responsible for building data partitioning and classification of data mining task. To improve the efficiency of the classifier model, we combine the NN with multiple variable regression for predicting accurate permeability values. In step-2, the proposed regression model was employed to determine the final permeability values from data partitioning and classification of data mining. The final result of this study revealed that the proposed approach which combines NN, multiple variable regression, and classification of data mining provide more uniform, accurate, and qualitative estimation of permeability compared with stand-alone generic or global regression model. Also electrofacies (EFs) classification was conducted over the model to validate the proposed approach.

Decision level ensemble method for classifying multi-media data

In the digital era, the data, for a given analytical task, can be collected in different formats, such as text, images and audio etc. The data with multiple formats are called multimedia data. Integrating and fusing multimedia datasets has become a challenging task in machine learning and data mining. In this paper, we present heterogeneous ensemble method that combines multi-media datasets at the decision level. Our method consists of several components, including extracting the features from multimedia datasets that are not represented by features, modelling independently on each of multimedia datasets, selecting models based on their accuracy and diversity and building the ensemble at the decision level. Hence our method is called decision level ensemble method (DLEM). The method is tested on multimedia data and compared with other heterogeneous ensemble based methods. The results show that the DLEM outperformed these methods significantly.

A visualization approach for discovering colocation patterns

ABSTRACTColocation mining is one of the major spatial data mining tasks. When discovering colocation patterns, spatial statistics or data mining approaches are commonly used. Colocation mining results are typically presented in a textual form and do not provide any spatial information; thus, the results lack an intuitive approach to obtain cognition of colocation rules. Here, we propose a visualization approach to discover colocation patterns for two independent point distributions and generate visual results. This approach makes use of the ability of human color perception. For two geographic features, our approach first generates density surfaces of the input features and then visualizes the density surfaces using a red or green light with different intensities. Then, based on the law of additive color mixing, our approach mixes the colors of the two density surfaces to generate a colocation rule map. The visualization approach can also provide local details of colocation and be used for local colocation analysis. Users can detect colocation patterns and their distribution from the colocation rule maps. We use both synthetic data and real data to test the performance of our approach.