Data Mining Tasks Research Articles

MDA: A Reconfigurable Memristor-Based Distance Accelerator for Time Series Mining on Data Centers

The rapid development of Internet-of-Things is yielding a huge volume of time series data, the real-time mining of which becomes a major load for data centers. The computation bottleneck in time series data mining is distance function, which is the fundamental element of many high data mining tasks. Recently various software optimization and hardware acceleration techniques have been proposed to tackle the challenge. However, each of these techniques is only designed or optimized for a specific distance function. To address this problem, in this paper we propose MDA, a high-throughput reconfigurable memristor-based distance accelerator for real-time and energy-efficient data mining with time series in data centers. Common circuit structure is extracted for efficiency, and the circuit can be configured to any specific distance functions. Particularly, we adopt the emerging device memristor for the design of MDA. Comprehensive experiments are presented with public available datasets to evaluate the performance of the proposed MDA. Experimental results show that compared with existing works, MDA has achieved a speedup of $3.5\times $ – $376\times $ on performance and an improvement of 1–3 orders of magnitude on energy efficiency with little accuracy loss.

AIS-Clus: A Bio-Inspired Method for Textual Data Stream Clustering

The spread of real-time applications has led to a huge amount of data shared between users. This vast volume of data rapidly evolving over time is referred to as data stream. Clustering and processing such data poses many challenges to the data mining community. Indeed, traditional data mining techniques become unfeasible to mine such a continuous flow of data where characteristics, features, and concepts are rapidly changing over time. This paper presents a novel method for data stream clustering. In this context, major challenges of data stream processing are addressed, namely, infinite length, concept drift, novelty detection, and feature evolution. To handle these issues, the proposed method uses the Artificial Immune System (AIS) meta-heuristic. The latter has been widely used for data mining tasks and it owns the property of adaptability required by data stream clustering algorithms. Our method, called AIS-Clus, is able to detect novel concepts using the performance of the learning process of the AIS meta-heuristic. Furthermore, AIS-Clus has the ability to adapt its model to handle concept drift and feature evolution for textual data streams. Experimental results have been performed on textual datasets where efficient and promising results are obtained.

A novel ensembling method to boost performance of neural networks

ABSTRACTClassification is one of the important tasks in data mining. Over the last few decades, neural networks have proved to be very effective in solving this task. This paper presents a novel algorithm, called Neural Network Boosting (NNBOOST) to improve the classification performance of neural networks using ensembling technique. The ensembling technique proposed here is based on the boosting concept to ensemble classifiers. A boosting algorithm sequentially learns a new model by reassigning pattern weights. NNBOOST algorithm adapts a new technique to assign weights to patterns based on the Euclidean distances of patterns from the centroids of respective classes. It assigns more weight to patterns closest to the centroid. The pattern weights are updated subsequently after building or learning a neural network model, based on the updated centroids to build the next neural network model. Weighted Majority voting scheme is used to combine all the learned neural network models. Weight or confidence of a learned neural network model is calculated as the ratio of correctly classified to misclassified training patterns. The algorithm is validated with six real life data sets taken from UCI repository. Results show the effectiveness of the algorithm in ensembling neural networks.

Efficiently mining cohesion-based patterns and rules in event sequences

Discovering patterns in long event sequences is an important data mining task. Traditionally, research focused on frequency-based quality measures that allow algorithms to use the anti-monotonicity property to prune the search space and efficiently discover the most frequent patterns. In this work, we step away from such measures, and evaluate patterns using cohesion — a measure of how close to each other the items making up the pattern appear in the sequence on average. We tackle the fact that cohesion is not an anti-monotonic measure by developing an upper bound on cohesion in order to prune the search space. By doing so, we are able to efficiently unearth rare, but strongly cohesive, patterns that existing methods often fail to discover. Furthermore, having found the occurrences of cohesive itemsets in the input sequence, we use them to discover the representative sequential patterns and the dominant partially ordered episodes, without going through the computationally expensive candidate generation procedures typically associated with sequential pattern and episode mining. Experiments show that our method efficiently discovers important patterns that existing state-of-the-art methods fail to discover.

High-dimensional similarity searches using query driven dynamic quantization and distributed indexing.

The concept of similarity is used as the basis for many data exploration and data mining tasks. Nearest neighbor (NN) queries identify the most similar items, or in terms of distance the closest points to a query point. Similarity is traditionally characterized using a distance function between multi-dimensional feature vectors. However, when the data is high-dimensional, traditional distance functions fail to significantly distinguish between the closest and furthest points, as few dissimilar dimensions dominate the distance function. Localized similarity functions, i.e. functions that only consider dimensions close to the query, quantize each dimension independently and only compute similarity for the dimensions where the query and the points fall into the same bin. These quantizations are query-agnostic and there is potential to improve accuracy when a query-dependent quantization is used. In this work we propose a query dependent equi-depth (QED) on-the-fly quantization method to improve high-dimensional similarity searches. The quantization is done for each dimension at query time and localized scores are generated for the closest p fraction of the points while a constant penalty is applied for the rest of the points. QED not only improves the quality of the distancemetric,but also improves query time performance by filtering out non relevant data. We propose a distributed indexing and query algorithm to efficiently compute QED. Our experimental results show improvements in classification accuracy as well as query performance up to one order of magnitude faster than Manhattan-based sequential scan NN queries over datasets with hundreds of dimensions. Furthermore, similarity searches with QED show linear or better scalability in relation to the number of dimensions, and the number of compute nodes.

WHO: A New Evolutionary Algorithm Bio-Inspired by Wildebeests with a Case Study on Bank Customer Segmentation

Numerous evolutionary algorithms have been proposed which are inspired by the amazing lives of creatures, such as animals, insects, and birds. Each inspired algorithm has its own advantages and disadvantages, and has its own way to accomplish exploration and exploitation. In this paper, a new evolutionary algorithm with novel concepts, called Wildebeests Herd Optimization (WHO), is proposed. This algorithm is inspired by the splendid life of wildebeests in Africa. Moving and migration are inseparable from wildebeests’ lives. When a wildebeest wants to choose its path during migration, it considers the best path known to itself, the location of the more mature wildebeests in the crowd, and the direction of wildebeests with high mobility. The WHO algorithm imitates these traits, and can concurrently explore and exploit the search space. For validating WHO, it is applied to optimization problems and data mining tasks. It is demonstrated that WHO outperforms other evolutionary algorithms, such as genetic algorithm (GA) and particle swarm optimization, in the assessed problems. Then, WHO is applied to the customer segmentation problem. Customer segmentation is one of the most important tasks of data mining, especially in the banking sector. In this paper, the customers of a bank with current accounts are segmented using WHO based on four aspects: profitability, cost, loyalty and credit; some of these aspects are calculated in a novel way. The results were welcome by the bank authorities.

Metaheuristics for data mining

In the context of big data, many scientific communities aim to provide efficient approaches to accommodate large-scale datasets. This is the case of the machine-learning community, and more generally, the artificial intelligence community. The aim of this article is to explain how data mining problems can be considered as combinatorial optimization problems, and how metaheuristics can be used to address them. Four primary data mining tasks are presented: clustering, association rules, classification, and feature selection. This article follows the publication of a book in 2016 concerning this subject (Dhaenens and Jourdan, Metaheuristics for big data, Wiley, New York, 2016); additionally, updated references and an analysis of the current trends are presented.

Structure-Preserved Unsupervised Domain Adaptation

Domain adaptation has been a primal approach to addressing the issues by lack of labels in many data mining tasks. Although considerable efforts have been devoted to domain adaptation with promising results, most existing work learns a classifier on a source domain and then predicts the labels for target data, where only the instances near the boundary determine the hyperplane and the whole structure information is ignored. Moreover, little work has been done regarding to multi-source domain adaptation. To that end, we develop a novel unsupervised domain adaptation framework, which ensures the whole structure of source domains is preserved to guide the target structure learning in a semi-supervised clustering fashion. To our knowledge, this is the first time when the domain adaptation problem is re-formulated as a semi-supervised clustering problem with target labels as missing values. Furthermore, by introducing an augmented matrix, a non-trivial solution is designed, which can be exactly mapped into a K-means-like optimization problem with modified distance function and update rule for centroids in an efficient way. Extensive experiments on several widely-used databases show the substantial improvements of our proposed approach over the state-of-the-art methods.

A Customized Non-Exclusive Clustering Algorithm for News Recommendation Systems

Clustering is one of the main tasks in machine learning and data mining and is being utilized in many applications including news recommendation systems. In this paper, we propose a new non-exclusive clustering algorithm named Ordered Clustering (OC) with the aim is to increase the accuracy of news recommendation for online users. The basis of OC is a new initialization technique that groups news items into clusters based on the highest similarities between news items to accommodate news nature in which a news item can belong to different categories. Hence, in OC, multiple memberships in clusters are allowed. An experiment is carried out using a real dataset which is collected from the news websites. The experimental results demonstrated that the OC outperforms the k-means algorithm with respect to Precision, Recall, and F1-Score.

Guest Editorial: Special Issue on Early Prediction and Supporting of Learning Performance

The papers in this special section focus on early prediction and the support of learning performance. Predicting student's learning performance in traditional face-to-face learning, online learning (LMS, MOOCs, etc.), and blended learning is a challenging but essential task in education [1]. On the one hand, it has become a difficult challenge due to the high number of factors that can influence a student’s final status. On the other hand, it is a critical issue in education because it concerns many students of all levels (primary education, secondary education, and tertiary or higher education) and institutions over the entire world. Moreover, also, an increase in the number of low performing students can cause a lower graduation rate, an inferior institution reputation in the eyes of all involved, and it usually results in overall financial loss. The task of predicting students’ performance is one of the oldest and most studied tasks in Educational Data Mining (EDM) and Learning Analytics (LA), and a wide range of classification and regression approaches have been successfully applied.

An Improved Naive Bayes Classifier on Imbalanced Attributes

Data plays a major and prominent role in this modern information era. Classification is a data mining task to discover the hidden information from large amounts of data stored in the repository. This process becomes extremely challenging in case of highly imbalanced dataset. Prediction from imbalanced attributes cannot be done accurately in the following case: During the training phase, the categorical variable is not observed but the test phase encounters the categorical variable and hence it assigns zero probability which leads to false prediction. To overcome this scenario, this article proposes a novel smoothing technique called optimized laplace smoothing estimation. This technique adds a bias value function to improve the accuracy of imbalanced attributes. For example, a child dataset has more attributes and the classification model is used to predict the child weight. Some of the attribute values may not be present in the child dataset due to which Naive Bayes assigns a zero for incomplete and an empty attribute. This leads to inaccurate prediction. In such cases, Naive Bayes can be further tuned by adding some new parameters as well as altering the existing optimization method. Experimental analysis shows that this novel smoothing technique enhances the classification accuracy by means of accurate predictions for imbalanced attributes.

CLS-Miner: efficient and effective closed high-utility itemset mining

High-utility itemset mining (HUIM) is a popular data mining task with applications in numerous domains. However, traditional HUIM algorithms often produce a very large set of high-utility itemsets (HUIs). As a result, analyzing HUIs can be very time consuming for users. Moreover, a large set of HUIs also makes HUIM algorithms less efficient in terms of execution time and memory consumption. To address this problem, closed high-utility itemsets (CHUIs), concise and lossless representations of all HUIs, were proposed recently. Although mining CHUIs is useful and desirable, it remains a computationally expensive task. This is because current algorithms often generate a huge number of candidate itemsets and are unable to prune the search space effectively. In this paper, we address these issues by proposing a novel algorithm called CLS-Miner. The proposed algorithm utilizes the utility-list structure to directly compute the utilities of itemsets without producing candidates. It also introduces three novel strategies to reduce the search space, namely chain-estimated utility co-occurrence pruning, lower branch pruning, and pruning by coverage. Moreover, an effective method for checking whether an itemset is a subset of another itemset is introduced to further reduce the time required for discovering CHUIs. To evaluate the performance of the proposed algorithm and its novel strategies, extensive experiments have been conducted on six benchmark datasets having various characteristics. Results show that the proposed strategies are highly efficient and effective, that the proposed CLS-Miner algorithm outperforms the current state-of-the-art CHUD and CHUI-Miner algorithms, and that CLS-Miner scales linearly.

Performance Analysis of Particle Swarm Optimization for Feature Selection

One of the key task in data mining is the selection of relevant features from datasets with high dimensionality. This is expected to reduce the time and space complexity, and consequently improve the performance of data mining algorithms for tasks such as classification. This study presents an empirical study of the effect of particle swarm optimization as a feature selection technique on the performance of classification algorithms. Two dataset from different domains were used: SMS spam detection and sentiment analysis datasets. Particle swarm optimization is applied on the datasets for feature selection. Both the reduced and raw dataset are separately classified using C4.5 decision tree, k-nearest neighbour and support vector machine. The result of the analysis showed that the improvement of classifier performance is case-dependent; some significant improvements are noticed in the sentiment analysis datasets and not in the SMS spam dataset. Although some marginal effect are observed on performance, it implies that with particle swarm optimization features selection the space complexity is reduced while maintaining the accuracy of the classifiers. Keywords—classification, feature selection, machine learning, particle swarm optimization, text mining

Mining high-utility itemsets in dynamic profit databases

High-Utility Itemset (HUI) mining is an important data-mining task which has gained popularity in recent years due to its applications in numerous fields. HUI mining aims at discovering itemsets that have high utility (e.g., yield a high profit) in transactional databases. Although several algorithms have been designed to enumerate all HUIs, an important issue is that they assume that the utilities (e.g., unit profits) of items are static. But this simplifying assumption does not hold in real-life situations. For example, the unit profits of items often vary over time in a retail store due to fluctuating supply costs and promotions. Ignoring this important characteristic of real-life transactional databases makes current HUI-mining algorithms inapplicable in many real-world applications. To address this critical limitation of current HUI-mining techniques, this paper studies the novel problem of mining HUIs in databases having dynamic unit profits. To accurately assess the utility of any itemset in this context, a redefined utility measure is introduced. Furthermore, a novel algorithm named MEFIM (Modified EFficient high-utility Itemset Mining), which relies on a novel compact database format to discover the desired itemsets efficiently, is designed. An improved version of the MEFIM algorithm, named iMEFIM, is also introduced. This algorithm employs a novel structure called P-set to reduce the number of transaction scans and to speed up the mining process. Experimental results show that the proposed algorithms considerably outperform the state-of-the-art HUI-mining algorithms on dynamic profit databases in terms of runtime, memory usage, and scalability.

Hybrid particle swarm optimization with spiral-shaped mechanism for feature selection

The “curse of dimensionality” is one of the largest problems that influences the quality of the optimization process in most data mining, pattern recognition, and machine learning tasks. Using high-dimensional datasets to train a classification model may reduce the generalization performance of the learned model. In addition, high dimensionality of the dataset results in high computational and memory costs. Feature selection is an important data preprocessing approach in many practical application domains that are relevant to expert and intelligent systems. Feature selection aims at selecting a subset of informative and relevant features from an original feature dataset. Therefore, using a feature selection approach to process the original data prior to the learning process is essential for enhancing the performance on the classification task. In this paper, hybrid particle swarm optimization with a spiral-shaped mechanism (HPSO-SSM) is proposed for selecting the optimal feature subset for classification via a wrapper-based approach. In HPSO-SSM, we make three improvements: First, a logistic map sequence is used to enhance the diversity in the search process. Second, two new parameters are introduced into the original position update formula, which can effectively improve the position quality of the next generation. Finally, a spiral-shaped mechanism is adopted as a local search operator around the known optimal solution region. For a complete evaluation, the proposed HPSO-SSM method is compared with six state-of-the-art meta-heuristic optimization algorithms, ten well-known wrapper-based feature selection techniques, and six classic filter-based feature selection methods. Various assessment indicators are used to properly evaluate and compare the performances of these approaches on twenty classic benchmark classification datasets from the UCI machine learning repository. According to the experimental results and statistical tests, the developed methods effectively and efficiently improve the classification accuracy compared with other wrapper-based approaches and filter-based approaches. The results demonstrate the high performance of the HPSO-SSM method in searching the feasible feature space and selecting the most informative attributes for solving classification problems. Therefore, the HPSO-SSM method has broad application prospects as a new feature selection approach.

Efficient algorithms to identify periodic patterns in multiple sequences

Periodic pattern mining is a popular data mining task, which consists of identifying patterns that periodically appear in data. Traditional periodic pattern mining algorithms are designed to find patterns in a single sequence. However, in several domains, it is desirable to discover patterns that are periodic in many sequences. An example of such application is market basket analysis. Given a database of sequences of transactions made by customers, discovering sets of items that are periodically bought by customers can help understand customer behavior. To discover periodic patterns common to multiple sequences, this paper extends the traditional problem of mining periodic patterns in a sequence. Two novel measures are defined called the standard deviation of periods and the sequence periodic ratio. Two algorithms are proposed to mine these patterns efficiently called MPFPSBFS and MPFPSDFS, which perform a breadth-first search and depth-first search, respectively. Because the sequence periodic ratio is neither monotone nor anti-monotone, these algorithms rely on a novel upper-bound called boundRa and two novel search space pruning properties to find periodic patterns efficiently. The algorithms have been evaluated on multiple datasets. Results show that they are efficient and can filter numerous non periodic itemsets to identify periodic patterns.

A Unified Entropy-Based Distance Metric for Ordinal-and-Nominal-Attribute Data Clustering.

Ordinal data are common in many data mining and machine learning tasks. Compared to nominal data, the possible values (also called categories interchangeably) of an ordinal attribute are naturally ordered. Nevertheless, since the data values are not quantitative, the distance between two categories of an ordinal attribute is generally not well defined, which surely has a serious impact on the result of the quantitative analysis if an inappropriate distance metric is utilized. From the practical perspective, ordinal-and-nominal-attribute categorical data, i.e., categorical data associated with a mixture of nominal and ordinal attributes, is common, but the distance metric for such data has yet to be well explored in the literature. In this paper, within the framework of clustering analysis, we therefore first propose an entropy-based distance metric for ordinal attributes, which exploits the underlying order information among categories of an ordinal attribute for the distance measurement. Then, we generalize this distance metric and propose a unified one accordingly, which is applicable to ordinal-and-nominal-attribute categorical data. Compared with the existing metrics proposed for categorical data, the proposed metric is simple to use and nonparametric. More importantly, it reasonably exploits the underlying order information of ordinal attributes and statistical information of nominal attributes for distance measurement. Extensive experiments show that the proposed metric outperforms the existing counterparts on both the real and benchmark data sets.

Surface Parameters Retrieval from Fully Bistatic Radar Scattering Data

Fully bistatic radar scattering from rough surfaces is of vital importance in terrain remote sensing, but results in bulky data volume. The scattering is dependent on physical parameters of the media and is controlled by the radar observation geometry. Together, the two sets of parameters determine the scattering patterns in a bistatic plane confined by incident and polar angles in both incident and scattering directions. For radar remote sensing, it is desirable to infer surface parameters of interest, with satisfactory accuracy, from large volumes of measured data sets. This is essentially a task of data mining. In this paper, we present model-generated bistatic radar scattering data, followed by a sensitivity analysis, to identify a suitable configuration in terms of parameter inversion from fully bistatic measurements by a Kalman filter-trained dynamic learning neural network (DLNN). Results indicate that with bistatic observation, superior retrieval performance (as compared to backscattering observation) can be readily achieved.

Heterogeneous Distributed Big Data Clustering on Sparse Grids

Clustering is an important task in data mining that has become more challenging due to the ever-increasing size of available datasets. To cope with these big data scenarios, a high-performance clustering approach is required. Sparse grid clustering is a density-based clustering method that uses a sparse grid density estimation as its central building block. The underlying density estimation approach enables the detection of clusters with non-convex shapes and without a predetermined number of clusters. In this work, we introduce a new distributed and performance-portable variant of the sparse grid clustering algorithm that is suited for big data settings. Our computed kernels were implemented in OpenCL to enable portability across a wide range of architectures. For distributed environments, we added a manager–worker scheme that was implemented using MPI. In experiments on two supercomputers, Piz Daint and Hazel Hen, with up to 100 million data points in a ten-dimensional dataset, we show the performance and scalability of our approach. The dataset with 100 million data points was clustered in 1198 s using 128 nodes of Piz Daint. This translates to an overall performance of 352 TFLOPS . On the node-level, we provide results for two GPUs, Nvidia’s Tesla P100 and the AMD FirePro W8100, and one processor-based platform that uses Intel Xeon E5-2680v3 processors. In these experiments, we achieved between 43% and 66% of the peak performance across all computed kernels and devices, demonstrating the performance portability of our approach.

A Novel Parameter-Light Subspace Clustering Technique Based on Single Linkage Method

Subspace clustering is a challenging high-dimensional data mining task. There have been several approaches proposed in the literature to identify clusters in subspaces, however their performance and quality is highly affected by input parameters. A little research is done so far on identifying proper parameter values automatically. Other observed drawbacks are requirement of multiple database scans resulting into increased demand for computing resources and generation of many redundant clusters. Here, we propose a parameter light subspace clustering method for numerical data hereafter referred to as CLUSLINK. The algorithm is based on single linkage clustering method and works in bottom up, greedy fashion. The only input user has to provide is how coarse or fine the resulting clusters should be, and if not given, the algorithm operates with default values. The empirical results obtained over synthetic and real benchmark datasets show significant improvement in terms of accuracy and execution time.