Big Data Classification Problems Research Articles

DK-MS: an efficient method for solving imbalanced Big Data classification problems

DK-MS: an efficient method for solving imbalanced Big Data classification problems

Distributed independent vector machine for big data classification problems

Distributed independent vector machine for big data classification problems

Classification method for imbalanced LiDAR point cloud based on stack autoencoder

<abstract><p>The existing classification methods of LiDAR point cloud are almost based on the assumption that each class is balanced, without considering the imbalanced class problem. Moreover, from the perspective of data volume, the LiDAR point cloud classification should be a typical big data classification problem. Therefore, by studying the existing deep network structure and imbalanced sampling methods, this paper proposes an oversampling method based on stack autoencoder. The method realizes automatic generation of synthetic samples by learning the distribution characteristics of the positive class, which solves the problem of imbalance training data well. It only takes the geometric coordinates and intensity information of the point clouds as the input layer and does not need feature construction or fusion, which reduces the computational complexity. This paper also discusses the influence of sampling number, oversampling method and classifier on the classification results, and evaluates the performance from three aspects: true positive rate, positive predictive value and accuracy. The results show that the oversampling method based on stack autoencoder is suitable for imbalanced LiDAR point cloud classification, and has a good ability to improve the effect of positive class. If it is combined with optimized classifier, the classification performance of imbalanced point cloud is greatly improved.</p></abstract>

Improved cost-sensitive representation of data for solving the imbalanced big data classification problem

Dimension reduction is a preprocessing step in machine learning for eliminating undesirable features and increasing learning accuracy. In order to reduce the redundant features, there are data representation methods, each of which has its own advantages. On the other hand, big data with imbalanced classes is one of the most important issues in pattern recognition and machine learning. In this paper, a method is proposed in the form of a cost-sensitive optimization problem which implements the process of selecting and extracting the features simultaneously. The feature extraction phase is based on reducing error and maintaining geometric relationships between data by solving a manifold learning optimization problem. In the feature selection phase, the cost-sensitive optimization problem is adopted based on minimizing the upper limit of the generalization error. Finally, the optimization problem which is constituted from the above two problems is solved by adding a cost-sensitive term to create a balance between classes without manipulating the data. To evaluate the results of the feature reduction, the multi-class linear SVM classifier is used on the reduced data. The proposed method is compared with some other approaches on 21 datasets from the UCI learning repository, microarrays and high-dimensional datasets, as well as imbalanced datasets from the KEEL repository. The results indicate the significant efficiency of the proposed method compared to some similar approaches.

Using Pyspark Environment for Solving a Big Data Problem: Searching for Supersymmetric Particles

Supersymmetry theory predicts that every particle in the standard model has a superpartner particle with a different mass. The Classification Problem of Supersymmetric Particles in High-Energy represents a major challenge for physicists. This paper aims to resolve the Big data Classification Problem in the area of Supersymmetric Particles using the Apache Spark Environment with the "MLlib" library. This contribution attempts to explore the performance of Machine Learning methods in the context of large data such as a "Susy" dataset, collected from the UCI Machine Learning repository. In this work, the performance is measured using three metrics: Accuracy, Area Under Curve (AUC), and training Computation Time (CT). The results are promising and show that the Gradient Boosted Tree (GBT) classifier achieves a high accuracy score (79%). While the Logistic Regression (LR) algorithm realizes a well AUC score (86%).

FUZZ-EQ: A data equalizer for boosting the discrimination power of fuzzy classifiers

The definition of linguistic terms is a critical part of the construction of any fuzzy classifier. Fuzzy partitioning methods (FPMs) range from simple uniform partitioning to sophisticated optimization algorithms. In this paper we present FUZZ-EQ, a preprocessing algorithm that facilitates the construction of meaningful fuzzy partitions regardless of the FPM used. The proposed approach is radically different from any existing FPM: instead of adjusting the fuzzy sets to the training data, FUZZ-EQ adjusts the training data to a hypothetical uniform partition before applying any FPM. To do so, the original data distribution is transformed into a uniform distribution by applying the probability integral transform. FUZZ-EQ allows FPMs to provide classifiers with more granularity on high density regions, increasing the overall discrimination capability. Additionally, we describe the procedure to reverse this transformation and recover the interpretability of linguistic terms. To assess the effectiveness of our proposal, we conducted an extensive empirical study consisting of 41 classification tasks and 9 fuzzy classifiers with different FPMs, rule induction algorithms, and rule structures. We also tested the scalability of FUZZ-EQ in Big Data classification problems such as HIGGS, with 11 million examples. Experimental results reveal that FUZZ-EQ significantly boosted the classification performance of those classifiers using the same linguistic terms for all rules, including state-of-the-art classifiers such as FARC-HD or IVTURS.

Redundancy and Complexity Metrics for Big Data Classification: Towards Smart Data

It is recognized the importance of knowing the descriptive properties of a dataset when tackling a data science problem. Having information about the redundancy, complexity and density of a problem allows us to make decisions as to which data preprocessing and machine learning techniques are most suitable. In classification problems, there are multiple metrics to describe the overlapping of the features between classes, class imbalances or separability, among others. However, these metrics may not scale up well when dealing with big datasets, or may not simply be sufficiently informative in this context. In this paper, we provide a package of metrics for big data classification problems. In particular, we propose two new big data metrics: Neighborhood Density and Decision Tree Progression, which study density and accuracy progression by discarding half of the samples. In addition, we enable a number of basic metrics to handle big data. The experimental study carried out in standard big data classification problems shows that our metrics can quickly characterize big datasets. We identified a clear redundancy of information in most datasets, so that, discarding randomly 75% of the samples does not drastically affect the accuracy of the classifiers used. Thus, the proposed big data metrics, which are available as a Spark-Package, provide a fast assessment of the shape of a classification dataset prior to applying big data preprocessing, toward smart data.

Multi-Objective Hyper-Heuristic Improved Particle Swarm Optimization Based Configuration of Support Vector Machines for Big Data Cyber Security

The massive increase of information in the big data era has not only created data processing problems, but also the data security issues. These big data cyber security issues can be handled effectively using machine learning algorithms among which the Support Vector Machines (SVM) has better results on big data classification problems. Defining the proper configuration of the SVM requires expert knowledge in selecting the kernel function and other parameters and this can significantly improve its classification results. In this paper, the SVM configuration process is modelled as a multi-objective optimization problem by considering the false positive rate, false negative rate and model complexity parameters. A Hyper-Heuristic Improved Particle Swarm Optimization (HHIPSO) framework is developed to optimize the SVM multi-objective optimization problem by incorporating the hyper-heuristics and improved particle swarm optimization algorithm. The proposed hyper-heuristic framework includes the high-level strategy for controlling the selection of low-level heuristics by search process and the low-level heuristics generate the new SVM configuration solutions using different rules of PSO. The effective selection of the kernel function and the respective parameters of the SVM should result in better values of false positive rate and false negative rate and also reduce the complexity. The evaluation of the proposed HHIPSO is performed on two cyber security problems and the obtained results illustrated that the proposed approach is effective in improving the classification of big data cyber security problems than the other algorithms.

CFM-BD: A Distributed Rule Induction Algorithm for Building Compact Fuzzy Models in Big Data Classification Problems

Interpretability has always been a major concern for fuzzy rule-based classifiers. The usage of human-readable models allows them to explain the reasoning behind their predictions and decisions. However, when it comes to Big Data classification problems, fuzzy rule-based classifiers have not been able to maintain the good trade-off between accuracy and interpretability that has characterized these techniques in non-Big Data environments. The most accurate methods build too complex models composed of a large number of rules and fuzzy sets, while those approaches focusing on interpretability do not provide state-of-the-art discrimination capabilities. In this paper, we propose a new distributed learning algorithm named CFM-BD to construct accurate and compact fuzzy rule-based classification systems for Big Data. This method has been specifically designed from scratch for Big Data problems and does not adapt or extend any existing algorithm. The proposed learning process consists of three stages: 1) pre-processing based on the probability integral transform theorem; 2) rule induction inspired by CHI-BD and Apriori algorithms; 3) rule selection by means of a global evolutionary optimization. We conducted a complete empirical study to test the performance of our approach in terms of accuracy, complexity, and runtime. The results obtained were compared and contrasted with four state-of-the-art fuzzy classifiers for Big Data (FBDT, FMDT, Chi-Spark-RS, and CHI-BD). According to this study, CFM-BD is able to provide competitive discrimination capabilities using significantly simpler models composed of a few rules of less than 3 antecedents, employing 5 linguistic labels for all variables.

Enabling Smart Data: Noise filtering in Big Data classification

In any knowledge discovery process the value of extracted knowledge is directly related to the quality of the data used. Big Data problems, generated by massive growth in the scale of data observed in recent years, also follow the same dictate. A common problem affecting data quality is the presence of noise, particularly in classification problems, where label noise refers to the incorrect labeling of training instances, and is known to be a very disruptive feature of data. However, in this Big Data era, the massive growth in the scale of the data poses a challenge to traditional proposals created to tackle noise, as they have difficulties coping with such a large amount of data. New algorithms need to be proposed to treat the noise in Big Data problems, providing high quality and clean data, also known as Smart Data. In this paper, two Big Data preprocessing approaches to remove noisy examples are proposed: an homogeneous ensemble and an heterogeneous ensemble filter, with special emphasis in their scalability and performance traits. The obtained results show that these proposals enable the practitioner to efficiently obtain a Smart Dataset from any Big Data classification problem.

A Micro-Extended Belief Rule-Based System for Big Data Multiclass Classification Problems

Big data classification problems have drawn great attention from diverse fields, and many classifiers have been developed. Among those classifiers, the extended belief rule-based system (EBRBS) has shown its potential in both big data and multiclass situations, while the time complexity and computing efficiency are two challenging issues to be handled in EBRBS. As such, three improvements of EBRBS are proposed first in this paper to decrease the time complexity and computing efficiency of EBRBS for multiclass classification under the assumption of large amount of data, including the strategy to skip rule weight calculation, a simplified evidential reasoning algorithm, and the domain division-based rule reduction method. This turns out to be a micro version of the EBRBS, called Micro-EBRBS. Moreover, one of commonly used cluster computing, named Apache Spark , is then applied to implement the parallel rule generation and inference schemes of the Micro-EBRBS for big data multiclass classification problems. The comparative analyses of experimental studies demonstrate that the Micro-EBRBS not only can obtain a desired accuracy but also has the comparatively better time complexity and computing efficiency than some popular classifiers, especially for multiclass classification problems.

Machine learning materials physics: Surrogate optimization and multi-fidelity algorithms predict precipitate morphology in an alternative to phase field dynamics

Machine learning has been effective at detecting patterns and predicting the response of systems that behave free of natural laws. Examples include learning crowd dynamics, recommender systems and autonomous mobility. There also have been applications to the search for new materials that draw upon big-data classification problems. However, when it comes to physical systems governed by conservation laws, the role of machine learning has been more limited. Here, we present our recent work in exploring the role of machine learning methods in discovering, or aiding, the search for physics. Specifically, we focus on using machine learning algorithms to represent high-dimensional free energy surfaces with the goal of identifying precipitate morphologies in alloy systems. Traditionally, this problem has been approached by combining phase field models, which impose first-order dynamics, with elasticity, to traverse a free energy landscape in search of minima. Equilibrium precipitate morphologies occur at these minima. Here, we exploit the machine learning methods to represent high-dimensional data, combined with surrogate optimization, sensitivity analysis and multifidelity modeling as an alternate framework to explore phenomena controlled by energy extremization. This combination of data-driven methods offers an alternative to the imposition of first-order dynamics via phase field methods, and represents one aspect of applying machine learning to materials physics.

FCNB: Fuzzy Correlative Naive Bayes Classifier with MapReduce Framework for Big Data Classification

Abstract The term “big data” means a large amount of data, and big data management refers to the efficient handling, organization, or use of large volumes of structured and unstructured data belonging to an organization. Due to the gradual availability of plenty of raw data, the knowledge extraction process from big data is a very difficult task for most of the classical data mining and machine learning tools. In a previous paper, the correlative naive Bayes (CNB) classifier was developed for big data classification. This work incorporates the fuzzy theory along with the CNB classifier to develop the fuzzy CNB (FCNB) classifier. The proposed FCNB classifier solves the big data classification problem by using the MapReduce framework and thus achieves improved classification results. Initially, the database is converted to the probabilistic index table, in which data and attributes are presented in rows and columns, respectively. Then, the membership degree of the unique symbols present in each attribute of data is found. Finally, the proposed FCNB classifier finds the class of data based on training information. The simulation of the proposed FCNB classifier uses the localization and skin segmentation datasets for the purpose of experimentation. The results of the proposed FCNB classifier are analyzed based on the metrics, such as sensitivity, specificity, and accuracy, and compared with the various existing works.

A scalable and distributed dendritic cell algorithm for big data classification

In the era of big data, scaling evolution up to large-scale data sets is a very interesting and challenging task. The application of standard biological systems in such data sets is not straightforward. Therefore, a new class of scalable biological systems that embraces the huge storage and processing capacity of distributed platforms is required. In this work, we focus on the Dendritic Cell Algorithm (DCA), a bio-inspired classifier, and its limitation when coping with very large data sets. To overcome this limitation, we propose a novel distributed DCA version for data classification based on the MapReduce framework to distribute the functioning of this algorithm through a cluster of computing elements. Our experimental results show that our proposed distributed solution is suitable to enhance the performance of the DCA enabling the algorithm to be applied over big data classification problems.

A New MapReduce Approach with Dynamic Fuzzy Inference for Big Data Classification Problems

Currently, big data and its applications have become one of the emergent topics. In practice, MapReduce framework and its different extensions are the most popular approaches for big data. Fuzzy system based models stand out for many applications. However, when a given observation has no overlap with antecedent values, no rule can be invoked in classical fuzzy inference can also appear in big data environment, and therefore no consequence can be derived. Fortunately, fuzzy rule interpolation techniques can support inference in such cases. Combining traditional fuzzy reasoning technique and fuzzy interpolation method may promote the accuracy of inference conclusion. Therefore, in this article, an initial investigation into the framework of MapReduce with dynamic fuzzy inference/interpolation for big data applications (BigData-DFRI) is reported. The results of an experimental investigation of this method are represented, demonstrating the potential and efficacy of the proposed approach.

CHI-PG: A fast prototype generation algorithm for Big Data classification problems

Abstract The growing amount of available data has become a serious challenge to data mining and machine learning techniques. Well-known classification methods that have been widely applied so far are no longer feasible in Big Data environments. For this reason, prototype reduction techniques (both selection and generation) come up as a candidate solution to build a reduced version of the dataset that speeds up the execution of algorithms such as k-Nearest Neighbors and overcome their memory constraints. However, these solutions generally have a quadratic O(N2) time complexity and share similar limitations to those encountered in data mining and machine learning algorithms in terms of time and memory requirements. In order to overcome these limitations, we introduce a new distributed MapReduce prototype generation method called CHI-PG that provides a linear O(N) time complexity and ensures constant accuracy regardless of the degree of parallelism. This approach builds prototypes by applying a simple scheme based on the rule generation process of the Chi et al. Fuzzy Rule-Based Classification System and takes advantage of the suitability of this classifier for the MapReduce paradigm. The empirical study shows that our new approach significantly improves the execution time of a state-of-the-art distributed prototype reduction algorithm (MRPR) without decreasing (and even improving) classification accuracy and reduction rates. Moreover, CHI-PG has been shown to be a candidate solution to the time and memory constraints of k-Nearest Neighbors when tackling large-scale datasets.

CHI-BD: A fuzzy rule-based classification system for Big Data classification problems

The previous Fuzzy Rule-Based Classification Systems (FRBCSs) for Big Data problems consist in concurrently learning multiple Chi et al. FRBCSs whose rule bases are then aggregated. The problem of this approach is that different models are obtained when varying the configuration of the cluster, becoming less accurate as more computing nodes are added. Our aim with this work is to design a new FRBCS for Big Data classification problems (CHI-BD) which is able to provide exactly the same model as the one that would be obtained by the original Chi et al. algorithm if it could be executed with this quantity of data. In order to do so, we take advantage of the suitability of the Chi et al. algorithm for the MapReduce paradigm, solving the problems of the previous approach, which lead us to obtain the same model (i.e., classification accuracy) regardless of the number of computing nodes considered.

Big Data Classification Using the SVM Classifiers with the Modified Particle Swarm Optimization and the SVM Ensembles

The problem with development of the support vector machine (SVM) classifiers using modified particle swarm optimization (PSO) algorithm and their ensembles has been considered. Solving this problem would allow fulfilling the high-precision data classification, especially Big Data classification, with the acceptable time expenditures. The modified PSO algorithm conducts a simultaneous search of the type of kernel functions, the parameters of the kernel function and the value of the regularization parameter for the SVM classifier. The idea of particles' «regeneration» served as the basis for the modified PSO algorithm. In the implementation of this algorithm, some particles change the type of their kernel function to the one which corresponds to the particle with the best value of the classification accuracy. The offered PSO algorithm allows reducing the time expenditures for the developed SVM classifiers, which is very important for Big Data classification problem. In most cases such SVM classifier provides the high quality of data classification. In exceptional cases the SVM ensembles based on the decorrelation maximization algorithm for the different strategies of the decision-making on the data classification and the majority vote rule can be used. Also, the two-level SVM classifier has been offered. This classifier works as the group of the SVM classifiers at the first level and as the SVM classifier on the base of the modified PSO algorithm at the second level. The results of experimental studies confirm the efficiency of the offered approaches for Big Data classification.

A MapReduce Approach to Address Big Data Classification Problems Based on the Fusion of Linguistic Fuzzy Rules

AbstractThe big data term is used to describe the exponential data growth that has recently occurred and represents an immense challenge for traditional learning techniques. To deal with big data classification problems we propose the Chi-FRBCS-BigData algorithm, a linguistic fuzzy rule-based classification system that uses the MapReduce framework to learn and fuse rule bases. It has been developed in two versions with different fusion processes. An experimental study is carried out and the results obtained show that the proposal is able to handle these problems providing competitive results.

MRPR: A MapReduce solution for prototype reduction in big data classification

In the era of big data, analyzing and extracting knowledge from large-scale data sets is a very interesting and challenging task. The application of standard data mining tools in such data sets is not straightforward. Hence, a new class of scalable mining method that embraces the huge storage and processing capacity of cloud platforms is required. In this work, we propose a novel distributed partitioning methodology for prototype reduction techniques in nearest neighbor classification. These methods aim at representing original training data sets as a reduced number of instances. Their main purposes are to speed up the classification process and reduce the storage requirements and sensitivity to noise of the nearest neighbor rule. However, the standard prototype reduction methods cannot cope with very large data sets. To overcome this limitation, we develop a MapReduce-based framework to distribute the functioning of these algorithms through a cluster of computing elements, proposing several algorithmic strategies to integrate multiple partial solutions (reduced sets of prototypes) into a single one. The proposed model enables prototype reduction algorithms to be applied over big data classification problems without significant accuracy loss. We test the speeding up capabilities of our model with data sets up to 5.7 millions of instances. The results show that this model is a suitable tool to enhance the performance of the nearest neighbor classifier with big data.