Large-scale Data Classification Research Articles

Random Sampling Method of Large-Scale Graph Data Classification

Graph data appears in broad real-world applications in modelling complex objects in big data. Effective analysis of graph data provides a deeper understanding of the data in data mining tasks, including classification, clustering, prediction, and recommendation systems. Mining a large number of graphs becomes a challenging task because state-of-the-art methods are not scalable due to the memory limit. To address this issue, we propose a novel approximate random sampling method for large-scale graph data classification. In this approach, we applied a representation method to encode each graph as a record of a vector string and a set of graphs as a set of N records in a file. Then, we partition the set of records into disjoint subsets of data blocks, making each data block a random sample of the data file. After that, we randomly select a subset of data blocks, each being a random sample of the graph dataset, and compute the different graph property distributions. Since the data blocks in this model are much smaller than the entire data set, it is more efficient to analyze them on a standalone small machine, and multiple data blocks can be analyzed on multiple nodes of the cluster in parallel. Finally, we classified the graphs of data blocks using the SVM algorithm. In experimental evaluation, our proposed method outperformed state-of-the-art graph kernels on graph classification datasets in terms of accuracy.

대용량 자료의 이항 분류에서의 충분 차원 축소를 위한 전향적 접근 방법

Sufficient dimension reduction, aimed at finding a lower-dimensional subspace in explanatory variables that contains response variable information, typically relies on inverse-based methodologies. These methods are easy to implement but often require linear or constant variance conditions. To address these limitations, techniques for forwardly estimating the central subspace have been developed. In particular, methods utilizing the Reproducing Kernel Hilbert Space have gained attention, but their use in analyzing large datasets is limited due to the characteristics of the kernel space. In this paper, we study a novel forward approach for sufficient dimension reduction in binomial classification of large-scale data. We propose a method that employs a divide-and-conquer technique to split data into subsets, then independently perform dimension reduction on each subset before synthesizing them into a final model. It was shown that when the number of partitions of data was appropriately selected, the loss in prediction accuracy was not significant compared to the existing method, while being efficient in terms of storage space and calculation cost. In addition, simulations in various models showed superior prediction accuracy than other inverse-based techniques. The utility of the proposed method was confirmed through the real data analysis.

Fast sparse twin learning framework for large-scale pattern classification

Lately, twin support vector machine and its variants have received extensive attention and in-depth research in the field of large-scale pattern classification. However, they may lead to high computational cost, which greatly affects their development and research. Aiming to solve this problem, this paper a novel fast sparse twin learning framework for large-scale data classification is proposed. In this learning framework, by introducing sparse constraints into the dual problem, the number of support vectors can be effectively reduced in dual space, so as to improve the computational speed of the model. Importantly, the learning framework is not only sparsity for large-scale samples classification, but also insensitive to sample noise, thus it is stable for resampling. In addition, we use the modified Newton’s method to deal with the optimization problem with sparse constraints. Numerical experiments are carried out on ten large-scale datasets. The results show that the proposed learning framework for large-scale data classification problems has significant advantages and comparability with other learning methods in terms of computational speed and classification accuracy.

Volumetric Model Genesis in Medical Domain for the Analysis of Multimodality 2-D/3-D Data Based on the Aggregation of Multilevel Features

The automatic and accurate classification of medical imaging data has potential applications in computer-aided disease diagnosis, prognosis, and treatment. However, it remains a challenge to optimize recent deep learning algorithms in medical domain for the accurate classification of large-scale 3D volumetric data. To address these challenges, we propose an efficient deep volumetric classification network based on the aggregation of multilevel deep features for accurate classification of large-scale medical 2D/3D imaging data. To perform a detailed quantitative analysis of our method, 26 different datasets were fused to construct a single large-scale multimodal database that comprises a total of seventy different classes, including 151,095 data samples. Additionally, 15 different baseline methods were configured under the same experimental protocol for volumetric model genesis and extensive performance comparison with our method. The experimental results of our method exhibited promising performance as area under the curve of 93.66% and outperformed various state-of-the-art methods.

Large-scale data classification based on the integrated fusion of fuzzy learning and graph neural network

Deep learning and fuzzy models provide powerful and practical techniques for solving large-scale deep-learning tasks. The fusion technique on deep learning and fuzzy system are generally classified into ensemble and integrated modes and materializes in information fusion, model fusion, and feature fusion. In an ensemble-based fusion, the fuzzy model either acts as an activation function or is operated as a separate process aggregating/preprocessing the information. Some early attempts in the field have successfully fused deep neural networks and fuzzy modeling concepts in ensemble mode. However, no effective attempts were made to fuse fuzzy models as an integrated feature-level fusion learning with graph neural networks (GNNs). This is mainly due to two challenges related to this fusion: (1) the number of fuzzy rules grows exponentially with the number of features that causes computational inefficiency, and (2) the solution space created by this fusion of fuzzy rules becomes complex due to multiple regression relations between inputs and outputs. Additionally, a simple linear regression at the output space would not be sufficient to model deep learning tasks. Therefore, this paper addresses these challenges by proposing a feature-level fusion method to fuse deep learning and fuzzy modeling where the latter technique is for integrated feature learning, called fuzzy forest graph neural network (FuzzyGNN), which creates a fuzzy learning forest fusing the linear graph transformers for deep learning tasks. We conducted experiments on fourteen machine learning datasets to test and validate the efficiency of the proposed FuzzyGNN model. Compared to state-of-the-art methods, our algorithm achieves the best results on four out of five machine learning datasets. The source code will be available at https://github.com/lingping-fuzzy/, https://github.com/vojha-code and https://github.com/P-N-Suganthan.

Classification of Large-Scale Mobile Laser Scanning Data in Urban Area with LightGBM

Automatic point cloud classification (PCC) is a challenging task in large-scale urban point clouds due to the heterogeneous density of points, the high number of points and the incomplete set of objects. Although recent PCC studies rely on automatic feature extraction through deep learning (DL), there is still a gap for traditional machine learning (ML) models with hand-crafted features, particularly after emerging gradient boosting machine (GBM) methods. In this study, we are using the traditional ML framework for the problem of PCC in large-scale datasets following the steps of neighborhood definition, multi-scale feature extraction, and classification. Different from others, our framework takes advantage of the fast feature calculation with multi-scale radius neighborhood and a recent state-of-the-art GBM classifier, LightGBM. We tested our framework using three mobile urban datasets, Paris–Rau–Madame, Paris–Rue–Cassette and Toronto3D. According to the results, our framework outperforms traditional machine learning models and competes with DL-based methods.

A stochastic configuration networks based on Harris hawks optimizer

Stochastic configuration networks (SCNs), as a novel incremental generation model with supervisory mechanism, has an excellent superiority in solving large-scale data regression and classification problems. However, the accuracy of the SCNs is affected by the assignation and selection of some network parameters significantly. Harris hawk optimizer (HHO) algorithm is a new meta-heuristic algorithm that simulates the foraging and anti-predation behavior of Harris hawk. In this paper, a SCNs based on HHO algorithm is first introduced, termed as HHO-SCNs. As the performance of SCNs is related to regularization parameter r and scale factor lambda of weights and biases, then HHO is employed to give better parameters for SCNs automatically. A numerical function and six benchmark datasets are used to verify the regression performance of the proposed model. Three benchmark datasets are introduced to illustrate the effectiveness of the proposed model for classification performance. Experimental results demonstrate the feasibility and validity of HHO-SCNs compared with incremental random vector functional link, SCNs, fast SCNs, and SCNs based on whale optimization algorithm. The proposed HHO-SCNs improves the generalization performance of standard SCNs, and provides a new idea for expanding the development and application of SCNs.

Distributed random vector functional link network with subspace-based local connections

In order to solve the problem of poor generalization ability and high computational complexity of random vector functional link (RVFL) network when dealing with large-scale data classification, we design and implement a distributed RVFL network with subspace-based local connections in Spark framework (DRVFL-SLC). Firstly, in order to take advantage of the partition parallelism of resilient distributed dataset (RDD), the large-scale dataset stored in the Hadoop distributed file system HDFS is randomly divided into random sample partition (RSP) data blocks and each RSP data block corresponds to a partition of the RDD, where the RSP data block is a subset of data that maintains probability distribution consistency with the big data at a given significance level. After that, the mapPartitions transformation is invoked on the RDD containing multiple partitions in a distributed environment and this operation trains the corresponding optimal RVFL-SLC efficiently in parallel. Then, the collect execution operator is used to efficiently fuse the optimal RVFL-SLC corresponding to each partition of the RDD to obtain DRVFL-SLC for realizing the classification of big data. Finally, the feasibility and effectiveness of DRVFL-SLC are verified based on several large-scale data set with at least million records on a Spark cluster deployed with 6 computing nodes. The results show that DRVFL-SLC has a good speedup ratio, scalability and scale growth, and can achieve better generalization performance than RVFL-SLC trained on a single machine with full data.

An Improved RandLa-Net Algorithm Incorporated with NDT for Automatic Classification and Extraction of Raw Point Cloud Data

A high-definition map of the autonomous driving system was built with the target points of interest, which were extracted from a large amount of unordered raw point cloud data obtained by Lidar. In order to better obtain the target points of interest, this paper proposes an improved RandLa-Net algorithm incorporated with NDT registration, which can be used to automatically classify and extract large-scale raw point clouds. First, based on the NDT registration algorithm, the frame-by-frame raw point cloud data were converted into a point cloud global map; then, the RandLa-Net network combined random sampling with a local feature sampler is used to classify discrete points in the point cloud map point by point. Finally, the corresponding point cloud data were extracted for the labels of interest through numpy indexing. Experiments on public datasets senmatic3D and senmatickitti show that the method has excellent accuracy and processing speed for the classification and extraction of large-scale point cloud data acquired by Lidar.

Applying the Properties of Neurons in Machine Learning: A Brain-like Neural Model with Interactive Stimulation for Data Classification

Some neural models achieve outstanding results in image recognition, semantic segmentation and natural language processing. However, their classification performance on structured and small-scale datasets that do not involve feature extraction is worse than that of traditional algorithms, although they require more time to train. In this paper, we propose a brain-like neural model with interactive stimulation (NMIS) that focuses on data classification. It consists of a primary neural field and a senior neural field that play different cognitive roles. The former is used to correspond to real instances in the feature space, and the latter stores the category pattern. Neurons in the primary field exchange information through interactive stimulation and their activation is transmitted to the senior field via inter-field interaction, simulating the mechanisms of neuronal interaction and synaptic plasticity, respectively. The proposed NMIS is biologically plausible and does not involve complex optimization processes. Therefore, it exhibits better learning ability on small-scale and structured datasets than traditional BP neural networks. For large-scale data classification, a nearest neighbor NMIS (NN_NMIS), an optimized version of NMIS, is proposed to improve computational efficiency. Numerical experiments performed on some UCI datasets show that the proposed NMIS and NN_NMIS are significantly superior to some classification algorithms that are widely used in machine learning.

A novel deep learning based security assessment framework for enhanced security in swarm network environment

Security assessments are essential in network systems to improve the reliability of the environment. This study presents a deep learning-based security assessment model as a proactive approach for monitoring network activities. This approach can improve security across the network environment and connected computing infrastructures by detecting and classifying various types of security attacks. Deep learning is one of the emerging solutions for integrating intelligent and smart techniques into traditional solutions for improving the performance of security detection. Leveraging a multilayer perceptron (MLP) combined with an XGBoost classifier for large-scale data processing and classification, the performance of the approach demonstrated an accuracy of 93.30% and a precision of 92.73% for malicious attack detection.

Evolving Metric Learning for Incremental and Decremental Features

Online metric learning has been widely exploited for large-scale data classification due to the low computational cost. However, amongst online practical scenarios where the features are evolving (<i>e.g.</i>, some features are vanished and some new features are augmented), most metric learning models cannot be successfully applied to these scenarios, although they can tackle the evolving instances efficiently. To address the challenge, we develop a new online Evolving Metric Learning (EML) model for incremental and decremental features, which can handle the instance and feature evolutions simultaneously by incorporating with a smoothed Wasserstein metric distance. Specifically, our model contains two essential stages: a Transforming stage (T-stage) and a Inheriting stage (I-stage). For the T-stage, we propose to extract important information from vanished features while neglecting non-informative knowledge, and forward it into survived features by transforming them into a low-rank discriminative metric space. It further explores the intrinsic low-rank structure of heterogeneous samples to reduce the computation and memory burden especially for highly-dimensional large-scale data. For the I-stage, we inherit the metric performance of survived features from the T-stage and then expand to include the new augmented features. Moreover, a smoothed Wasserstein distance is utilized to characterize the similarity relationships among the heterogeneous and complex samples, since the evolving features are not strictly aligned in the different stages. In addition to tackling the challenges in one-shot case, we also extend our model into multi-shot scenario. After deriving an efficient optimization strategy for both T-stage and I-stage, extensive experiments on several datasets verify the superior performance of our EML model.

A fuzzy spectral clustering algorithm for hyperspectral image classification

Spectral clustering is an unsupervised clustering algorithm, and is widely used in the field of pattern recognition and computer vision due to its good clustering performance. However, the traditional spectral clustering algorithm is not suitable for large-scale data classification, such as hyperspectral remote sensing image, because of its high computational complexity, and it is difficult to characterize the inherent uncertainty of the hyperspectral remote sensing image. This paper uses fuzzy anchors to process hyperspectral image classification and proposes a novel spectral clustering algorithm based on fuzzy similarity measure. The proposed algorithm utilizes the fuzzy similarity measure to obtain the similarity between the data points and the anchors, and then gets the similarity matrix. Finally, spectral clustering is performed on the similarity matrix to compute the classification results. The experimental results on the hyperspectral remote sensing image data sets have demonstrated the effectiveness of the proposed algorithm, and the introduction of fuzzy similarity measure gives rise to a more robust similarity matrix. Compared with existing methods, the proposed algorithm has a better classification result on the hyperspectral remote sensing image, and the kappa coefficient obtained by the proposed algorithm is 2% higher than the traditional algorithms.

การจำแนกข้อมูลขนาดใหญ่โดยใช้การจัดกลุ่มด้วยวิธีเคมีนและวิธีการเรียนรู้เชิงลึก

การจำแนกข้อมูลขนาดใหญ่โดยใช้การจัดกลุ่มด้วยวิธีเคมีนและวิธีการเรียนรู้เชิงลึก

A stochastic configuration network based on chaotic sparrow search algorithm

Stochastic configuration network (SCN), as a novel incremental generation model with supervisory mechanism, has an excellent superiority in solving large-scale data regression and classification problems. However, the accuracy of the SCN is affected by the assignation and selection of some network parameters significantly Sparrow search algorithm (SSA) is a new meta-heuristic algorithm that simulates the foraging and anti-predation behavior of sparrow population. In this paper, a stochastic configuration network based on chaotic sparrow search algorithm is first introduced, termed as CSSA-SCN. Firstly, chaotic sparrow search algorithm (CSSA) is designed which mainly utilizes logistic mapping, self-adaptive hyper-parameters, mutation operator to enhance the global optimization capability of SSA; Secondly, as the performance of SCN is related to regularization parameter r and scale factor λ of weights and biases, then CSSA is employed to give better parameters for SCN automatically; Finally, 13 benchmark functions and several datasets are used to evaluate the performance of CSSA and CSSA-SCN respectively. Experimental results demonstrate the feasibility and validity of CSSA-SCN compared with SCN and other contrast algorithms.

MapReduce-Based Improved Random Forest Model for Massive Educational Data Processing and Classification

This paper takes education data mining as the research theme, mine the existing massive education big data, compares the analysis methods of existing data models, and proposes an improved random forest reference model. The information gain of various features is calculated by introducing the feature weighting system, and the evaluation index is used to improve the existing data analysis. The simulation results show that the improved model is highly efficient as compared to the existing models for classification. In order to resolve the performance bottleneck of a single node in multiple data classification tasks in the era of big data, a classification and prediction model of graduates’ large-scale employment data, based on distributed improved RF algorithm, is proposed. The MapReduce distributed computing framework is used to complete the serial writing and deserialization loading of the training model between the local disk and the distributed file system, and realizing the distributed expansion of the large-scale data classification model based on the improved RF model.

Multi-category Classification Problem Oriented Subsampling-Based Active Learning Method

Traditional active learning methods have achieved gratifying results in the classification tasks of less categories such as binary classification, the application research of active learning in the field of big data problems still faces enormous challenges. Since many active learning query strategies need to perform matrix inversion, the amount of calculation increases exponentially with the increase of the scale of the problem, it is difficult to apply these active learning methods in large scale multi-category data classification task. In order to solve this problem, this paper designed a subsampling-based active learning model, and integrate unsupervised clustering algorithm with traditional active learning method, then conducted experiments on Binary Alphadigits and OMNIGLOT data sets. This paper compares the performance of five traditional active learning algorithms using this subsampling method, namely random sampling, uncertainty sampling, query-by-committee, density weighting and learning-based active learning. Through comparative experiments, the feasibility of active learning based on subsampling for solving the multi-category classification problem is verified, and it is found that the subsampling-based method can break the limitations of traditional active learning methods that cannot deal with large-scale data classification.

EFFICIENT LARGE-SCALE AIRBORNE LIDAR DATA CLASSIFICATION VIA FULLY CONVOLUTIONAL NETWORK

Abstract. Nowadays, we are witnessing an increasing availability of large-scale airborne LiDAR (Light Detection and Ranging) data, that greatly improve our knowledge of urban areas and natural environment. In order to extract useful information from these massive point clouds, appropriate data processing is required, including point cloud classification. In this paper we present a deep learning method to efficiently perform the classification of large-scale LiDAR data, ensuring a good trade-off between speed and accuracy. The algorithm employs the projection of the point cloud into a two-dimensional image, where every pixel stores height, intensity, and echo information of the point falling in the pixel. The image is then segmented by a Fully Convolutional Network (FCN), assigning a label to each pixel and, consequently, to the corresponding point. In particular, the proposed approach is applied to process a dataset of 7700 km2 that covers the entire Friuli Venezia Giulia region (Italy), allowing to distinguish among five classes (ground, vegetation, roof, overground and power line), with an overall accuracy of 92.9%.

High-Performance Machine Learning for Large-Scale Data Classification considering Class Imbalance

Currently, data classification is one of the most important ways to analysis data. However, along with the development of data collection, transmission, and storage technologies, the scale of the data has been sharply increased. Additionally, due to multiple classes and imbalanced data distribution in the dataset, the class imbalance issue is also gradually highlighted. The traditional machine learning algorithms lack of abilities for handling the aforementioned issues so that the classification efficiency and precision may be significantly impacted. Therefore, this paper presents an improved artificial neural network in enabling the high-performance classification for the imbalanced large volume data. Firstly, the Borderline-SMOTE (synthetic minority oversampling technique) algorithm is employed to balance the training dataset, which potentially aims at improving the training of the back propagation neural network (BPNN), and then, zero-mean, batch-normalization, and rectified linear unit (ReLU) are further employed to optimize the input layer and hidden layers of BPNN. At last, the ensemble learning-based parallelization of the improved BPNN is implemented using the Hadoop framework. Positive conclusions can be summarized according to the experimental results. Benefitting from Borderline-SMOTE, the imbalanced training dataset can be balanced, which improves the training performance and the classification accuracy. The improvements for the input layer and hidden layer also enhance the training performances in terms of convergence. The parallelization and the ensemble learning techniques enable BPNN to implement the high-performance large-scale data classification. The experimental results show the effectiveness of the presented classification algorithm.

Secure and Efficient k NN Classification for Industrial Internet of Things

The ${k}$ -nearest neighbors ( ${k}$ NN) classification has been widely used for defective product identification and anomaly detection in the Industrial Internet of Things (IIoT). In this article, we propose a secure and efficient distributed ${k}$ NN classification algorithm (SEED- ${k}$ NN) to prevent information and control flow exposure while supporting large-scale data classification on distributed servers. Specifically, we first design a secure and efficient vector homomorphic encryption (VHE) scheme by constructing a key-switching matrix and a noise matrix for data encryption. Based on the designed VHE, SEED- ${k}$ NN is proposed to efficiently achieve the confidentiality of data flow, ${k}$ NN query, and class label, while enabling homomorphic operations on the encrypted data. Moreover, by leveraging the Map/Reduce architecture, SEED- ${k}$ NN enables the ${k}$ NN classification over the large-scale encrypted data on distributed servers for industrial control systems. Finally, we demonstrate that SEED- ${k}$ NN achieves semantic security and high classification accuracy, and is applicable in IIoT due to its high efficiency.