Volume Of Training Data Research Articles

Cloud-Based Parallel Machine Learning for Tool Wear Prediction

The emergence of cloud computing, industrial internet of things (IIoT), and new machine learning techniques have shown the potential to advance prognostics and health management (PHM) in smart manufacturing. While model-based PHM techniques provide insight into the progression of faults in mechanical components, certain assumptions on the underlying physical mechanisms for fault development are required to develop predictive models. In situations where there is a lack of adequate prior knowledge of the underlying physics, data-driven PHM techniques have been increasingly applied in the field of smart manufacturing. One of the limitations of current data-driven methods is that large volumes of training data are required to make accurate predictions. Consequently, computational efficiency remains a primary challenge, especially when large volumes of sensor-generated data need to be processed in real-time applications. The objective of this research is to introduce a cloud-based parallel machine learning algorithm that is capable of training large-scale predictive models more efficiently. The random forests (RFs) algorithm is parallelized using the MapReduce data processing scheme. The MapReduce-based parallel random forests (PRFs) algorithm is implemented on a scalable cloud computing system with varying combinations of processors and memories. The effectiveness of this new method is demonstrated using condition monitoring data collected from milling experiments. By implementing RFs in parallel on the cloud, a significant increase in the processing speed (14.7 times in terms of increase in training time) has been achieved, with a high prediction accuracy of tool wear (eight times in terms of reduction in mean squared error (MSE)).

Improving Label Noise Filtering by Exploiting Unlabeled Data

With the significant growth in the scale of data, an increasing amount of training data is available in many machine learning tasks. However, it is difficult to ensure perfect labeling with a large volume of training data. Some labels can be incorrect, resulting in label noise, which could lead to deterioration in learning performance. A common way to address label noise is to apply noise filtering techniques to identify and remove noise prior to learning. Multiple noise filtering approaches have been proposed. However, almost all existing works focus on only mislabeled training data and ignore the existence of unlabeled data. In fact, unlabeled data are common in many applications, and their values have been extensively studied and recognized. Therefore, in this paper, we explore the effective use of unlabeled data to improve the noise filtering performance. To this end, we propose a novel noise filtering algorithm called enhanced soft majority voting by exploiting unlabeled data (ESMVU), which is an ensemble-learning-based filter that adopts a soft majority voting strategy. ESMVU provides a systematic way to measure the value of unlabeled data by considering different aspects, such as label confidence and the sample distribution. Finally, the effectiveness of the proposed method is confirmed by experiments and comparison with other methods.

Interactive Visual Decision Tree for Developing Detection Rules of Attacks on Web Applications

Creating detection rules of attacks on web applications is not a trivial task, especially when the attacks are launched by experienced hackers. In such a situation, human expertise is essential to produce effective results. However, human users are easily overloaded by the huge input data, which is meant to be analyzed, learned from, and used to develop appropriate detection rules. To support human users in dealing with the information overload problem while developing detection rules of web application attacks, we propose a novel technique and tool called Interactive Visual Decision Tree (IVDT). IVDT is a variant of the popular decision tree learning technique introduced in research fields such as machine learning and data mining, with two additionally important features: visually supported data analysis and user-guided tree growing. Visually supported data analysis helps human users cope with high volume of training data while analyzing each node in the tree being built. On the other hand, user-guided tree growing allows human users to apply their own expertise and experience to create custom split condition for each tree node. A prototype implementation of IVDT is built and experimented to evaluate its effectiveness in terms of detection accuracy achieved by its users as well as ease of working with. The experiment results prove some advantages of IVDT over traditional decision tree learning method, but also point out its problems that should be handled in future improvements.

Fine-grained recognition of plants from images

BackgroundFine-grained recognition of plants from images is a challenging computer vision task, due to the diverse appearance and complex structure of plants, high intra-class variability and small inter-class differences. We review the state-of-the-art and discuss plant recognition tasks, from identification of plants from specific plant organs to general plant recognition “in the wild”.ResultsWe propose texture analysis and deep learning methods for different plant recognition tasks. The methods are evaluated and compared them to the state-of-the-art. Texture analysis is only applied to images with unambiguous segmentation (bark and leaf recognition), whereas CNNs are only applied when sufficiently large datasets are available. The results provide an insight in the complexity of different plant recognition tasks. The proposed methods outperform the state-of-the-art in leaf and bark classification and achieve very competitive results in plant recognition “in the wild”.ConclusionsThe results suggest that recognition of segmented leaves is practically a solved problem, when high volumes of training data are available. The generality and higher capacity of state-of-the-art CNNs makes them suitable for plant recognition “in the wild” where the views on plant organs or plants vary significantly and the difficulty is increased by occlusions and background clutter.

Anomaly detection and localization in crowded scenes using connected component analysis

This paper presents a new method for detecting abnormal events in the surveillance systems. This method does not employ object detection or tracking and thus it does not fail in the crowded scenes. In the first step of proposed method, the appropriate cell size is determined by calculating the prevalent size of the connected components. Then, the redundant information is eliminated and the important regions are extracted from training data. This preprocessing significantly reduces the volume of training data in the learning phase. Next, using the HOG descriptors and a multivariate Gaussian model, the appearance anomalies i.e. the abnormality in terms of physical characteristics is detected. Besides that, a simple algorithm is provided to detect the abnormal motion using the average optical flow of the cells. Experimental results on the UCSD-PED2 datasets show that the proposed method can reliably detect abnormal events in video sequences, outperforming the current state-of-the-art methods.

Deep learning in remote sensing scene classification: a data augmentation enhanced convolutional neural network framework

The recent emergence of deep learning for characterizing complex patterns in remote sensing imagery reveals its high potential to address some classic challenges in this domain, e.g. scene classification. Typical deep learning models require extremely large datasets with rich contents to train a multilayer structure in order to capture the essential features of scenes. Compared with the benchmark datasets used in popular deep learning frameworks, however, the volumes of available remote sensing datasets are particularly limited, which have restricted deep learning methods from achieving full performance gains. In order to address this fundamental problem, this article introduces a methodology to not only enhance the volume and completeness of training data for any remote sensing datasets, but also exploit the enhanced datasets to train a deep convolutional neural network that achieves state-of-the-art scene classification performance. Specifically, we propose to enhance any original dataset by applying three operations – flip, translation, and rotation to generate augmented data – and use the augmented dataset to train and obtain a more descriptive deep model. The proposed methodology is validated in three recently released remote sensing datasets, and confirmed as an effective technique that significantly contributes to potentially revolutionary changes in remote sensing scene classification, empowered by deep learning.

Accurate Periocular Recognition Under Less Constrained Environment Using Semantics-Assisted Convolutional Neural Network

Accurate biometric identification under real environments is one of the most critical and challenging tasks to meet growing demand for higher security. This paper proposes a new framework to efficiently and accurately match periocular images that are automatically acquired under less-constrained environments. Our framework, referred to as semantics-assisted convolutional neural networks (SCNNs) in this paper, incorporates explicit semantic information to automatically recover comprehensive periocular features. This strategy enables superior matching accuracy with the usage of relatively smaller number of training samples, which is often an issue with several biometrics. Our reproducible experimental results on four different publicly available databases suggest that the SCNN-based periocular recognition approach can achieve outperforming results, both in achievable accuracy and matching time, for less-constrained periocular matching. Additional experimental results presented in this paper also indicate that the effectiveness of proposed SCNN architecture is not only limited to periocular recognition but it can also be useful for generalized image classification. Without increasing the volume of training data, the SCNN is able to automatically extract more discriminative features from the input data than a single CNN, therefore can consistently improve the recognition performance. The experimental results presented in this paper validate such an approach to enable faster and more accurate periocular recognition under less constrained environments.

Improving deep neural network design with new text data representations

Using traditional machine learning approaches, there is no single feature engineering solution for all text mining and learning tasks. Thus, researchers must determine and implement the best feature engineering approach for each text classification task; however, deep learning allows us to skip this step by extracting and learning high-level features automatically from low-level text representations. Convolutional neural networks, a popular type of neural network for deep learning, have been shown to be effective at performing feature extraction and classification for many domains including text. Recently, it was demonstrated that convolutional neural networks can be used to train classifiers from character-level representations of text. This approach achieved superior performance compared to classifiers trained on word-level text representations, likely due to the use of character-level representations preserving more information from the data. Training neural networks from character level data requires a large volume of instances; however, the large volume of training data and model complexity makes training these networks a slow and computationally expensive task. In this paper, we propose a new method of creating character-level representations of text to reduce the computational costs associated with training a deep convolutional neural network. We demonstrate that our method of character embedding greatly reduces training time and memory use, while significantly improving classification performance. Additionally, we show that our proposed embedding can be used with padded convolutional layers to enable the use of current convolutional network architectures, while still facilitating faster training and higher performance than the previous approach for learning from character-level text.

Practicability study on the suitability of artificial, neural networks for the approximation of unknown steering torques

ABSTRACTFor steer-by-wire systems, the steering feedback must be generated artificially due to the system characteristics. Classical control concepts require operating-point driven optimisations as well as increased calibration efforts in order to adequately simulate the steering torque in all driving states. Artificial neural networks (ANNs) are an innovative control concept; they are capable of learning arbitrary non-linear correlations without complex knowledge of physical dependencies. The present study investigates the suitability of neural networks for approximating unknown steering torques. To ensure robust processing of arbitrary data, network training with a sufficient volume of training data is required, that represents the relation between the input and target values in a wide range. The data were recorded in the course of various test drives. In this research, a variety of network topologies were trained, analysed and evaluated. Though the fundamental suitability of ANNs for the present control task was demonstrated.

Enhancing quality of statistic monitoring models by training set design with active learning approach

The objective of this paper is to enhance the quality of the process monitoring models by designing a training set through an active learning approach. Although conventional process monitoring models are effective in many manufacturing processes, these models falter when confronted by a set of training data with poor quality or a small volume of training data. As the limitations of the monitoring models become increasingly obvious in face of even more complex manufacturing processes, in this work, the active learning process monitoring (AL-PM) model is developed. To design a good training set, Gaussian process (GP) models are first used to construct the relationships between the score variables of the latent structure model and the designable process variables because the GP model is capable of providing the accurate predictive mean and variance. The variance can quantify its prediction uncertainty. Second, the uncertainty index is presented and utilized to adequately explore for which regions the new data samples should be used to enhance the quality of the monitoring model. The proposed AL-PM model can be applied to any types of latent structure-based monitoring models. Its effectiveness and promising results have been demonstrated by its applications to a numerical example and a penicillin benchmark process.

Differential privacy in telco big data platform

Differential privacy (DP) has been widely explored in academia recently but less so in industry possibly due to its strong privacy guarantee. This paper makes the first attempt to implement three basic DP architectures in the deployed telecommunication (telco) big data platform for data mining applications. We find that all DP architectures have less than 5% loss of prediction accuracy when the weak privacy guarantee is adopted (e.g., privacy budget parameter ε ≥ 3). However, when the strong privacy guarantee is assumed (e.g., privacy budget parameter ε ≤ 0:1), all DP architectures lead to 15% ~ 30% accuracy loss, which implies that real-word industrial data mining systems cannot work well under such a strong privacy guarantee recommended by previous research works. Among the three basic DP architectures, the Hybridized DM (Data Mining) and DB (Database) architecture performs the best because of its complicated privacy protection design for the specific data mining algorithm. Through extensive experiments on big data, we also observe that the accuracy loss increases by increasing the variety of features, but decreases by increasing the volume of training data. Therefore, to make DP practically usable in large-scale industrial systems, our observations suggest that we may explore three possible research directions in future: (1) Relaxing the privacy guarantee (e.g., increasing privacy budget ε) and studying its effectiveness on specific industrial applications; (2) Designing specific privacy scheme for specific data mining algorithms; and (3) Using large volume of data but with low variety for training the classification models.

On Vectorization of Deep Convolutional Neural Networks for Vision Tasks

We recently have witnessed many ground-breaking results in machine learning and computer vision, generated by using deep convolutional neural networks (CNN). While the success mainly stems from the large volume of training data and the deep network architectures, the vector processing hardware (e.g. GPU) undisputedly plays a vital role in modern CNN implementations to support massive computation. Though much attention was paid in the extent literature to understand the algorithmic side of deep CNN, little research was dedicated to the vectorization for scaling up CNNs. In this paper, we studied the vectorization process of key building blocks in deep CNNs, in order to better understand and facilitate parallel implementation. Key steps in training and testing deep CNNs are abstracted as matrix and vector operators, upon which parallelism can be easily achieved. We developed and compared six implementations with various degrees of vectorization with which we illustrated the impact of vectorization on the speed of model training and testing. Besides, a unified CNN framework for both high-level and low-level vision tasks is provided, along with a vectorized Matlab implementation with state-of-the-art speed performance.

Distributed Extreme Learning Machine with kernels based on MapReduce

Extreme Learning Machine (ELM) has shown its good generalization performance and extremely fast learning speed in many learning applications. Recently, it has been proved that ELM outperforms Support Vector Machine (SVM) with less constraints from the optimization point of view. ELM provides unified learning schemes with a widespread type of feature mappings. Among these unified algorithms, ELM with kernels applies kernels instead of random feature mappings. However, with the exponentially increasing volume of training data in massive learning applications, centralized ELM with kernels suffers from the great memory consumption of large matrix operations. Besides, due to the high communication cost, some of these matrix operations cannot be directly implemented on shared-nothing distributed computing model like MapReduce. This paper proposes a distributed solution named Distributed Kernelized ELM (DK-ELM), which realizes an implementation of ELM with kernels on MapReduce. Distributed kernel matrix calculation and multiplication of matrix with vector are also applied to realize parallel calculation of DK-ELM. Extensive experiments on massive datasets are conducted to verify both the scalability and training performance of DK-ELM. Experimental results show that DK-ELM has good scalability for massive learning applications.

Efficient classification using parallel and scalable compressed model and its application on intrusion detection

In order to achieve high efficiency of classification in intrusion detection, a compressed model is proposed in this paper which combines horizontal compression with vertical compression. OneR is utilized as horizontal compression for attribute reduction, and affinity propagation is employed as vertical compression to select small representative exemplars from large training data. As to be able to computationally compress the larger volume of training data with scalability, MapReduce based parallelization approach is then implemented and evaluated for each step of the model compression process abovementioned, on which common but efficient classification methods can be directly used. Experimental application study on two publicly available datasets of intrusion detection, KDD99 and CMDC2012, demonstrates that the classification using the compressed model proposed can effectively speed up the detection procedure at up to 184 times, most importantly at the cost of a minimal accuracy difference with less than 1% on average.

Estimation of Subsurface Temperatures in the Tattapani Geothermal Field, Central India, from Limited Volume of Magnetotelluric Data and Borehole Thermograms Using a Constructive Back-Propagation Neural Network

Abstract A constructive back-propagation code that was designed to run as a single-hidden-layer, feed-forward neural network (SLFFNN) has been adapted and used to estimate subsurface temperature from a small volume of magnetotelluric (MT)-derived electrical resistivity data and borehole thermograms. The code was adapted to use a looping procedure in searching for better initialization conditions that can optimally solve nonlinear problems using the random weight initialization approach. Available one-dimensional (1D) MT-derived resistivity data and borehole temperature records from the Tattapani geothermal field, central India, were collated and digitized at 10-m intervals. The two datasets were paired to form a set of input–output pairs. The paired data were randomized, standardized, and partitioned into three mutually exclusive subsets. The various subsets had 52% (later increased to 61%), 30%, and 18% (later reduced to 9%) for training, validation, and testing, respectively, in the first and second training phases. The second training phase was meant to assess the influence of the training data volume on network performance. Standard statistical techniques including adjusted coefficient of determination (R2a), relative error (ɛ), absolute average deviation (AAD), root-mean-square error (RMSE), and regression analysis were used to quantitatively rate network performance. A manually designed two-hidden-layer, feed-forward network with 20 and 15 neurons in the first and second layers was also adopted in solving the same problem. Performance ratings were observed to be 0.97, 3.75, 4.09, 1.41, 1.18, and 1.08 for R2a, AAD, ɛ, RMSE, slope, and intercept, respectively, compared to an ɛ of 20.33 observed with the manually designed network. The SLFFNN is thus a structurally flexible network that performs better in spite of the small volume of data used in testing the network. The network needs to be tested further.

Decision Support System to Diagnosis and Classification of Epilepsy in Children

Clinical decision support systems play an important role in organizations. They have a tight relation with the information systems. Our goal is to develop a system to support the diagnosis and the classification of epilepsy in children. Around 50 million people in the world have epilepsy. Epilepsy diagnosis can be an extremely complex process, demanding considerable time and effort from physicians and healthcare infrastructures. Exams such as electroencephalograms and magnetic resonances are often used to create a more accurate diagnosis in a short amount of time. After the diagnosis process, physicians classify epilepsy according to the International Classification of Diseases, ninth revision (ICD-9). Physicians need to classify each specific type of epilepsy based on different data, e.g., types of seizures, events and exams' results. The classification process is time consuming and, in some cases, demands for complementary exams. This work presents a text mining approach to support medical decisions relating to epilepsy diagnosis and ICD-9-based classification in children. We put forward a text mining approach using electronically processed medical records, and apply the K-Nearest Neighbor technique as a white-box multiclass classifier approach to classify each instance, mapping it to the corresponding ICD-9-based standard code. Results on real medical records suggest that the proposed framework shows good performance and clear interpretations, albeit the reduced volume of available training data. To overcome this hurdle, in this work we also propose and explore ways of expanding the dataset.

INCREMENTAL DEVELOPMENT OF FAULT PREDICTION MODELS

The identification of fault-prone modules has a significant impact on software quality assurance. In addition to prediction accuracy, one of the most important goals is to detect fault prone modules as early as possible in the development lifecycle. Requirements, design, and code metrics have been successfully used for predicting fault-prone modules. In this paper, we investigate the benefits of the incremental development of software fault prediction models. We compare the performance of these models as the volume of data and their life cycle origin (design, code, or their combination) evolve during project development. We analyze 14 data sets from publicly available software engineering data repositories. These data sets offer both design and code metrics. Using a number of modeling techniques and statistical significance tests, we confirm that increasing the volume of training data improves model performance. Further models built from code metrics typically outperform those that are built using design metrics only. However, both types of models prove to be useful as they can be constructed in different phases of the life cycle. Code-based models can be used to increase the effectiveness of assigning verification and validation activities late in the development life cycle. We also conclude that models that utilize a combination of design and code level metrics outperform models which use either one metric set exclusively.

Visual learning and classification of human epithelial type 2 cell images through spontaneous activity patterns

Identifying the presence of anti-nuclear antibody (ANA) in human epithelial type 2 (HEp-2) cells via the indirect immunofluorescence (IIF) protocol is commonly used to diagnose various connective tissue diseases in clinical pathology tests. As it is a labour and time intensive diagnostic process, several computer aided diagnostic (CAD) systems have been proposed. However, the existing CAD systems suffer from numerous shortcomings due to the selection of features, which is commonly based on expert experience. Such a choice of features may not work well when the CAD systems are retasked to another dataset. To address this, in our previous work, we proposed a novel approach that learns a set of filters from HEp-2 cell images. It is inspired by the receptive fields in the mammalian's vision system, since the receptive fields can be thought as a set of filters for similar shapes. We obtain robust filters for HEp-2 cell classification by employing the independent component analysis (ICA) framework. Although, this approach may be held back due to one particular problem; ICA learning requires a sufficiently large volume of training data which is not always available. In this paper, we demonstrate a biologically inspired solution to address this issue via the use of spontaneous activity patterns (SAP). The spontaneous activity patterns, which are related to the spontaneous neural activities initialised by the chemical release in the brain, are found as the typical stimuli for the visual cell development of newborn animals. In the classification system for HEp-2 cells, we propose to model SAP as a set of small image patches containing randomly positioned Gaussian spots. The SAP image patches are generated and mixed with the training images in order to learn filters via the ICA framework. The obtained filters are adopted to extract the set of responses from a HEp-2 cell image. We then employ regions from this set of responses and stack them into “cubic regions”, and apply a classification based on the correlation information of the features. We show that applying the additional SAP leads to a better classification performance on HEp-2 cell images compared to using only the existing patterns for training ICA filters. The improvement on classification is particularly significant when there are not enough specimen images available in the training set, as SAP adds more variations to the existing data that makes the learned ICA model more robust. We show that the proposed approach consistently outperforms three recently proposed CAD systems on two publicly available datasets: ICPR HEp-2 contest and SNPHEp-2.

Neural network predictions of acoustical parameters in multi-purpose performance halls

A detailed binaural sound measurement was carried out in two multi-purpose performance halls of different seating capacities and designs in Hong Kong in the present study. The effectiveness of using neural network in the predictions of the acoustical properties using a limited number of measurement points was examined. The root-mean-square deviation from measurements, statistical parameter distribution matching, and the results of a t-test for vanishing mean difference between simulations and measurements were adopted as the evaluation criteria for the neural network performance. The audience locations relative to the sound source were used as the inputs to the neural network. Results show that the neural network training scheme using nine uniformly located measurement points in each specific hall area is the best choice regardless of the hall setting and design. It is also found that the neural network prediction of hall spaciousness does not require a large amount of training data, but the accuracy of the reverberance related parameter predictions increases with increasing volume of training data.

ICD9-based Text Mining Approach to Children Epilepsy Classification

According with the World Health Organization, around 50 million people in the world have epilepsy. After the diagnosis process, physicians classify epilepsy according to the International Classification of Diseases, Ninth Revision (ICD-9). Often exams as electroencephalograms and magnetic resonances are used to create a more accurate diagnosis in a short amount of time. The classification process is time consuming and demands the realization of complementary exams. To circumvent this laborious process we propose an automatic process of classifying epileptic diagnoses based on ICD-9. We put forward a text mining approach, using processed electronic medical records and a K-Nearest Neighbor is applied as a white-box multi classifier approach to classify each instance mapping into the corresponding standard code. Results suggests a good performance proposing a diagnosis from electronic medical records, despite of the reduced volume of available training data.