Training Neural Network Classifiers Research Articles

Artificial neural network and discrete wavelet transform for inter-turn short circuit and broken rotor bars faults diagnosis under various operating conditions

Introduction. This work presents a methodology for detecting inter-turn short circuit (ITSC) and broken rotor bars (BRB) fault in variable speed induction machine controlled by field oriented control. If any of these faults are not detected at an early stage, it may cause an unexpected shutdown of the industrial processes and significant financial losses. Purpose. For these reasons, it is important to develop a new diagnostic system to detect in a precautionary way the ITSC and BRB at various load condition. We propose the application of discrete wavelet transform to overcome the limitation of traditional technique for no-stationary signals. The novelty of the work consists in developing a diagnosis system that combines the advantages of both the discrete wavelet transform (DWT) and artificial neural network (ANN) to identify and diagnose defects, related to both ITSC and BRB faults. Methods. The suggested method involves analyzing the electromagnetic torque signal using DWT to calculate the stored energy at each level of decomposition. Then, this energy is applied to train neural network classifier. The accuracy of ANN based on DWT, was improved by testing different orthogonal wavelet functions on simulated signal. The selection process identified 5 pertinent wavelet energies, concluding that, Daubechies44 (db44) is the best suitable mother wavelet function for effectively detecting and classifying failures in machines. Results. We applied numerical simulations by MATLAB/Simulink software to demonstrate the validity of the suggested techniques in a closed loop induction motor drive. The obtained results prove that this method can identify and classify these types of faults under various loads of the machine. References 31, table 1, figures 9.

COMIRE: A Consistence-Based Mislabeled Instances Removal Method.

Training neural network classifiers (NNCs) usually requires all instances to be correctly labeled, which is difficult and/or expensive to satisfy in some practical applications. When label noise is present, mislabeled data will severely mislead the training of NNCs, resulting in poor generalization performance. In this work, we address the label noise issue by removing mislabeled instances from the training data. A COnsistence-based Mislabeled Instances REmoval (COMIRE) method is proposed. The main idea is based on the observation that during the training of the NNC, the training loss and the model's prediction uncertainty of correctly labeled instances show similar trends, while those of mislabeled instances have quite different trends. Thus, the consistency between the two trends can be used to distinguish correctly labeled instances from mislabeled ones. On this basis, an iteration scheme is introduced to further increase the separability between the two types of data. Experimental results show that COMIRE can effectively identify the mislabeled instances. Moreover, the classification performance is significantly improved after removing the identified instances from the noisy training data.

Development and Analysis of a CNN- and Transfer-Learning-Based Classification Model for Automated Dairy Cow Feeding Behavior Recognition from Accelerometer Data

Due to technological developments, wearable sensors for monitoring the behavior of farm animals have become cheaper, have a longer lifespan and are more accessible for small farms and researchers. In addition, advancements in deep machine learning methods provide new opportunities for behavior recognition. However, the combination of the new electronics and algorithms are rarely used in PLF, and their possibilities and limitations are not well-studied. In this study, a CNN-based model for the feeding behavior classification of dairy cows was trained, and the training process was analyzed considering a training dataset and the use of transfer learning. Commercial acceleration measuring tags, which were connected by BLE, were fitted to cow collars in a research barn. Based on a dataset including 33.7 cow × days (21 cows recorded during 1-3 days) of labeled data and an additional free-access dataset with similar acceleration data, a classifier with F1 = 93.9% was developed. The optimal classification window size was 90 s. In addition, the influence of the training dataset size on the classifier accuracy was analyzed for different neural networks using the transfer learning technique. While the size of the training dataset was being increased, the rate of the accuracy improvement decreased. Beginning from a specific point, the use of additional training data can be impractical. A relatively high accuracy was achieved with few training data when the classifier was trained using randomly initialized model weights, and a higher accuracy was achieved when transfer learning was used. These findings can be used for the estimation of the necessary dataset size for training neural network classifiers intended for other environments and conditions.

Classifier for the functional state of the respiratory system via descriptors determined by using multimodal technology

Currently, intelligent systems built on a multimodal basis are used to study the functional state of living objects. Its essence lies in the fact that a decision is made through several independent information channels with the subsequent aggregation of these decisions. The method of forming descriptors for classifiers of the functional state of the respiratory system includes the study of the spectral range of the respiratory rhythm and the construction of the wavelet plane of the monitoring electrocardiosignal overlapping this range. Then, variations in the breathing rhythm are determined along the corresponding lines of the wavelet plane. Its analysis makes it possible to select slow waves corresponding to the breathing rhythm and systemic waves of the second order. Analysis of the spectral characteristics of these waves makes it possible to form a space of informative features for classifiers of the functional state of the respiratory system. To construct classifiers of the functional state of the respiratory system, hierarchical classifiers were used. As an example, we took a group of patients with pneumonia with a well-defined diagnosis (radiography, X-ray tomography, laboratory data) and a group of volunteers without pulmonary pathology. The diagnostic sensitivity of the obtained classifier was 76% specificity with a diagnostic specificity of 82%, which is comparable to the results of X-ray studies. It is shown that the corresponding lines of the wavelet planes are correlated with the respiratory system and, using their Fourier analysis, descriptors can be obtained for training neural network classifiers of the functional state of the respiratory system.

A new density core graph-cut class decomposition to improve neural network classification performance

<span>This research presents a new pre-processed class decomposition technique called density core graph-cut (DCGC). The method uses supervised clustering instead of a traditional unsupervised one to decompose the class. Supervised clustering requires additional label information to function and with that it gains a better understanding of the distribution. DCGC employs nearest neighbors to form a density core graph for each class. Then, the edges of each graph to be removed or cut is identified utilizing class information. Lastly, it yields final clusters by grouping all connected cores and assigning the remaining samples to a cluster where the nearest core belongs. Training neural network classifiers on complex label data will yield a better accuracy with the modified class representation. Intuitively, the decision boundaries separating classes based on the modified labels are less complex, and classifiers’ chance to reach these hyperplanes is higher. The results present that training neural networks using label representations from DCGC significantly helps improve the classification accuracy of neural networks on syntactic datasets as high as 30%. For real-world problems, the experiment presents a mixed result in which some datasets moderately benefit from DCGC.</span>

CIRAL: a hybrid active learning framework for plankon taxa labeling

Abstract With the complex structure of planktonic species and an immense amount of data captured from autonomous underwater vehicles (AUVs), a large burden is placed on the domain experts for plankton taxa labeling. At the same time, the most prominent machine learning (ML) methods for classification rely heavily on a massive amount of labeled datasets to create and train neural network classifier models that perform their tasks accurately. Active Learning (AL) is an ML paradigm that reduces this manual effort by proposing algorithms that support the construction of the training datasets, thus enlarging the sets while minimizing human involvement. To build the training set, AL methods apply heuristics to select a subset of images, i.e., samples, from the entire data. The selected samples that capture the common statistical patterns or feature space are likely to include all the information needed for the training and the learning processes. In addition, the algorithm should prioritize samples that are likely belonging to multiple classes, i.e., having close inter-class boundaries, and might lead to model confusion. Many of the current AL approaches fail to incorporate both types of samples representing the statistical pattern and the samples in which the particular machine learning model is uncertain about. In this paper, we extend our framework which addresses these challenges with an augmentation module to increase the robustness of the model and ensure its adaptability to the planktonic domain. We compare the framework with existing hybrid AL techniques and test an adaption of our extended framework on the planktonic domain. The empirical results from the experiments exerted in this paper confirm higher accuracy achieved by the new extended framework.

Machine Learning Pipelines with Modern Big Data Tools for High Energy Physics

The effective utilization at scale of complex machine learning (ML) techniques for HEP use cases poses several technological challenges, most importantly on the actual implementation of dedicated end-to-end data pipelines. A solution to these challenges is presented, which allows training neural network classifiers using solutions from the Big Data and data science ecosystems, integrated with tools, software, and platforms common in the HEP environment. In particular, Apache Spark is exploited for data preparation and feature engineering, running the corresponding (Python) code interactively on Jupyter notebooks. Key integrations and libraries that make Spark capable of ingesting data stored using ROOT format and accessed via the XRootD protocol, are described and discussed. Training of the neural network models, defined using the Keras API, is performed in a distributed fashion on Spark clusters by using BigDL with Analytics Zoo and also by using TensorFlow, notably for distributed training on CPU and GPU resourcess. The implementation and the results of the distributed training are described in detail in this work.

Training neural network classifiers through Bayes risk minimization applying unidimensional Parzen windows

A new training algorithm for neural networks in binary classification problems is presented. It is based on the minimization of an estimate of the Bayes risk by using Parzen windows applied to the final one-dimensional nonlinear transformation of the samples to estimate the probability of classification error. This leads to a very general approach to error minimization and training, where the risk that is to be minimized is defined in terms of integrated one-dimensional Parzen windows, and the gradient descent algorithm used to minimize this risk is a function of the window that is used. By relaxing the constraints that are typically applied to Parzen windows when used for probability density function estimation, for example by allowing them to be non-symmetric or possibly infinite in duration, an entirely new set of training algorithms emerge. In particular, different Parzen windows lead to different cost functions, and some interesting relationships with classical training methods are discovered. Experiments with synthetic and real benchmark datasets show that with the appropriate choice of window, fitted to the specific problem, it is possible to improve the performance of neural network classifiers over those that are trained using classical methods.

Arctic Sea Ice Characterization Using RISAT-1 Compact-Pol SAR Imagery and Feature Evaluation: A Case Study Over Northeast Greenland

Synthetic Aperture Radar (SAR) polarimetry has become a valuable tool in space-borne SAR-based sea ice analysis. The two major objectives in SAR-based remote sensing of sea ice are, on the one hand, to have a large coverage and, on the other hand, to obtain a radar response that carries as much information as possible in order to characterize sea ice. Single-polarimetric acquisitions of existing sensors offer a wide coverage on the ground, whereas dual polarimetric or even better fully polarimetric data offer a higher information content, which allows for a more reliable automated sea ice analysis at a cost of smaller swath. In order to reconcile the advantages of fully polarimetric acquisitions with the higher ground coverage of acquisitions with fewer polarimetric channels, hybrid/compact polarimetric acquisitions offer an excellent tradeoff between the mentioned objectives. With the advent of the RISAT-1 satellite platform, we are able to explore the potential of compact dual pol acquisitions for sea ice analysis and classification. Our algorithmic approach for an automated sea ice classification consist of two steps. In the first step, we perform a feature extraction followed by a feature evaluation procedure. The resulting feature vectors are then ingested into a trained artificial neural network classifier to arrive at a pixel-wise supervised classification. We present a comprehensive polarimetric feature analysis and classification results on a dataset acquired off the eastern Greenland coast, along with comparisons of results obtained from near-coincident (spatially and temporally) C -band fully polarimetric imagery acquired by RADARSAT-2.

Face recognition from a single image per person using deep architecture neural networks

Implementing an accurate face recognition system requires images in different variations, and if our database is large, we suffer from problems such as storing cost and low speed in recognition algorithms. On the other hand, in some applications there is only one image available per person for training recognition model. In this article, we propose a neural network model inspired of bidirectional analysis and synthesis brain network which can learn nonlinear mapping between image space and components space. Using a deep neural network model, we have tried to separate pose components from person ones. After setting apart these components, we can use them to synthesis virtual images of test data in different pose and lighting conditions. These virtual images are used to train neural network classifier. The results showed that training neural classifier with virtual images gives better performance than training classifier with frontal view images.

Proto-object categorisation and local gist vision using low-level spatial features

Object categorisation is a research area with significant challenges, especially in conditions with bad lighting, occlusions, different poses and similar objects. This makes systems that rely on precise information unable to perform efficiently, like a robotic arm that needs to know which objects it can reach. We propose a biologically inspired object detection and categorisation framework that relies on robust low-level object shape. Using only edge conspicuity and disparity features for scene figure-ground segregation and object categorisation, a trained neural network classifier can quickly categorise broad object families and consequently bootstrap a low-level scene gist system. We argue that similar processing is possibly located in the parietal pathway leading to the LIP cortex and, via areas V5/MT and MST, providing useful information to the superior colliculus for eye and head control.

MULTIPLE-TARGET CLASSIFICATION AND TRACKING FOR MOBILE ROBOTS USING A 2D LASER RANGE SCANNER

In human–robot interaction developments, detection, tracking and identification of moving objects (DATMO) constitute an important problem. More specifically, in mobile robots this problem becomes harder and more computationally expensive as the environments become dynamic and more densely populated. The problem can be divided into a number of sub-problems, which include the compensation of the robot's motion, measurement clustering, feature extraction, data association, targets' trajectory estimation and finally, target classification. Here, a mobile robot uses 2D laser range data to identify and track moving targets. A Joint Probabilistic Data Association with Interacting Multiple Model (JPDA-IMM) tracking algorithm associates the available laser data to track and provide an estimated state vector of targets' position and velocity. Potential moving objects are initially learned in a supervised manner and later on are autonomously classified in real-time using a trained Fuzzy ART neural network classifier. The recognized targets are fed back to the tracker to further improve the track initiation process. The resulting technique introduces a computationally efficient approach to already existing target-tracking and identification research, which is especially suited for real time application scenarios.

Retinal image analysis aimed at blood vessel tree segmentation and early detection of neural-layer deterioration

An automatic method of segmenting the retinal vessel tree and estimating status of retinal neural fibre layer (NFL) from high resolution fundus camera images is presented. First, reliable blood vessel segmentation, using 2D directional matched filtering, enables to remove areas occluded by blood vessels thus leaving remaining retinal area available to the following NFL detection. The local existence of rather faint and hardly visible NFL is detected by combining several newly designed local textural features, sensitive to subtle NFL characteristics, into feature vectors submitted to a trained neural-network classifier. Obtained binary retinal maps of NFL distribution show a good agreement with both medical expert evaluations and quantitative results obtained by optical coherence tomography.

Decoding subtle forearm flexions using fractal features of surface electromyogram from single and multiple sensors

BackgroundIdentifying finger and wrist flexion based actions using a single channel surface electromyogram (sEMG) can lead to a number of applications such as sEMG based controllers for near elbow amputees, human computer interface (HCI) devices for elderly and for defence personnel. These are currently infeasible because classification of sEMG is unreliable when the level of muscle contraction is low and there are multiple active muscles. The presence of noise and cross-talk from closely located and simultaneously active muscles is exaggerated when muscles are weakly active such as during sustained wrist and finger flexion. This paper reports the use of fractal properties of sEMG to reliably identify individual wrist and finger flexion, overcoming the earlier shortcomings.MethodsSEMG signal was recorded when the participant maintained pre-specified wrist and finger flexion movements for a period of time. Various established sEMG signal parameters such as root mean square (RMS), Mean absolute value (MAV), Variance (VAR) and Waveform length (WL) and the proposed fractal features: fractal dimension (FD) and maximum fractal length (MFL) were computed. Multi-variant analysis of variance (MANOVA) was conducted to determine the p value, indicative of the significance of the relationships between each of these parameters with the wrist and finger flexions. Classification accuracy was also computed using the trained artificial neural network (ANN) classifier to decode the desired subtle movements.ResultsThe results indicate that the p value for the proposed feature set consisting of FD and MFL of single channel sEMG was 0.0001 while that of various combinations of the five established features ranged between 0.009 - 0.0172. From the accuracy of classification by the ANN, the average accuracy in identifying the wrist and finger flexions using the proposed feature set of single channel sEMG was 90%, while the average accuracy when using a combination of other features ranged between 58% and 73%.ConclusionsThe results show that the MFL and FD of a single channel sEMG recorded from the forearm can be used to accurately identify a set of finger and wrist flexions even when the muscle activity is very weak. A comparison with other features demonstrates that this feature set offers a dramatic improvement in the accuracy of identification of the wrist and finger movements. It is proposed that such a system could be used to control a prosthetic hand or for a human computer interface.

Training neural network classifiers for medical decision making: The effects of imbalanced datasets on classification performance

This study investigates the effect of class imbalance in training data when developing neural network classifiers for computer-aided medical diagnosis. The investigation is performed in the presence of other characteristics that are typical among medical data, namely small training sample size, large number of features, and correlations between features. Two methods of neural network training are explored: classical backpropagation (BP) and particle swarm optimization (PSO) with clinically relevant training criteria. An experimental study is performed using simulated data and the conclusions are further validated on real clinical data for breast cancer diagnosis. The results show that classifier performance deteriorates with even modest class imbalance in the training data. Further, it is shown that BP is generally preferable over PSO for imbalanced training data especially with small data sample and large number of features. Finally, it is shown that there is no clear preference between oversampling and no compensation approach and some guidance is provided regarding a proper selection.

Optimizing the Area Under a Receiver Operating Characteristic Curve With Application to Landmine Detection

A common approach to training neural network classifiers in a supervised learning setting is to minimize the mean-square error (mse) between the network output for each labeled training sample and some desired output. In the context of landmine detection and discrimination, although the performance of an algorithm is correlated with the mse, it is ultimately evaluated by using receiver operating characteristic (ROC) curves. In general, the larger the area under the ROC curve (AUC), the better. We present a new method for maximizing the AUC. Desirable properties of the proposed algorithm are derived and discussed that differentiate it from previously proposed algorithms. A hypothesis test is used to compare the proposed algorithm to an existing algorithm. The false alarm rate achieved by the proposed algorithm is found to be less than that of the existing algorithm with 95% confidence

Revisiting squared-error and cross-entropy functions for training neural network classifiers

This paper investigates the efficacy of cross-entropy and square-error objective functions used in training feed-forward neural networks to estimate posterior probabilities. Previous research has found no appreciable difference between neural network classifiers trained using cross-entropy or squared-error. The approach employed here, though, shows cross-entropy has significant, practical advantages over squared-error.

An empirical evaluation of Bayesian sampling with hybrid Monte Carlo for training neural network classifiers

This article gives a concise overview of Bayesian sampling for neural networks, and then presents an extensive evaluation on a set of various benchmark classification problems. The main objective is to study the sensitivity of this scheme to changes in the prior distribution of the parameters and hyperparameters, and to evaluate the efficiency of the so-called automatic relevance determination (ARD) method. The article concludes with a comparison of the achieved classification results with those obtained with (i) the evidence scheme and (ii) with non-Bayesian methods.

Learning criteria for training neural network classifiers

This paper presents a study of two learning criteria and two approaches to using them for training neural network classifiers, specifically a Multi-Layer Perceptron (MLP) and Radial Basis Function (RBF) networks. The first approach, which is a traditional one, relies on the use of two popular learning criteria, i.e. learning via minimising a Mean Squared Error (MSE) function or a Cross Entropy (CE) function. It is shown that the two criteria have different charcteristics in learning speed and outlier effects, and that this approach does not necessarily result in a minimal classification error. To be suitable for classification tasks, in our second approach an empirical classification criterion is introduced for the testing process while using the MSE or CE function for the training. Experimental results on several benchmarks indicate that the second approach, compared with the first, leads to an improved generalisation performance, and that the use of the CE function, compared with the MSE function, gives a faster training speed and improved or equal generalisation performance.

Two algorithms for neural-network design and training with application to channel equalization

We describe two algorithms for designing and training neural-network classifiers. The first, the linear programming slab algorithm (LPSA), is motivated by the problem of reconstructing digital signals corrupted by passage through a dispersive channel and by additive noise. It constructs a multilayer perceptron (MLP) to separate two disjoint sets by using linear programming methods to identify network parameters. The second, the perceptron learning slab algorithm (PLSA), avoids the computational costs of linear programming by using an error-correction approach to identify parameters. Both algorithms operate in highly constrained parameter spaces and are able to exploit symmetry in the classification problem. Using these algorithms, we develop a number of procedures for the adaptive equalization of a complex linear 4-quadrature amplitude modulation (QAM) channel, and compare their performance in a simulation study. Results are given for both stationary and time-varying channels, the latter based on the COST 207 GSM propagation model.