Single Hidden Layer Neural Network Research Articles

A Comparison of Various Algorithms for Classification of Food Scents Measured with an Ion Mobility Spectrometry.

The present aim was to compare the accuracy of several algorithms in classifying data collected from food scent samples. Measurements using an electronic nose (eNose) can be used for classification of different scents. An eNose was used to measure scent samples from seven food scent sources, both from an open plate and a sealed jar. The k-Nearest Neighbour (k-NN) classifier provides reasonable accuracy under certain conditions and uses traditionally the Euclidean distance for measuring the similarity of samples. Therefore, it was used as a baseline distance metric for the k-NN in this paper. Its classification accuracy was compared with the accuracies of the k-NN with 66 alternative distance metrics. In addition, 18 other classifiers were tested with raw eNose data. For each classifier various parameter settings were tried and compared. Overall, 304 different classifier variations were tested, which differed from each other in at least one parameter value. The results showed that Quadratic Discriminant Analysis, MLPClassifier, C-Support Vector Classification (SVC), and several different single hidden layer Neural Networks yielded lower misclassification rates applied to the raw data than k-NN with Euclidean distance. Both MLP Classifiers and SVC yielded misclassification rates of less than when applied to raw data. Furthermore, when applied both to the raw data and the data preprocessed by principal component analysis that explained at least or of the total variance in the raw data, Quadratic Discriminant Analysis outperformed the other classifiers. The findings of this study can be used for further algorithm development. They can also be used, for example, to improve the estimation of storage times of fruit.

Rogue wave solutions and the bright and dark solitons of the (3+1)-dimensional Jimbo–Miwa equation

It is well known that most classical test functions to solve nonlinear partial differential equations can be constructed via single hidden layer neural network model by using Bilinear Neural Network Method (BNNM). In this paper, the neural network model of test function for the (3+1)-dimensional Jimbo–Miwa equation is extended to the “4-2-3” model. By giving some specific activation functions, new test function is constructed to obtain analytical solutions of the (3+1)-dimensional Jimbo–Miwa equation. Rogue wave solutions and the bright and dark solitons are obtained by giving some specific parameters. Via curve plots, three-dimensional plots, contour plots and density plots, dynamical characteristics of these waves are exhibited.

A rapid neural network–based state of health estimation scheme for screening of end of life electric vehicle batteries

There is growing interest in recycling and re-use of electric vehicle batteries owing to their growing market share and use of high-value materials such as cobalt and nickel. To inform the subsequent applications at battery end of life, it is necessary to quantify their state of health. This study proposes an estimation scheme for the state of health of high-power lithium-ion batteries based on extraction of parameters from impedance data of 13 Nissan Leaf 2011 battery modules modelled by a modified Randles equivalent circuit model. Using the extracted parameters as predictors for the state of health, a baseline single hidden layer neural network was evaluated by root mean square and peak state of health prediction errors and refined using a Gaussian process optimisation procedure. The optimised neural network predicted state of health with a root mean square error of (1.729 ± 0.147)%, which is shown to be competitive with some of the most performant existing neural network–based state of health estimation schemes, and is expected to outperform the baseline model with ∼50 training samples. The use of equivalent circuit model parameters enables more in-depth analysis of the battery degradation state than many similar neural network–based schemes while maintaining similar accuracy despite a reduced dataset, while there is demonstrated potential for measurement times to be reduced to as little as 30 s with frequency targeting of the impedance measurements.

Predicting Mental Decline Rates in Mild Cognitive Impairment From Baseline MRI Volumetric Data.

In mild cognitive impairment (MCI), identifying individuals at high risk for progressive cognitive deterioration can be useful for prognostication and intervention. This study quantitatively characterizes cognitive decline rates in MCI and tests whether volumetric data from baseline magnetic resonance imaging (MRI) can predict accelerated cognitive decline. The authors retrospectively examined Alzheimer Disease Neuroimaging Initiative data to obtain serial Mini-Mental Status Exam (MMSE) scores, diagnoses, and the following baseline MRI volumes: total intracranial volume, whole-brain and ventricular volumes, and volumes of the hippocampus, entorhinal cortex, fusiform gyrus, and medial temporal lobe. Subjects with <24 months or <4 measurements of MMSE data were excluded. Predictive modeling of fast cognitive decline (defined as >0.6/year) from baseline volumetric data was performed on subjects with MCI using a single hidden layer neural network. Among 698 baseline MCI subjects, the median annual decline in the MMSE score was 1.3 for converters to dementia versus 0.11 for stable MCI (P<0.001). A 0.6/year threshold captured dementia conversion with 82% accuracy (sensitivity 79%, specificity 85%, area under the receiver operating characteristic curve 0.88). Regional volumes on baseline MRI predicted fast cognitive decline with a test accuracy of 71%. An MMSE score decrease of >0.6/year is associated with MCI-to-dementia conversion and can be predicted from baseline MRI.

The application of a neural network to predict hypotension and vasopressor requirements non-invasively in obstetric patients having spinal anesthesia for elective cesarean section (C/S)

BackgroundNeural networks are increasingly used to assess physiological processes or pathologies, as well as to predict the increased likelihood of an impending medical crisis, such as hypotension.MethodWe compared the capabilities of a single hidden layer neural network of 12 nodes to those of a discrete-feature discrimination approach with the goal being to predict the likelihood of a given patient developing significant hypotension under spinal anesthesia when undergoing a Cesarean section (C/S). Physiological input information was derived from a non-invasive blood pressure device (Caretaker [CT]) that utilizes a finger cuff to measure blood pressure and other hemodynamic parameters via pulse contour analysis. Receiver-operator-curve/area-under-curve analyses were used to compare performance.ResultsThe results presented here suggest that a neural network approach (Area Under Curve [AUC] = 0.89 [p < 0.001]), at least at the implementation level of a clinically relevant prediction algorithm, may be superior to a discrete feature quantification approach (AUC = 0.87 [p < 0.001]), providing implicit access to a plurality of features and combinations thereof. In addition, the expansion of the approach to include the submission of other physiological data signals, such as heart rate variability, to the network can be readily envisioned.ConclusionThis pilot study has demonstrated that increased coherence in Arterial Stiffness (AS) variability obtained from the pulse wave analysis of a continuous non-invasive blood pressure device appears to be an effective predictor of hypotension after spinal anesthesia in the obstetrics population undergoing C/S.This allowed us to predict specific dosing thresholds of phenylephrine required to maintain systolic blood pressure above 90 mmHg.

Aircraft parameter estimation using ELM network

Purpose The purpose of this paper is to investigate the estimation methodology with a highly generalized cost-effective single hidden layer neural network. Design/methodology/approach The aerodynamic parameter estimation is a challenging research area of aircraft system identification, which finds various applications such as flight control law design and flight simulators. With the availability of the large database, the data-driven methods have gained attention, which is primarily based on the nonlinear function approximation using artificial neural networks. A novel single hidden layer feed-forward neural network (FFNN) known as extreme learning machine (ELM), which overcomes the issues such as learning rate, number of epochs, local minima, generalization performance and computational cost, as encountered in the conventional gradient learning-based FFNN has been used for the nonlinear modeling of the aerodynamic forces and moments. A mathematical formulation based on the partial differentiation is proposed to estimate the aerodynamic parameters. Findings The real flight data of longitudinal and lateral-directional motion have been considered to estimate their respective aerodynamic parameters using the proposed methodology. The efficacy of the estimates is verified with the results obtained through the conventional parameter estimation methods such as the equation-error method and filter-error method. Originality/value The present study is an outcome of the research conducted on ELM for the estimation of aerodynamic parameters from the real flight data. The proposed method is capable to estimate the parameters in the presence of noise.

Modeling the Impedance Behavior of Ionic Conductors (AgPO&lt;sub&gt;3&lt;/sub&gt;)&lt;sub&gt;1-x&lt;/sub&gt;(Ag&lt;sub&gt;2&lt;/sub&gt;SO&lt;sub&gt;4&lt;/sub&gt;)&lt;sub&gt;x&lt;/sub&gt; Glass System Using Artificial Neural Network

The dielectric permittivity and conductivity of (AgPO₃)_1-x(Ag₂SO₄)_x compound was investigated at different concentrations of (Ag₂SO₄). The effect of concentration on AC conductivity and permittivity as well as temperature and frequency was investigated in order to model this behavior. Multidimensional mathematical models were as proposed to predict the impedance components <img src=image/15115401_01.gif> and the dielectric permittivity components <img src=image/15115401_02.gif> of the glass system as a function of temperatures, frequencies and concentrations. Artificial Neural Network (ANN) and nonlinear regression approaches were set as curve fitting techniques in order to construct models based on 1700 points of data. This model can be then used to predict these proprieties at any concentration and therefore helping the product designer to choose the proper mixing and temperature conditions. For ANN, 20, 50, and 100 nodes in a single hidden layer neural network were considered. Although data results of the two approaches showed a good alignment with experimental data, the ANN model with twenty nodes was able to predict the outputs with lower MSE values range from 0.008 to 0.012 for impedance and from 0.006 to 0.008 for dielectric losses than the regression technique. Moreover, R² values for the neural network were over 99% in both training and testing of impedance and dielectric permittivity while R² values for non-linear regression vary between 73.86% to 94.75%. The proposed ANN model can be of a great help to find the optimal dielectric permittivity and conductivity of (AgPO₃)_1-x(Ag₂SO₄)_x compound when dealing with a specific application.

Balinese glyph recognition with gabor filters

Recognizing Balinese glyphs from the Balinese script on palm leaf manuscripts is not trivial. In Balinese script, there are more than a hundred glyphs which represent basic syllables and compound syllables, and also some punctuation marks. They naturally share a strong interclass similarity between each other related to the form of their writing curves. The degraded image of textured palm leaf manuscript also offer some challenging parts in recognizing the Balinese glyph. In this paper, we investigated the use of Gabor filter bank as the feature extraction method to recognize the Balinese glyphs. By using Gabor filter, we can detect many texture variations with different orientations and frequencies. In our experiments, the published dataset of AMADI_LontarSet for glyph recognition was used. It showed a very promising result by using a single hidden layer Neural Network as the classifier. Gabor filters with Zoning method achieved a high enough recognition rate. For future works, Gabor filters will be analyzed in combination with the Histogram of Gradient, Neighborhood Pixel Weight and Kirsch Edges features.

An Improved LDA-Based ELM Classification for Intrusion Detection Algorithm in IoT Application.

The Internet of Things (IoT) is widely applied in modern human life, e.g., smart home and intelligent transportation. However, it is vulnerable to malicious attacks, and the current existing security mechanisms cannot completely protect the IoT. As a security technology, intrusion detection can defend IoT devices from most malicious attacks. However, unfortunately the traditional intrusion detection models have defects in terms of time efficiency and detection efficiency. Therefore, in this paper, we propose an improved linear discriminant analysis (LDA)-based extreme learning machine (ELM) classification for the intrusion detection algorithm (ILECA). First, we improve the linear discriminant analysis (LDA) and then use it to reduce the feature dimensions. Moreover, we use a single hidden layer neural network extreme learning machine (ELM) algorithm to classify the dimensionality-reduced data. Considering the high requirement of IoT devices for detection efficiency, our scheme not only ensures the accuracy of intrusion detection, but also improves the execution efficiency, which can quickly identify the intrusion. Finally, we conduct experiments on the NSL-KDD dataset. The evaluation results show that the proposed ILECA has good generalization and real-time characteristics, and the detection accuracy is up to 92.35%, which is better than other typical algorithms.

Spoken Language Identification Based on Particle Swarm Optimisation–Extreme Learning Machine Approach

The determination and classification of natural language based on specified content and data set involves a process known as spoken language identification (LID). To initiate the process, useful features of the given data need to be extracted first in a mature process where the standard LID features have been previously developed by employing the use of MFCC, SDC, GMM and the i-vector-based framework. Nevertheless, optimisation of the learning process is still required to enable a comprehensive capturing of the extracted features’ embedded knowledge. The training of a single hidden layer neural network can be done using the extreme learning machine (ELM), which is an effective learning model for conducting classification and regression analysis. Nevertheless, the learning process of this model is not entirely effective (i.e. optimised) due to the random selection of weights within the input hidden layer. This study employs ELM as the LID learning model centred upon the extraction of the standard features. The enhanced self-adjusting extreme learning machine (ESA–ELM) is one of the ELM’s optimisation techniques which has been chosen as the benchmark and is enhanced by adopting a new alternative optimisation approach (PSO) instead of (EATLBO) in terms of achieving high performance. The improved ESA–ELM is named particle swarm optimisation–extreme learning machine (PSO–ELM). The generated results are based on LID with the same benchmarked data set derived from eight languages, which indicated the superior performance of the particle swarm optimisation–extreme learning machine LID (PSO–ELM LID) with an accuracy of 98.75% in comparison with the ESA–ELM LID which only achieved 96.25%.

Efficient hinging hyperplanes neural network and its application in nonlinear system identification

In this paper, the efficient hinging hyperplanes (EHH) neural network is proposed, which is basically a single hidden layer neural network. Different from the dominant single hidden layer neural networks, the hidden layer in the EHH neural network can be seen as a directed acyclic graph (DAG) and all the nodes in the DAG contribute to the output. It is proved that for every EHH neural network, there is an equivalent adaptive hinging hyperplanes (AHH) tree, the model of which was proposed based on the hinging hyperplanes (HH) model and finds good applications in system identification. Analog to the proof for the AHH model, the universal approximation ability of the EHH neural network is provided. Different from other neural networks, the EHH neural network has interpretability, which can be easily obtained through its ANOVA decomposition (or interaction matrix). The interpretability can then be used as an indication for the importance of the input variables. The construction of the EHH neural network includes initial network generation and parameter optimization (including the structure and weights parameter optimization). A descent algorithm for searching the locally optimal EHH neural network is proposed and the worst-case complexity of the algorithm is also provided. The EHH neural network is applied in nonlinear system identification, the simulation results show that satisfactory accuracy can be achieved with relatively low computational cost, and at the same time, some insights into the importance of the regressors and the interactions among the regressors can be revealed.

High-accuracy motion control of a motor servo system with dead-zone based on a single hidden layer neural network

High-accuracy motion control of a motor servo system with dead-zone based on a single hidden layer neural network

Convexifying Sparse Interpolation With Infinitely Wide Neural Networks: An Atomic Norm Approach

This work examines the problem of exact data interpolation via sparse (neuron count), infinitely wide, single hidden layer neural networks with leaky rectified linear unit activations. Using the atomic norm framework of [Chandrasekaran et al., 2012], we derive simple characterizations of the convex hulls of the corresponding atomic sets for this problem under several different constraints on the weights and biases of the network, thus obtaining equivalent convex formulations for these problems. A modest extension of our proposed framework to a binary classification problem is also presented. We explore the efficacy of the resulting formulations experimentally, and compare with networks trained via gradient descent.

Expressive Numbers of Two or More Hidden Layer ReLU Neural Networks

One of the reasons why neural networks are used in machine learning is their high power, that is, the ability to express functions. Expressive power of neural networks depends on its structures and is measured by some indices. In this paper, we focus on one of these measures named expressive number, which is based on the number of data that can be expressed. Expressive numbers enable us to see whether the size of a neural network is suitable for the given training data before we conduct machine learning. However, existing works on numbers mainly target single hidden layer neural networks, and little is known about those with two or more hidden layers. In this paper, we give a lower bound of the maximum number of two hidden layer neural networks and an upper bound of that of multilayer neural networks with ReLU activation function. This result shows the number of two hidden layer neural networks is in O(a_1a_2) where a_1 and a_2 are the numbers of each hidden layer's neurons.

Output feedback fault-tolerant control for a class of nonlinear systems via dynamic gain and neural network

In this paper, by combining the dynamic gain and the self-adaptive neural network, an output feedback fault-tolerant control method was proposed for a class of nonlinear uncertain systems with actuator faults. First, the dynamic gain was introduced and the coordinate transformation of the state variables of the system was performed to design the corresponding state observers. Then, the observer-based output feedback controller was designed through the back-stepping method. The output feedback control method based on the dynamic gain can solve the adaptive fault-tolerant control problem when there are simple nonlinear functions with uncertain parameters in the system. For the more complex uncertain nonlinear functions in the system, in this paper, a single hidden layer neural network was used for compensation and the fault-tolerant control was realized by combining the dynamic gain. Finally, the height and posture control system of the unmanned aerial vehicle with actuator faults was taken as an example to verify the effectiveness of the proposed method.

Extreme Learning Machine-Based Deep Model for Human Activity Recognition With Wearable Sensors

Human activity recognition (HAR) is a main research field of context-aware computing; the performance of HAR mainly depends on the feature extraction method and classification algorithm. Extreme learning machine (ELM) is a single hidden layer neural network, which has better classification and generalization ability. However, ELM is not suitable for feature extraction. Deep learning is a hot research field as it can automatically extract significant features from raw data. In this paper, we propose an approach: an ELM-based deep model, which combined convolutional neural network (CNN), multilayer ELM (ML-ELM) as feature extractor, and used kernel ELM (KELM) as classifier. We used CNN and ML-ELM to extract significant features, and used KELM to achieve stable performance. The performance of proposed approach is validated on two public HAR datasets, and the experimental results show that the proposed approach is effective.

Spoken language identification based on optimised genetic algorithm–extreme learning machine approach

The determination and classification of a recognized spoken language based on certain contents and datasets is known as the process of language identification (LID). The common process in carrying out LID entails the mandatory processing of data which enables the extraction of the necessary features for the process. The extraction involves a mature process whereby the development of the standard LID features have been conducted much earlier by means of a mel-frequency cepstral coefficients, shifted delta cepstral, Gaussian mixture model and i-vector-based framework. Despite that, improvement or rather optimisation still needs to be done on the learning process based on the extracted features so as to obtain all the knowledge embedded within them. The classification and regression analysis can benefit tremendously from the use of the extreme learning machine (ELM) which is a particularly effective and useful learning model for training a single-hidden layer neural network. However, owing to the randomly selected weights embedded in the input’s hidden layers, the model’s learning process is rendered to be ineffective or not optimised in its entirety. In this study, the ELM is employed as the learning model for LID due to the standard feature extraction. In addition, this study proposes a new optimised genetic algorithm (OGA) with three different selection criteria (i.e., roulette wheel, K-tournament and random) to select the appropriate initial weights and biases of the input hidden layer of the ELM, thereby minimising the classification error and improving the general performance of the ELM for LID. Results show the excellent performance of the proposed OGA–ELM with three different selection criteria, namely, roulette wheel, K-tournament and random, with the highest accuracies of 99.50%, 100% and 99.38%, respectively.

A robust multilayer extreme learning machine using kernel risk-sensitive loss criterion

More recently, extreme learning machine (ELM) has emerged as a novel computing paradigm that enables the neural network (NN) based learning to be achieved with fast training speed and good generalization performance. However, the single hidden layer NN using ELM may be not effective in addressing some large-scale problems with more computational efforts. To avoid such limitation, we utilize the multilayer ELM architecture in this article to reduce the computational complexity, without the physical memory limitation. Meanwhile, it is known to us all that there are a lot of noises in the practical applications, and the traditional ELM may not perform well in this instance. Considering the existence of noises or outliers in training dataset, we develop a more practical approach by incorporating the kernel risk-sensitive loss (KRSL) criterion into ELM, on the basis of the efficient performance surface of KRSL with high accuracy while still maintaining the robustness to outliers. A robust multilayer ELM, i.e., the stacked ELM using the minimum KRSL criterion (SELM-MKRSL), is accordingly proposed in this article to enhance the outlier robustness on large-scale and complicated dataset. The simulation results on some synthetic datasets indicate that the proposed approach SELM-MKRSL can achieve higher classification accuracy and is more robust to the noises compared with other state-of-the-art algorithms related to multilayer ELM.

Computing the Approximation Error for Neural Networks with Weights Varying on Fixed Directions

We obtain a sharp lower bound estimate for the approximation error of a continuous function by single hidden layer neural networks with a continuous activation function and weights varying on two fixed directions. We show that for a certain class of activation functions this lower bound estimate turns into equality. The obtained result provides us with a method for direct computation of the approximation error. As an application, we give a formula, which can be used to compute instantly the approximation error for a class of functions having second order partial derivatives.

Anomaly detecting and ranking of the cloud computing platform by multi-view learning

Anomaly detecting as an important technical in cloud computing is applied to support smooth running of the cloud platform. Traditional detecting methods based on statistic, analysis, etc. lead to the high false-alarm rate due to non-adaptive and sensitive parameters setting. We presented an online model for anomaly detecting using machine learning theory. However, most existing methods based on machine learning linked all features from difference sub-systems into a long feature vector directly, which is difficult to both exploit the complement information between sub-systems and ignore multi-view features enhancing the classification performance. Aiming to this problem, the proposed method automatic fuses multi-view features and optimize the discriminative model to enhance the accuracy. This model takes advantage of extreme learning machine (ELM) to improve detection efficiency. ELM is the single hidden layer neural network, which is transforming iterative solution the output weights to solution of linear equations and avoiding the local optimal solution. Moreover, we rank anomies according to the relationship between samples and the classification boundary, and then assigning weights for ranked anomalies, retraining the classification model finally. Our method exploits the complement information between sub-systems sufficiently, and avoids the influence from imbalance dataset, therefore, deal with various challenges from the cloud computing platform. We deploy the privately cloud platform by Openstack, verifying the proposed model and comparing results to the state-of-the-art methods with better efficiency and simplicity.