Learning Process Of Neural Network Research Articles

Crowd Behavior Recognition Using Hybrid Tracking Model and Genetic algorithm Enabled Neural Network

In the current era, crowd behavior analysis is important topic due to the significance of video surveillance in the public area. Literature presents a handful of works for crowd behavior detection and analysis. Even though, the complicated challenges such as, low quality video, wide variation in the density of crowds and difficult motion patterns pose a complicated challenges for the researchers in crowd behavior detection. In order to alleviate these issues, we develop a crowd behavior detection system using hybrid tracking model and integrated features enabled neural network. The proposed crowd behavior detection system estimate the direction of movement of objects as well their activity using proposed GLM-based neural network. The proposed GLM-based neural network integrates the LM algorithm with genetic algorithm to improve the learning process of neural network. The performance of the proposed crowd behavior detection algorithm is validated with five different video and the performance is extensively analyzed using accuracy. From research outcome, we proved that the proposed system obtained the maximum accuracy of 95% which is higher than the existing methods taken for comparison.

Optimizing connection weights in neural networks using the whale optimization algorithm

The learning process of artificial neural networks is considered as one of the most difficult challenges in machine learning and has attracted many researchers recently. The main difficulty of training a neural network is the nonlinear nature and the unknown best set of main controlling parameters (weights and biases). The main disadvantages of the conventional training algorithms are local optima stagnation and slow convergence speed. This makes stochastic optimization algorithm reliable alternative to alleviate these drawbacks. This work proposes a new training algorithm based on the recently proposed whale optimization algorithm (WOA). It has been proved that this algorithm is able to solve a wide range of optimization problems and outperform the current algorithms. This motivated our attempts to benchmark its performance in training feedforward neural networks. For the first time in the literature, a set of 20 datasets with different levels of difficulty are chosen to test the proposed WOA-based trainer. The results are verified by comparisons with back-propagation algorithm and six evolutionary techniques. The qualitative and quantitative results prove that the proposed trainer is able to outperform the current algorithms on the majority of datasets in terms of both local optima avoidance and convergence speed.

Mimicking of pulse shape-dependent learning rules with a quantum dot memristor

We present the realization of four different learning rules with a quantum dot memristor by tuning the shape, the magnitude, the polarity and the timing of voltage pulses. The memristor displays a large maximum to minimum conductance ratio of about 57 000 at zero bias voltage. The high and low conductances correspond to different amounts of electrons localized in quantum dots, which can be successively raised or lowered by the timing and shapes of incoming voltage pulses. Modifications of the pulse shapes allow altering the conductance change in dependence on the time difference. Hence, we are able to mimic different learning processes in neural networks with a single device. In addition, the device performance under pulsed excitation is emulated combining the Landauer-Büttiker formalism with a dynamic model for the quantum dot charging, which allows explaining the whole spectrum of learning responses in terms of structural parameters that can be adjusted during fabrication, such as gating efficiencies and tunneling rates. The presented memristor may pave the way for future artificial synapses with a stimulus-dependent capability of learning.

Research and Simulation on an Improved Compression Reconstruction Algorithm of Multimedia Image

The traditional algorithm cannot simultaneously optimize the two indicators of high compression ratio and low information loss rate. To this end, a compression transmission method for multimedia image is raised based on improved neural network. Through the construction of image compression to transmit neural network mapping relation, and neural network was utilized to compress, transmit and reconstruct multimedia image, the learning process of neural network was improved also, by the introduction of game theory to make the corresponding parameters of compression and reconstruction self-adaptive change according to the results of the game, and study concurrently under the guidance of two objective optimization function, which improves the quality of image reconstruction after image compression and transmission. The experimental results show that, the improved algorithm can effectively improve the quality of reconstructed multimedia image after compression and transmission, the results is satisfactory. Keywords: Compression and transmission, image reconstruction, multimedia image, neural network.

Design of a heart rate controller for treadmill exercise using a recurrent fuzzy neural network

Background and objectiveIn this study, we developed a computer controlled treadmill system using a recurrent fuzzy neural network heart rate controller (RFNNHRC). Treadmill speeds and inclines were controlled by corresponding control servo motors. The RFNNHRC was used to generate the control signals to automatically control treadmill speed and incline to minimize the user heart rate deviations from a preset profile. MethodsThe RFNNHRC combines a fuzzy reasoning capability to accommodate uncertain information and an artificial recurrent neural network learning process that corrects for treadmill system nonlinearities and uncertainties. Treadmill speeds and inclines are controlled by the RFNNHRC to achieve minimal heart rate deviation from a pre-set profile using adjustable parameters and an on-line learning algorithm that provides robust performance against parameter variations. The on-line learning algorithm of RFNNHRC was developed and implemented using a dsPIC 30F4011 DSP. ResultsApplication of the proposed control scheme to heart rate responses of runners resulted in smaller fluctuations than those produced by using proportional integra control, and treadmill speeds and inclines were smoother. The present experiments demonstrate improved heart rate tracking performance with the proposed control scheme. ConclusionsThe RFNNHRC scheme with adjustable parameters and an on-line learning algorithm was applied to a computer controlled treadmill system with heart rate control during treadmill exercise. Novel RFNNHRC structure and controller stability analyses were introduced. The RFNNHRC were tuned using a Lyapunov function to ensure system stability. The superior heart rate control with the proposed RFNNHRC scheme was demonstrated with various pre-set heart rates.

AggNet: Deep Learning From Crowds for Mitosis Detection in Breast Cancer Histology Images.

The lack of publicly available ground-truth data has been identified as the major challenge for transferring recent developments in deep learning to the biomedical imaging domain. Though crowdsourcing has enabled annotation of large scale databases for real world images, its application for biomedical purposes requires a deeper understanding and hence, more precise definition of the actual annotation task. The fact that expert tasks are being outsourced to non-expert users may lead to noisy annotations introducing disagreement between users. Despite being a valuable resource for learning annotation models from crowdsourcing, conventional machine-learning methods may have difficulties dealing with noisy annotations during training. In this manuscript, we present a new concept for learning from crowds that handle data aggregation directly as part of the learning process of the convolutional neural network (CNN) via additional crowdsourcing layer (AggNet). Besides, we present an experimental study on learning from crowds designed to answer the following questions. 1) Can deep CNN be trained with data collected from crowdsourcing? 2) How to adapt the CNN to train on multiple types of annotation datasets (ground truth and crowd-based)? 3) How does the choice of annotation and aggregation affect the accuracy? Our experimental setup involved Annot8, a self-implemented web-platform based on Crowdflower API realizing image annotation tasks for a publicly available biomedical image database. Our results give valuable insights into the functionality of deep CNN learning from crowd annotations and prove the necessity of data aggregation integration.

Implementation of the RBF neural chip with the back-propagation algorithm for on-line learning

This article presents the hardware implementation of the floating-point processor (FPP) to develop the radial basis function (RBF) neural network for the general purpose of pattern recognition and nonlinear control. The floating-point processor is designed on a field programmable gate array (FPGA) chip to execute nonlinear functions required in the parallel calculation of the back-propagation algorithm. Internal weights of the RBF network are updated by the online learning back-propagation algorithm. The on-line learning process of the RBF chip is compared numerically with the results of the RBF neural network learning process written in the MATLAB program. The performance of the designed RBF neural chip is tested for the real-time pattern classification of the XOR logic. Performances are evaluated by comparing results from the MATLAB through extensive experimental studies.

English

The complexity of domain problem can slow or even hinder the learning process of neural networks. It is rather difficult to overcome such an obstacle because neural networks, as cited today in the literature, lack the interpretability of their internal structures. In this paper, we present a visualization approach capable of enhancing the understanding of neural networks. Our approach visualizes input and weight contributions, sensitivity analysis, and provides guidance in pruning less influential features and consequently reducing the complexity of domain problem while maintaining acceptable error rates. We conduct experiments on various datasets to show the effectiveness of our approach. Key words: Neural network, visualization, input contribution, sensitivity analysis

Combining additive input noise annealing and pattern transformations for improved handwritten character recognition

Two problems that burden the learning process of Artificial Neural Networks with Back Propagation are the need of building a full and representative learning data set, and the avoidance of stalling in local minima. Both problems seem to be closely related when working with the handwritten digits contained in the MNIST dataset. Using a modest sized ANN, the proposed combination of input data transformations enables the achievement of a test error as low as 0.43%, which is up to standard compared to other more complex neural architectures like Convolutional or Deep Neural Networks.

Intelligent complementary sliding-mode control with dead-zone parameter modification

This paper proposes an intelligent complementary sliding-mode control (ICSMC) system which is composed of a computed controller and a robust controller. The computed controller includes a neural dynamics estimator and the robust compensator is designed to prove a finite L2-gain property. The neural dynamics estimator uses a recurrent neural fuzzy inference network (RNFIN) to approximate the unknown system term in the sense of the Lyapunov function. In traditional neural network learning process, an over-trained neural network would force the parameters to drift and the system may become unstable eventually. To resolve this problem, a dead-zone parameter modification is proposed for the parameter tuning process to stop when tracking performance index is smaller than performance threshold. To investigate the capabilities of the proposed ICSMC approach, the ICSMC system is applied to a one-link robotic manipulator and a DC motor driver. The simulation and experimental results show that favorable control performance can be achieved in the sense of the L2-gain robust control approach by the proposed ICSMC scheme.

Prediction of water evaporation rate for indoor swimming hall using neural networks

The forecast of water evaporation rate is important in building and energy sectors. However, due to its stochastic nature and complexity, its forecast is rare in the literature. This paper presents a novel neural network approach to predicting water evaporation rate without occupant information for an indoor swimming hall containing five pools in Finland. Input sensitivity is analyzed and two step ahead predictions are compared. The neural networks using water evaporation rate and a binary representation form of time as inputs outperform other models. Experimental data show rapid fluctuations in water evaporation rate during operating hours although relatively stable during non-operating hours. The developed neural network model, however, is able to adapt to fluctuations and reaches good and acceptable accuracies for one- and two-step ahead predictions even for operating hours. The binary form of time simplifies learning process of neural networks. This paper demonstrates the capability of water evaporation rate forecasting without occupant information by neural networks, which might not be possible with traditional empirical models, and their positive impacts on promoting energy efficiency in various applications in general. Finally, the developed method is sufficiently general and can be extended to other systems for forecasting water evaporation rate as well.

A Numerical Methodology for the Multi-objective Optimization of the DI Diesel Engine Combustion

DI Diesel engine are widely used both for industrial and automotive applications due to their durability and fuel economy. Nonethe- less, increasing environmental concerns force that type of engine to comply with increasingly demanding emission limits, so that, it has become mandatory to develop a robust design methodology of the DI Diesel combustion system focused on reduction of soot and NOx simultaneously while maintaining a reasonable fuel economy. In recent years, genetic algorithms and CFD three-dimensional combustion simulations have been successfully applied to that kind of problem. However, combining GAs op- timization with actual CFD three-dimensional combustion simulations can be too onerous since a large number of calculations is usually needed for the genetic algorithm to converge, resulting in a high computational cost and, thus, limiting the suitability of this method for industrial processes. In order to make the optimization process less time-consuming, CFD simulations can be more conveniently used to generate a training set for the learning process of an artifical neural network which, once correctly trained, can be used to forecast the engine outputs as a function of the design parameters during a GA optimization performing a so-called virtual optimization.In the current paper, a numerical methodology for the multi-objective virtual optimization of the combustion of an automotive DI Diesel engine, which relies on artificial neural networks and genetic algorithms, was developed.

Sequential learning processes in neural networks applied as models of nonlinear systems

Sequential learning processes in neural networks applied as models of nonlinear systems

Mixture of Merged Gaussian Algorithm using RTDENN

Computer vision has been a widely developed research area in the last years, and it has been used for a broad range of applications, including surveillance systems. In the pursuit of an autonomous and smart motion detection system, a reliable segmentation algorithm is required. The main problems of present segmentation solutions are their high execution time and the lack of robustness against changes in the environment due to variations in lighting, shadows, occlusions or the movement of secondary objects. This paper proposes a new algorithm named Mixture of Merged Gaussian Algorithm (MMGA) that aims to achieve a substantial improvement in execution speed to enable real-time implementation, without compromising the reliability and accuracy of the segmentation. The MMGA is based on the combination of a probabilistic model for the background, similar to the Mixture of Gaussian Model (MGM), with the learning processes of Real-Time Dynamic Ellipsoidal Neural Networks (RTDENN) for the update of the model. The proposed algorithm has been tested for different videos and compared to the MGM and SDGM algorithms. Results show a reduction of 30 to 50 % in execution times. Furthermore, the segmentation is more robust against the effect of noise and adapts faster to lighting changes.

Adaptive learning with large variability of teaching signals for neural networks and its application to motion control of an industrial robot

Recently, various control methods represented by proportional-integral-derivative (PID) control are used for robotic control. To cope with the requirements for high response and precision, advanced feedforward controllers such as gravity compensator, Coriolis/centrifugal force compensator and friction compensators have been built in the controller. Generally, it causes heavy computational load when calculating the compensating value within a short sampling period. In this paper, integrated recurrent neural networks are applied as a feedforward controller for PUMA560 manipulator. The feedforward controller works instead of gravity and Coriolis/centrifugal force compensators. In the learning process of the neural network by using back propagation algorithm, the learning coefficient and gain of sigmoid function are tuned intuitively and empirically according to teaching signals. The tuning is complicated because it is being conducted by trial and error. Especially, when the scale of teaching signal is large, the problem becomes crucial. To cope with the problem which concerns the learning performance, a simple and adaptive learning technique for large scale teaching signals is proposed. The learning techniques and control effectiveness are evaluated through simulations using the dynamic model of PUMA560 manipulator.

Analysis of the reference open system interconnection model clustering of by MLP and RBF network toolkit

Algorithm of the classification with representing the set of functions’ taxons by the cortege, which implements the learning process of multilayer artificial neural networks (ANN) MLP and RBF, are proposed to apply for the reference open systems interconnection model (OSI RM). The quality criteria for the definition of the cortege are formed using the magnitudes that characterize the relative number of common parameters for each pair of OSI RM functions. By the experimental studies of the algo-rithm changing the thresholds values in the MLP and RBF ANN models’ constraints it is revealed a nonexisting in the classical OSI RM distribution of the functions between the hierarchical levels that stipulates the development of improved recommendations with the appearance of a new technologies

The removal of EMG in EEG by neural networks

In this paper, it is presented that electromyography (EMG) is a shot noise based on the generation of EMG. A novel filter is proposed by applying a neural network (NN) ensemble where the noisy input signal and the desired one are the same in a learning process. Both incremental and batch mode are applied in the learning process of NNs that is better than generalized NN filters. This NN ensemble filter not only reduces additive and multiplicative white noise inside signals, but also preserves the signals' characteristics. In clinical EEG and EMG signals processing, the filter is capable of reducing EMG in the clinical EEG, and it is proved that there is randomness in EMG.

Decision support system for predicting tsunami characteristics along coastline areas based on database modelling development

Tsunamis are extraordinary occurrences that are difficult to identify; most of the incidents have no recorded predictions and tsunamis are generally infrequent events with poor data acquisition. The development of a tsunami database system has become important for improving the management of information with regard to a tsunami early warning system for vulnerable communities along coastline areas. Numerical modelling is usually employed to simulate the wave height and travel time of a wave arriving at a coastline area. However, numerical modelling for tsunami prediction is too time-consuming to be useful as an early warning system for the mitigation of tsunami-related damage and loss. Therefore, this model was used to develop a tsunami database system, based on hypothetical data, in order to develop a recognition pattern for a neural network learning process that will improve the speed and accuracy of tsunami prediction. To improve the accuracy of numerical modelling and observation, an adjustment was established for an advanced training process which used a generalised regression neural network. In other words, the training and testing datasets were obtained by correcting near-field tsunami numerical models from hypothetical earthquakes. The case study was performed on part of the Southern Pangandaran coastline in West Java, Indonesia.

Multi-dimensional data construction method with its application to learning from small-sample-sets

Insufficient training data is one of the major problems in neural network learning, because it leads to poor learning performance. In order to enhance an intelligent learning process, it is necessary to exploit the features of the problem from the available information even with limited scale. Due to the shortcomings of the existing methods for data generation; and also in general, a problem is described by multiple attributes, this study has first extended the developed one-dimensional Data Construction Method (DCM) for virtual data generation to multidimensional continuous space as denoted by m-DCM. Then, sensitivity analysis and numerical illustration have been carried out. By incorporating m-DCM into a supervised neural network learning process, we have shown to overcome the existing unbounded and immeasurable problems and provided a better learning performance in a comparative manner.

Dynamics of chaotic systems with attractive and repulsive couplings

Together with attractive couplings, repulsive couplings play crucial roles in determining important evolutions in natural systems, such as in learning and oscillatory processes of neural networks. The complex interactions between them have great influence on the systems. A detailed understanding of the dynamical properties under this type of couplings is of practical significance. In this paper, we propose a model to investigate the dynamics of attractive and repulsive couplings, which give rise to rich phenomena, especially for amplitude death (AD). The relationship among various dynamics and possible transitions to AD are illustrated. When the system is in the maximally stable AD, we observe the transient behavior of in-phase (high frequency) and out-of-phase (low frequency) motions. The mechanism behind the phenomenon is given.