Circular Backpropagation Research Articles

Brain signal based human emotion analysis by circular back propagation and Deep Kohonen Neural Networks

Human emotion analysis is one of the challenging tasks in today's scenario. The success rate of human emotion recognition has high implication in practical applications such as Human Machine Interaction, anomaly detection, surveillance, etc. Artificial Neural Networks (ANN) is one of the highly favored computational intelligence techniques for human emotion recognition. However, the performance of traditional ANN is not satisfactory in case of applications such as human emotion analysis. This leads to the necessity of modified ANN with better performance than the conventional systems. In this paper, we propose Circular Back Propagation Neural Network (CBPN) and Deep Kohonen Neural Network (DKNN) to overcome drawbacks of the traditional neural networks regarding computational complexity and accuracy. Performance of the proposals is explored in classifying different emotions of humans using Electroencephalography (EEG) signals. It has been validated that the proposals have better performance than the related methods.

Circular-ELM for the reduced-reference assessment of perceived image quality

Providing a satisfactory visual experience is one of the main goals for present-day electronic multimedia devices. All the enabling technologies for storage, transmission, compression, rendering should preserve, and possibly enhance, the quality of the video signal; to do so, quality control mechanisms are required. These mechanisms rely on systems that can assess the visual quality of the incoming signal consistently with human perception. Computational Intelligence (CI) paradigms represent a suitable technology to tackle this challenging problem. The present research introduces an augmented version of the basic Extreme Learning Machine (ELM), the Circular-ELM (C-ELM), which proves effective in addressing the visual quality assessment problem. The C-ELM model derives from the original Circular BackPropagation (CBP) architecture, in which the input vector of a conventional MultiLayer Perceptron (MLP) is augmented by one additional dimension, the circular input; this paper shows that C-ELM can actually benefit from the enhancement provided by the circular input without losing any of the fruitful properties that characterize the basic ELM framework. In the proposed framework, C-ELM handles the actual mapping of visual signals into quality scores, successfully reproducing perceptual mechanisms. Its effectiveness is proved on recognized benchmarks and for four different types of distortions.

The build of n-Bits Binary Coding ICBP Ensemble System

To optimize the construct of an ensemble of many relatively small and simple classifiers may be more realistic than to optimize the design of an individual large and complex classifier. Problems of local minima and slow convergence may be mitigated within an ensemble system where the decisions obtained from locally optimal components classifiers are integrated together. However, it is very difficult to design the structure of individual neural networks (NNs) in an ensemble and the architecture of a whole ensemble. In n-Bits Binary Coding ICBP Ensemble System ( nBBC-ICBP-ES), the crucial parameter that is required to be set a priori is an appropriate number of hidden nodes of the corresponding improved circular back-propagation (ICBP) root model. Thereby, both the number of individual ICBPs and the architecture of each ICBP component in an nBBC-ICBP-ES can be decided straightly. nBBC-ICBP-ES is computationally more efficient, with relatively fewer user specified parameters, and does not need any manual division to the training data set for the purpose of its construction. It is rather easy to be understood and implemented, while inheriting the benefits of ICBP root model in a natural manner. Simulation and t-test results on four large-scale benchmark classification data sets demonstrated that, in most cases, nBBC-ICBP-ES significantly improved the classification and generalization performances of the two typically large single ICBPs, Same- 2 N h -ICBP-ES (i.e., an ensemble system with the same 2 N h ICBP components), conventional Bagging and AdaBoost ensemble. We conclude that, for NN applications in pattern recognition, assembling many small NNs might be better than just utilizing an individual large one, and further, assembling many heterogeneous small NNs might be better than assembling many homogeneous ones. The proposed nBBC-ICBP-ES is simple but efficient and effective, and also potentially significant for the applications of NN ensemble.

No-Reference JPEG image quality assessment based on Visual sensitivity

No-Reference JPEG image quality assessment based on Visual sensitivity

The build of a dynamic classifier selection ICBP system and its application to pattern recognition

In neural network ensemble, the diversity of its constitutive component networks is a crucial factor to boost its generalization performance. In terms of how each ensemble system solves the problem, we can roughly categorize the existing ensemble mechanism into two groups: data-driven and model-driven ensembles. The former engenders diversity to ensemble members by manipulating the data, while the latter realizes ensemble diversity by manipulating the component models themselves. Within a neural network ensemble, standard back-propagation (BP) networks are usually used as a base component. However, in this article, we will use our previously designed improved circular back-propagation (ICBP) neural network to establish such an ensemble. ICBP differentiates from BP network not only because an extra anisotropic input node is added, but also more importantly, because of the introduction of the extra node, it possesses an interesting property apart from the BP network, i.e., just through directly assigning different sets of values 1 and −1 to the weights connecting the extra node to all the hidden nodes, we can construct a set of heterogeneous ICBP networks with different hidden layer activation functions, among which we select four typical heterogeneous ICBPs to build a dynamic classifier selection ICBP system (DCS-ICBP). The system falls into the category of model-driven ensemble. The aim of this article is to explore the relationship between the explicitly constructed ensemble and the diversity scale, and further to verify feasibility and effectiveness of the system on classification problems through empirical study. Experimental results on seven benchmark classification tasks show that our DCS-ICBP outperforms each individual ICBP classifier and surpasses the performance of combination of ICBP using the majority voting technique, i.e. majority voting ICBP system (MVICBP). The successful simulation results validate that in DCS-ICBP we provide a new constructive method for diversity enforcement for ICBP ensemble systems.

Integrating the improved CBP model with kernel SOM

In this paper, we first design a more generalized network model, Improved circular back propagation (CBP), based on the same structure as CBP proposed by Ridella et al. The novelty of Improved circular back propagation neural network (ICBP) lies in: (1) it substitutes the original extra added node with the isotropic quadratic form input in CBP with the one with an anisotropic quadratic form input; (2) particularly, the weights between the extra node and all the hidden nodes are endowed fixed values instead of the original changeable values. As a result, ICBP possesses better generalization and adaptability although it has less adjustable weights compared to CBP. Secondly, we propose a new kernel-based self-organizing maps (KSOM) algorithm using the kernel method. Our main motives of using the kernel method are (1) to induce a class of robust non-Euclidean distance measures for the original input space and establish a new objective functions for self-organizing maps (SOM), and thus make the newly established SOM able to cluster the non-Euclidean structures in data; (2) to enhance robustness of the SOM algorithms to noise and outliers and at the same time still retain computational simplicity. And then, with the combined BP-SOM idea of Weijters, we construct a new integrated network ICBP–KSOM. Our motivation of presenting the integration is to construct a high-performance classifier by utilizing both ICBP's good generalization and adaptability and KSOM's higher classification performance and robustness comparing to SOM. Finally, the experimental results on three benchmark data sets show the superiority and effectiveness of our new integration in terms of the t-test.

Discounted least squares-improved circular back-propogation neural networks with applications in time series prediction

As a generalization of the multi-layer perceptron (MLP), the circular back-propagation neural network (CBP) possesses better adaptability. An improved version of the CBP (the ICBP) is presented in this paper. Despite having less adjustable weights, the ICBP has better adaptability than the CBP, which quite equals the famous Occam’s razor principle for model selection. In its application to time series, considering both structural changes and correlations of time series itself, we introduce the principle of the discounted least squares (DLS) in CBP and ICBP, respectively, and investigate their predicting capacity further. Introduction of DLS improves the predicting performance of both on a benchmark time series data set. Finally, the comparison of experimental results shows that ICBP with DLS (DLS-ICBP) has better predicting performance than DLS-CBP.

Neural networks for the no-reference assessment of perceived quality

Imaging algorithms often require reliable methods to evaluate the quality effects of the visual artifacts that digital processing brings about. We adopt a no-reference objective method for predicting the perceived quality of images in a deterministic fashion. JPEG coding provides a significant and interesting case study. To enhance the coherence of the quality estimates with respect to the empirical evidence of the perceptual phenomenon, the system parameters are adjusted using subjective scores obtained from human assessors. Principal component analysis is first used to assemble a set of objective features that best characterize the information in image data. Then a neural network, based on the circular backpropagation (CBP) model, associates the selected features with the corresponding predictions of quality ratings and reproduces the scores process of human assessors. The neural model enables one to decouple the process of feature selection from the task of mapping features into a quality score. Results on a public database for an image-quality experiment involving JPEG-compressed images and comparisons with existing objective methods confirm the approach effectiveness.

Objective quality assessment of displayed images by using neural networks

Considerable research effort is being devoted to the development of image-enhancement algorithms, which improve the quality of displayed digital pictures. Reliable methods for measuring perceived image quality are needed to evaluate the performances of those algorithms, and such measurements require a univariant (i.e., no-reference) approach. The system presented in this paper applies concepts derived from computational intelligence, and supports an objective quality-assessment method based on a circular back-propagation (CBP) neural model. The network is trained to predict quality ratings, as scored by human assessors, from numerical features that characterize images. As such, the method aims at reproducing perceived image quality, rather than defining a comprehensive model of the human visual system. The connectionist approach allows one to decouple the task of feature selection from the consequent mapping of features into an objective quality score. Experimental results on the perceptual effects of a family of contrast-enhancement algorithms confirm the method effectiveness, as the system renders quite accurately the image quality perceived by human assessors.

Chained DLS-ICBP Neural Networks with Multiple Steps Time Series Prediction

Based on the circular back-propagation (CBP) network, the improved circular back-propagation (ICBP) neural network was previously put forward and exhibits more general architecture than the former. It has a favorable characteristic that ICBP is better than CBP in generalization and adaptation though the number of its adaptable weights is generally less than that of CBP. The forecasting experiments on chaotic time series, multiple-input multiple-output (MIMO) systems and the data sets of daily life water consumed quantity have proved that ICBP has better capabilities of prediction and approximation than CBP. But in the above predicting process, ICBP neglects inherent structural changes and time correlation in time series themselves. In other words, they do not take into account the influence of different distances between observations and the predicting point on forecasting performance. The principle of discounted least-square (DLS) formulates this influence exactly. In this paper, the DLS principle is borrowed to construct the learning algorithm of DLS-ICBP. On this basis we construct chained DLS-ICBP neural networks by combining a new kind of chain structure to DLS-ICBP and investigate multiple steps time series prediction. We prove that DLS-ICBP has better single and multiple step predictive capabilities than ICBP through experiments on the data sets of Benchmarks and water consumed quantity.

Improved CBP Neural Network Model with Applications in Time Series Prediction

Circular back-propagation neural network (CBP) put forward by Sandro Ridella and Stefano Rovetta, a generalized model of multi-layer perceptron (MLP), possesses strong capabilities of generalization and adaptation to unknown inputs. And they can flexibly construct vector quantization (VQ) and radial basis function (RBF) networks under the CBP framework. With the original structure of CBP remaining unchanged, in this Letter a more generalized network model ICBP (Improved Circular Back-Propagation Neural Network) was designed by adding an extensive node with quadratic form to the original CBP inputs and endowing fixed values to the weights between this node and all the hidden nodes. An interesting property of ICBP is that although it has less adaptable weights, it is better in generalization and adaptability than CBP. Moreover, in order to partially solve the problem of local minima, we adopt the method of adding controlled noise to desired outputs. Finally, ithas been proved by experiments that ICBP is better than CBP in the capabilities of forecasting and function approximation.

Objective quality assessment of MPEG-2 video streams by using CBP neural networks

The increasing use of compression standards in broadcasting digital TV has raised the need for established criteria to measure perceived quality. Novel methods must take into account the specific artifacts introduced by digital compression techniques. This paper presents a methodology using circular backpropagation (CBP) neural networks for the objective quality assessment of motion picture expert group (MPEG) video streams. Objective features are continuously extracted from compressed video streams on a frame-by-frame basis; they feed the CBP network estimating the corresponding perceived quality. The resulting adaptive modeling of subjective perception supports a real-time system for monitoring displayed video quality. The overall system mimics perception but does not require an analytical model of the underlying physical phenomenon. The ability to process compressed video streams represents a crucial advantage over existing approaches, as avoiding the decoding process greatly enhances the system's real-time performance. Experimental evidence confirmed the approach validity. The system was tested on real test videos; they included different contents ranging from fiction to sport. The neural model provided a satisfactory, continuous-time approximation for actual scoring curves, which was validated statistically in terms of confidence analysis. As expected, videos with slow-varying contents such as fiction featured the best performances.

Circular backpropagation networks embed vector quantization

This letter proves the equivalence between vector quantization (VQ) classifiers and circular backpropagation (CBP) networks. The calibrated prototypes for a VQ schema can be plugged in a CBP feedforward structure having the same number of hidden neurons and featuring the same mapping. The letter describes how to exploit such equivalence by using VQ prototypes to perform a meaningful initialization for BP optimization. The approach effectiveness was tested considering a real classification problem (NIST handwritten digits).

Circular backpropagation networks for classification

The class of mapping networks is a general family of tools to perform a wide variety of tasks. This paper presents a standardized, uniform representation for this class of networks, and introduces a simple modification of the multilayer perceptron with interesting practical properties, especially well suited to cope with pattern classification tasks. The proposed model unifies the two main representation paradigms found in the class of mapping networks for classification, namely, the surface-based and the prototype-based schemes, while retaining the advantage of being trainable by backpropagation. The enhancement in the representation properties and the generalization performance are assessed through results about the worst-case requirement in terms of hidden units and about the Vapnik-Chervonenkis dimension and cover capacity. The theoretical properties of the network also suggest that the proposed modification to the multilayer perceptron is in many senses optimal. A number of experimental verifications also confirm theoretical results about the model's increased performances, as compared with the multilayer perceptron and the Gaussian radial basis functions network.

Adaptive internal representation in circular back-propagation networks

Surface- and prototype-based models are often regarded as alternative paradigms to represent internal knowledge in trained neural networks. This paper analyses a network model (Circular Back-Propagation) that overcomes such dualism by choosing the best-fitting representation adaptively. The model involves a straightforward modification to classical feed-forward structures to let neurons implement hyperspherical boundaries; as a result, it exhibits a notable representation power, and benefits from the simplicity and effectiveness of classical back-propagation training. Artificial testbeds support the model definition by demonstrating its basic properties; an application to a real, complex problem in the clinical field shows the practical advantages of the approach.