Radial Basis Function Neurons Research Articles

An Improved Radial Basis Function Neuron Network Based on the l1 Regularization

The radial basis function neural network (RBFNN) is a widely used tool for interpolation and prediction problems. In this paper, we propose to improve the traditional RBFNN by automatically identifying core neurons in the hidden layer, based on the [Formula: see text] regularization. Our proposed approach will greatly reduce the number of neurons required, which will save the memory and also the computational cost. To determine the radial parameter [Formula: see text] in the RBFs, we propose to use the [Formula: see text]-fold cross-validation method. Moreover, the principal component analysis (PCA) method is used to reconstruct the distance between samples for high-dimensional data sets. Numerical experiments are provided to demonstrate the effectiveness of the proposed approach.

Self-Learning Event Mistiming Detector Based on Central Pattern Generator.

A repetitive movement pattern of many animals, a gait, is controlled by the Central Pattern Generator (CPG), providing rhythmic control synchronous to the sensed environment. As a rhythmic signal generator, the CPG can control the motion phase of biomimetic legged robots without feedback. The CPG can also act in sensory synchronization, where it can be utilized as a sensory phase estimator. Direct use of the CPG as the estimator is not common, and there is little research done on its utilization in the phase estimation. Generally, the sensory estimation augments the sensory feedback information, and motion irregularities can reveal from comparing measurements with the estimation. In this work, we study the CPG in the context of phase irregularity detection, where the timing of sensory events is disturbed. We propose a novel self-supervised method for learning mistiming detection, where the neural detector is trained by dynamic Hebbian-like rules during the robot walking. The proposed detector is composed of three neural components: (i) the CPG providing phase estimation, (ii) Radial Basis Function neuron anticipating the sensory event, and (iii) Leaky Integrate-and-Fire neuron detecting the sensory mistiming. The detector is integrated with the CPG-based gait controller. The mistiming detection triggers two reflexes: the elevator reflex, which avoids an obstacle, and the search reflex, which grasps a missing foothold. The proposed controller is deployed and trained on a hexapod walking robot to demonstrate the mistiming detection in real locomotion. The trained system has been examined in the controlled laboratory experiment and real field deployment in the Bull Rock cave system, where the robot utilized mistiming detection to negotiate the unstructured and slippery subterranean environment.

Ultra-Short-Term Forecasting of Photo-Voltaic Power via RBF Neural Network

With the fast expansion of renewable energy systems during recent years, the stability and quality of smart grids using solar energy have been challenged because of the intermittency and fluctuations. Hence, forecasting photo-voltaic (PV) power generation is essential in facilitating planning and managing electricity generation and distribution. In this paper, the ultra-short-term forecasting method for solar PV power generation is investigated. Subsequently, we proposed a radial basis function (RBF)-based neural network. Additionally, to improve the network generalization ability and reduce the training time, the numbers of hidden layer neurons are limited. The input of neural network is selected as the one with higher Spearman correlation among the predicted power features. The data are normalized and the expansion parameter of RBF neurons are adjusted continuously in order to reduce the calculation errors and improve the forecasting accuracy. Numerous simulations are carried out to evaluate the performance of the proposed forecasting method. The mean absolute percentage error (MAPE) of the testing set is within 10%, which show that the power values of the following 15 min. can be predicted accurately. The simulation results verify that our method shows better performance than other existing works.

Integrative Design of an Emergency Resource Predicting-Scheduling-Repairing Method for Rail Track Faults

To solve the problem of track failure in railway system, integrative design of an emergency resource forecasting-scheduling-repairing method is proposed in this paper. In the current railway repair process, resource dispatching centers dispatch transport resources to the fault point for the repair work. In order to optimize this process, a hybrid prediction method consisting of radial basis function neuron network (RBFNN) and expert prediction method to rationally predict repair resources is proposed. In order to optimize the resource transportation path and reduce the transportation cost, a bi-objective resource scheduling method is presented in this paper. A fuzzy evaluation method combining with the prospect theory to optimally select the repairing for the repair work is formulated. Finally, experiment studies are carried out and the results illustrate the effectiveness of the proposed method.

Training Lightweight Deep Convolutional Neural Networks Using Bag-of-Features Pooling.

Convolutional neural networks (CNNs) are predominantly used for several challenging computer vision tasks achieving state-of-the-art performance. However, CNNs are complex models that require the use of powerful hardware, both for training and deploying them. To this end, a quantization-based pooling method is proposed in this paper. The proposed method is inspired from the bag-of-features model and can be used for learning more lightweight deep neural networks. Trainable radial basis function neurons are used to quantize the activations of the final convolutional layer, reducing the number of parameters in the network and allowing for natively classifying images of various sizes. The proposed method employs differentiable quantization and aggregation layers leading to an end-to-end trainable CNN architecture. Furthermore, a fast linear variant of the proposed method is introduced and discussed, providing new insight for understanding convolutional neural architectures. The ability of the proposed method to reduce the size of CNNs and increase the performance over other competitive methods is demonstrated using seven data sets and three different learning tasks (classification, regression, and retrieval).

Multi-mode soft switching control for variable pitch of wind turbines based on T-S fuzzy weighted

Variable pitch control is an effective way to ensure the constant power operation of the wind turbines over rated wind speed. The pitch actuator acts frequently with larger amplitude and the increasing mechanical fatigue load of parts of wind turbines affects the output quality of generator and damages the service life of wind turbines. The existing switching control methods only switch at a certain threshold, which can result in switch oscillation. In order to deal with these problems, a multi-mode soft switching variable pitch control strategy was put forward based on Takagi-Sugeno (T-S) fuzzy weighted to accomplish soft switch, which combined intelligent control with classical control. The T-S fuzzy inference was carried out according to the error and its change rate, which was used to smooth the modal outputs of fuzzy control, radial basis function neuron network proportion integration differentiation (RBFNN PID) control and proportion integration (PI) control. This method takes the advantages of the three controllers into consideration. A multi-mode soft switch control model for variable pitch of permanent magnet direct drive wind turbines was built in the paper. The simulation results show that this method has the advantages of three control modes, switch oscillation is overcome. The integrated control performance is superior to the others, which can not only stabilize the output power of wind turbines but also reduce the fatigue load.

Learning Neural Bag-of-Features for Large-Scale Image Retrieval

In this paper, the well-known bag-of-features (BoFs) model is generalized and formulated as a neural network that is composed of three layers: 1) a radial basis function (RBF) layer; 2) an accumulation layer; and 3) a fully connected layer. This formulation allows for decoupling the representation size from the number of used codewords, as well as for better modeling the feature distribution using a separate trainable scaling parameter for each RBF neuron. The resulting network, called retrieval-oriented neural BoF (RN-BoF), is trained using regular back propagation and allows for fast extraction of compact image representations. It is demonstrated that the RN-BoF model is capable of: 1) increasing the object encoding and retrieval speed; 2) reducing the extracted representation size; and 3) increasing the retrieval precision. A symmetry-aware spatial segmentation technique is also proposed to further reduce the encoding time and the storage requirements and allows the method to efficiently scale to large datasets. The proposed method is evaluated and compared to other state-of-the-art techniques using five different image datasets, including the large-scale YouTube Faces database.

A New Rate Control Scheme For Video Coding Based On Region Of Interest

The quality fluctuation of video is significant in human visual system, and thus, many rate control schemes are widely developed in the area of video communication. In recent years, researchers show more interests in region of interest (ROI)-based encoding, and it is widely applied in the latest video codecs, such as HEVC and VP9. This paper presents a new rate control scheme for ROI mode coding based on discrete fourier transform coefficient model and radial basis function neuron network. A new R-D model is proposed by classifying blocks into different depth, ROI groups, and so on. Then, rate and distortion are described based on the Laplacian distribution model using mathematical ways. A machine learning approach is induced to enhance the accuracy of the distortion estimation. By utilizing the new R-D model, a new rate control scheme is designed for ROI mode coding from the group of picture layer to coding unit layer. By comparisons with other rate control approaches, the proposed one has a better result in terms of visual quality, R-D performance, bitrate accuracy, and so on. Hence, it outperforms the conventional schemes especially for sequences with obvious ROI details.

An Effective Solution to Regression Problem by RBF Neuron Network

Radial Basis Function (RBF) neuron network is being applied widely in multivariate function regression. However, selection of neuron number for hidden layer and definition of suitable centre in order to produce a good regression network are still open problems which have been researched by many people. This article proposes to apply grid equally space nodes as the centre of hidden layer. Then, the authors use k-nearest neighbour method to define the value of regression function at the center and an interpolation RBF network training algorithm with equally spaced nodes to train the network. The experiments show the outstanding efficiency of regression function when the training data has Gauss white noise.

Optimization design based on ensemble surrogate models for DNAPLs-contaminated groundwater remediation

The optimization design of surfactant-enhanced aquifer remediation for dense non-aqueous phase liquids (DNAPLs)-contaminated groundwater is proposed through integrating simulation and optimization models. Many studies have demonstrated that the surrogate model is an effective tool for building a bridge between simulation and optimization models. In this paper, the simulation model was first established to simulate a surfactant-enhanced aquifer remediation process, and on this basis, the ensemble surrogate models were constructed by applying the polynomial regression model, radial basis function neurons network and Kriging surrogate models. Second, the best surrogate model was selected in terms of the three performance indices. Lastly, a non-linear programming optimization model was constructed with the target of minimizing the DNAPLs-contaminated aquifer remediation cost. Meanwhile, the best surrogate model was embedded into the optimization model as a constrained condition, and it was used to reflect the non-linear complex relationship between injection and extraction rates with DNAPLs removal rates instead of simulation models. The results showed that the ensemble surrogate models improved the performance of the single surrogate models. Moreover, ensemble surrogate models improved the computational efficiency, and the optimal strategies have been proved to be an effective guide for contaminant remediation processes.

An online supervised learning method for spiking neural networks with adaptive structure

A novel online learning algorithm for Spiking Neural Networks (SNNs) with dynamically adaptive structure is presented. The main contribution of this work lies in the fact that the proposed adaptive SNN is able to classify spike-based spatio-temporal inputs after just one presentation of the training set, i.e. in one pass only, and does not require the entire training set to be available at once. Both the structure and weights of the SNN are learned dynamically through a combination of unsupervised and supervised learning paradigms. The proposed feed-forward SNN consists of three layers of spiking neurons: an input layer which temporally encodes real valued features into spike-based spatio-temporal patterns, a hidden layer of dynamically grown and pruned neurons which perform spatio-temporal clustering, and an output layer for classification. An unsupervised spiking-based clustering algorithm is implemented by the hidden layer whose spiking neurons are trained to compute a temporal Radial Basis Function (RBF) where incoming inputs will selectively activate hidden neurons based on how close the inputs are to the preferred inputs of the hidden neurons. The centre of each hidden RBF spiking neuron is represented by its time to first spike. In addition, a growing and pruning strategy is proposed to adjust the structure of the hidden layer ‘on-the-fly’ as inputs are presented to the SNN. Both the weights and the centres of the hidden RBF neurons are learned in an unsupervised way and classification at the output layer is achieved through supervised learning where the learning windows proposed for STDP and anti-STDP are used to adjust the weights of the output neurons afferent connections. Competition at both the hidden and the output layers is achieved through the use of lateral inhibitory connections between the neurons of each layer. The proposed online learning algorithm is validated on several benchmark datasets. The evaluation results demonstrate that SNNs trained with the proposed approach require only one pass through the training set in order to classify the inputs with comparable accuracies to existing SNN-based approaches as well as traditional representative classifiers.

A growing and pruning sequential learning algorithm of hyper basis function neural network for function approximation

Radial basis function (RBF) neural network is constructed of certain number of RBF neurons, and these networks are among the most used neural networks for modeling of various nonlinear problems in engineering. Conventional RBF neuron is usually based on Gaussian type of activation function with single width for each activation function. This feature restricts neuron performance for modeling the complex nonlinear problems. To accommodate limitation of a single scale, this paper presents neural network with similar but yet different activation function—hyper basis function (HBF). The HBF allows different scaling of input dimensions to provide better generalization property when dealing with complex nonlinear problems in engineering practice. The HBF is based on generalization of Gaussian type of neuron that applies Mahalanobis-like distance as a distance metrics between input training sample and prototype vector. Compared to the RBF, the HBF neuron has more parameters to optimize, but HBF neural network needs less number of HBF neurons to memorize relationship between input and output sets in order to achieve good generalization property. However, recent research results of HBF neural network performance have shown that optimal way of constructing this type of neural network is needed; this paper addresses this issue and modifies sequential learning algorithm for HBF neural network that exploits the concept of neuron’s significance and allows growing and pruning of HBF neuron during learning process. Extensive experimental study shows that HBF neural network, trained with developed learning algorithm, achieves lower prediction error and more compact neural network.

An online self-adaptive modular neural network for time-varying systems

We propose an online self-adaptive modular neural network (OSAMNN) for time-varying systems. Starting with zero subnetworks, OSAMNN uses a single-pass subtractive cluster algorithm to update the centers of radial-basis function (RBF) neurons for learning. Then the input space can be partitioned. The OSAMNN structure is capable of growing or merging subnetworks to maintain suitable model complexity, and the centers of RBF neurons can also be dynamically adjusted according to changes in the data environment. A fuzzy strategy is applied to select suitable subnetworks to learn the current sample. This method yields improved learning efficiency and accuracy. OSAMNN can adapt its architecture to realize online modeling of time-varying nonlinear input–output maps. Results for experiments on benchmark and real-world time-varying systems support the proposed techniques.

A Self-Organizing Fuzzy Neural Network Based on a Growing-and-Pruning Algorithm

A novel growing-and-pruning (GP) approach is proposed, which optimizes the structure of a fuzzy neural network (FNN). This GP-FNN is based on radial basis function neurons, which have center and width vectors. The structure-learning phase and the parameter-training phase are performed concurrently. The structure-learning approach relies on the sensitivity analysis of the output. A set of fuzzy rules can be inserted or reduced during the learning process. The parameter-training algorithm is implemented using a supervised gradient decent method. The convergence of the GP-FNN-learning process is also discussed in this paper. The proposed method effectively generates a fuzzy neural model with a highly accurate and compact structure. Simulation results demonstrate that the proposed GP-FNN has a self-organizing ability, which can determine the structure and parameters of the FNN automatically. The algorithm performs better than some other existing self-organizing FNN algorithms.

An improved neural network method for solving the Schrödinger equation

We propose a neural network (NN) based algorithm for calculating vibrational energies and wave functions and apply it to problems in 2-, 4-, and 6-dimensions. By using neurons as basis functions and methods of nonlinear optimization, we are able to compute three states of a 6-D Hamiltonian using only 50 basis functions. In a standard direct product basis, thousands of basis functions would be necessary. Previous NN methods for solving the Schrödinger equation computed one level at a time and optimized all of the parameters using expensive nonlinear optimization methods. Using our approach, linear coefficients in the NN representation of wave functions are determined with methods of linear algebra and many levels are computed at the same time from one set of nonlinear NN parameters. In addition, we use radial basis function neurons to ensure the correct boundary conditions. The use of linear algebra methods makes it possible to treat systems of higher dimensionality.

Using a neural network based method to solve the vibrational Schrödinger equation for H 2O

A neural network (NN) algorithm is used to solve the vibrational Schrödinger equation for a molecule. Previous NN methods computed one level at a time, optimized all of the parameters using non-linear optimization methods, and were tested only on model potentials. Our approach combines non-linear optimization of neuron parameters with a linear matrix method. This improves dimensionality scaling and permits computing many levels. We use composite, flexible shape, radial basis function neurons. The algorithm avoids the calculation of integrals and of a potential energy function. We demonstrate that only a few dozen neurons are needed to compute five levels of water from a small set of potential points.

Optimization and stabilization of sequential learning in RBF network for nonlinear function approximation

This paper proposes a solution for inconsistency pruning of neurons within a sequential learning Radial Basis Function (RBF) Network. This paper adopts the concept that a specific RBF neuron which continuously exhibits low output in a sequence of training patterns does not justify the proposition that the neuron is insignificant to the whole function to be approximated. We establish additional criterions to provide protection from error in pruning RBF neurons within the hidden layer, which we prove is able to improve consistency and stability of neuron evolution. With such stability within the sequential learning process, we also show how the convergence speed of the network can be improved by reducing the number of consecutive observations required to prune a neuron in the hidden layer.

A Pulsed Neural Network With On-Chip Learning and Its Practical Applications

This paper proposes a new model for the pulsed neural network. In this model, the information is coded in terms of firing times of pulses that are generated by the neuron. The pulses transmit through the network and excite the dynamics of the neuron. Their synchronism is utilized to design the architecture of the neural network such that it acts as a radial basis function (RBF) network. A new network-learning algorithm is also developed for this pulsed RBF network. The RBF neurons are generated based on the feature of the training data, and the synaptic delays can be adjusted to distribute these RBF neurons in the training data space. The pulse neural network has been implemented compactly with multiplierless approach for both the forward computation and learning algorithm with a field programmable gate array board. As an application demonstration, it is extended to a nonlinear look-up table and applied to estimate the friction occurs in a precision linear stage

Steel columns under fire—a neural network based strength model

This paper presents a strength model of steel columns under elevated temperatures using the artificial neural network. The many influencing parameters make it difficult to build an analytical steel strength model. Being a flexible model building method, the artificial neural network is an ideal tool to construct the complex relationship between the input and the output parameters accurately. A hybrid neural network, which combines the sigmoid neurons and the radial basis function neurons at the hidden layer, is proposed to better map the input–output relationship both locally and globally. The use of the genetic algorithm approach in searching the best-hidden neurons makes the hybrid neural network less likely to be trapped in local minima than the traditional gradient-based search algorithms. The genetic algorithm based hybrid neural network is applied to model the strength of steel columns under fire. The neural network results are compared with the modified Rankine formula.

On the power of Boolean computations in generalized RBF neural networks

Generalized radial basis function (RBF) neurons are extensions of the RBF neuron model where the Euclidean norm is replaced by a weighted norm. We study binary-valued variants of generalized RBF neurons and compare their computational power in the Boolean domain with linear threshold neurons. As one of the main results, we show that generalized binary RBF neurons with any weighted norm can compute every Boolean function that is computed by a linear threshold neuron. While this inclusion turns into an equality if the RBF neuron uses the Euclidean norm, we exhibit a weighted norm where the inclusion is proper. Applications of the results yield bounds on the Vapnik–Chervonenkis (VC) dimension of RBF neural networks with binary inputs.