Standard Radial Basis Function Network Research Articles

A Radial Basis Function Network-Based Surrogate-Assisted Swarm Intelligence Approach for Fast Optimization of Power Delivery Networks

The design and optimization of Power Delivery Networks (PDNs) in Very Large Scale Integration (VLSI) systems is becoming very challenging with the increasing complexity of such systems. Decoupling capacitors are the key elements used in a PDN to minimize power supply noise and to maintain low impedance of the PDN to avoid system failure. In this paper, a novel approach using surrogate assisted swarm intelligence is presented for efficient and fast optimization of power delivery networks. For generating the surrogate models, a standard Radial Basis Function Network (RBFN) is used. Using the proposed approach, the decoupling capacitors are selected and placed optimally, eventually reducing the cumulative impedance of the PDN below the target impedance. The performance comparison between the conventional and the surrogate-assisted approach is presented. Three case studies are presented on a practical system to demonstrate the competence of the proposed approach. The results obtained by the proposed approach are also compared with the same obtained by the state-of-the-art approaches. For the proposed approach the run time is drastically reduced compared to the state-of-the-art approaches for the optimization problem without having any effect on the performance. The consistency of results in all of the case studies confirms the validity of the proposed approach.

An Improved Radial Basis Function Networks Based on Quantum Evolutionary Algorithm for Training Nonlinear Datasets

In neural networks, the accuracies of its networks are mainly relying on two important factors which are the centers and spread value. Radial basis function network (RBFN) is a type of feedforward network that capable of perform nonlinear approximation on unknown dataset. It has been widely used in classification, pattern recognition, nonlinear control and image processing. Thus, with the increases in RBFN application, some problems and weakness of RBFN network is identified. Through the combination of quantum computing and RBFN provides a new research idea in design and performance improvement of RBFN system. This paper describes the theory and application of quantum computing and cloning operators, and discusses the superiority of these theories and the feasibility of their optimization algorithms. This proposed improved RBFN (I-RBFN) that combined with cloning operator and quantum computing algorithm demonstrated its ability in global search and local optimization to effectively speed up learning and provides better accuracy in prediction results. Both the algorithms that combined with RBFN optimize the centers and spread value of RBFN. The proposed I-RBFN was tested against the standard RBFN in predictions. The experimental models were tested on four literatures nonlinear function and four real-world application problems, particularly in Air pollutant problem, Biochemical Oxygen Demand (BOD) problem, Phytoplankton problem, and forex pair EURUSD. The results are compared to I-RBFN for root mean square error (RMSE) values with standard RBFN. The proposed I-RBFN yielded better results with an average improvement percentage more than 90 percent in RMSE.

An Improved Radial Basis Function Networks in Networks Weights Adjustment for Training Real-World Nonlinear Datasets

In neural networks, the accuracies of its networks are mainly relying on two important factors which are the centers and the networks weight. The gradient descent algorithm is a widely used weight adjustment algorithm in most of neural networks training algorithm. However, the method is known for its weakness for easily trap in local minima. It suffers from a random weight generated for the networks during initial stage of training at input layer to hidden layer networks. The performance of radial basis function networks (RBFN) has been improved from different perspectives, including centroid initialization problem to weight correction stage over the years. Unfortunately, the solution does not provide a good trade-off between quality and efficiency of the weight produces by the algorithm. To solve this problem, an improved gradient descent algorithm for finding initial weight and improve the overall networks weight is proposed. This improved version algorithm is incorporated into RBFN training algorithm for updating weight. Hence, this paper presented an improved RBFN in term of algorithm for improving the weight adjustment in RBFN during training process. The proposed training algorithm, which uses improved gradient descent algorithm for weight adjustment for training RBFN, obtained significant improvement in predictions compared to the standard RBFN. The proposed training algorithm was implemented in MATLAB environment. The proposed improved network called IRBFN was tested against the standard RBFN in predictions. The experimental models were tested on four literatures nonlinear function and four real-world application problems, particularly in Air pollutant problem, Biochemical Oxygen Demand (BOD) problem, Phytoplankton problem, and forex pair EURUSD. The results are compared to IRBFN for root mean square error (RMSE) values with standard RBFN. The IRBFN yielded a promising result with an average improvement percentage more than 40 percent in RMSE.

Distance Weighted K-Means Algorithm for Center Selection in Training Radial Basis Function Networks

The accuracies rates of the neural networks mainly depend on the selection of the correct data centers. The K-means algorithm is a widely used clustering algorithm in various disciplines for centers selection. However, the method is known for its sensitivity to initial centers selection. It suffers not only from a high dependency on the algorithm's initial centers selection but, also from data points. The performance of K-means has been enhanced from different perspectives, including centroid initialization problem over the years. Unfortunately, the solution does not provide a good trade-off between quality and efficiency of the centers produces by the algorithm. To solve this problem, a new method to find the initial centers and improve the sensitivity to the initial centers of K-means algorithm is proposed. This paper presented a training algorithm for the radial basis function network (RBFN) using improved K-means (KM) algorithm, which is the modified version of KM algorithm based on distance-weighted adjustment for each centers, known as distance-weighted K-means (DWKM) algorithm. The proposed training algorithm, which uses DWKM algorithm select centers for training RBFN obtained better accuracy in predictions and reduced network architecture compared to the standard RBFN. The proposed training algorithm was implemented in MATLAB environment; hence, the new network was undergoing a hybrid learning process. The network called DWKM-RBFN was tested against the standard RBFN in predictions. The experimental models were tested on four literatures nonlinear function and four real-world application problems, particularly in Air pollutant problem, Biochemical Oxygen Demand (BOD) problem, Phytoplankton problem, and forex pair EURUSD. The results are compared to proposed method for root mean square error (RMSE) in radial basis function network (RBFN). The proposed method yielded a promising result with an average improvement percentage more than 50 percent in RMSE.

New Strategies for Initialization and Training of Radial Basis Function Neural Networks

In this paper we proposed two new strategies for initialization and training of Radial Basis Function (RBF) Neural Network. The first approach takes into consideration the between the input vector p of the network and the x-axis, which are the centers of radial functions. The second approach takes into account the between the input vector p and the network output. In order to check the performances of these strategies, we used Brazilian financial market data for the RBF networks training, specifically the adjusted prices of the 10 greater weighted shares in the Bovespa index at the time of data collection ‑ from April 8th, 2009 to October 31th, 2014. The first approach presented a 52% of improvement in the mean squared error (MSE) compared to the standard RBF network, while the improvement for the second approach was 38%. The strategies proved to be consistent for the time series tested, in addition to having a low computational cost. It is proposed that these strategies be improved by testing them with the Levenberg-Marquardt algorithm.

A Chebyshev polynomial radial basis function neural network for automated shoreline extraction from coastal imagery

This paper investigates the potential of using a polynomial radial basis function (RBF) neural network to extract the shoreline position from coastal video images. The basic structure of the proposed network encompasses a standard RBF network module, a module of nodes that use Chebyshev polynomials as activation functions, and an inference module. The experimental setup is an operational coastal video monitoring system deployed in two sites in Southern Europe to generate variance coastal images. Thehistogram of each image isapproximated by non-linear regression, and associated witha manually extracted intensity threshold value that quantifies the shoreline position. The key idea is to use the set of the resulting regression parameters as input data, and the intensity threshold values as output data of the network. In summary, the data set is extracted by quantifying the qualitative image information, and the proposed network takes the advantage of the powerful approximation capabilities of the Chebyshev polynomials by utilizing a small number of coefficients. For comparative reasons, we apply a polynomial RBF network trained by fuzzy clustering, and a feed-forward neural network trained by the back propagation algorithm. The comparison criteria used are the standard mean square error; the data return rates, and the root mean square error of the cross- shore shoreline position, calculated against the shorelines extracted by the aforementioned annotated threshold values. The main conclusions of the simulation study are: (a) the proposed method outperforms the other networks, especially in extracting the shoreline from images used as testing data; (b) for higher polynomial orders it obtains data return rates greater than 84%, and the root mean square error of the cross-shore shoreline position is less than 1.8 meters.

Time-series event-based prediction: An unsupervised learning framework based on genetic programming

In this paper, we propose an unsupervised learning framework based on Genetic Programming (GP) to predict the position of any particular target event (defined by the user) in a time-series. GP is used to automatically build a library of candidate temporal features. The proposed framework receives a training set S={(Va)|a=0⋯n}, where each Va is a time-series vector such that ∀Va∈S,Va={(xt)|t=0⋯tmax} where tmax is the size of the time-series. All Va∈S are assumed to be generated from the same environment. The proposed framework uses a divide-and-conquer strategy for the training phase. The training process of the proposed framework works as follow. The user specifies the target event that needs to be predicted (e.g., Highest value, Second Highest value, …, etc.). Then, the framework classifies the training samples into different Bins, where Bins={(bi)|i=0⋯tmax}, based on the time-slot t of the target event in each Va training sample. Each bi∈Bins will contain a subset of S. For each bi, the proposed framework further classifies its samples into statistically independent clusters. To achieve this, each bi is treated as an independent problem where GP is used to evolve programs to extract statistical features from each bi’s members and classify them into different clusters using the K-Means algorithm. At the end of the training process, GP is used to build an ‘event detector’ that receives an unseen time-series and predicts the time-slot where the target event is expected to occur. Empirical evidence on artificially generated data and real-world data shows that the proposed framework significantly outperforms standard Radial Basis Function Networks, standard GP system, Gaussian Process regression, Linear regression, and Polynomial Regression.

Estimation of Engine Maps: A Regularized Basis-Function Networks Approach

In this brief, a new methodology for the identification of engine maps from static data is presented. In order to enhance the flexibility of the model and exploit prior knowledge on the boundary conditions of the maps, a basis function neural network with a large number of neurons is used. To ensure smoothness of the estimated map as well as guarantee reliable extrapolation properties, the weights are estimated via a regularization strategy. Dynamic data are used to validate the new methodology. For this purpose, the estimated maps are included in a mean value model whose simulated manifold pressure and crankshaft speed are compared with the experimental ones. The results show a clear improvement with respect to the performances obtained resorting to standard radial basis function networks.

An Improved Fast Training Algorithm for RBF Networks Using Symmetry-Based Fuzzy C-Means Clustering

In fuzzy C-means (FCM) clustering, each data point belongs to a cluster to a degree specified by a membership grade. FCM partitions a collection of vectors in c fuzzy groups and finds a cluster center in each group such that the dissimilarity measure is minimized. This paper presents a training algorithm for the radial basis function (RBF) network using symmetry-based Fuzzy C-means (SFCM) clustering method which is the modified version of FCM clustering method based on point symmetry distance measure. The training algorithm which uses SFCM clustering method to train the network has a number of advantages such as faster training time, more accurate predictions and reduced network architecture compared to the standard RBF networks. The proposed training algorithm has been implemented in the RBF networks created by the newrb function of MATLAB which uses gradient based iterative method as learning strategy, therefore the new network will undergo a hybrid learning process. The networks called Symmetry-based Fuzzy C-means Clustering–Radial Basis Function Network (SFCM/RBF) has been tested against the standard RBF network and the networks called standard Fuzzy C-means Clustering (FCM)-RBF network (FCM/RBF) in forecasting. The experimental models has been tested on three real world application problems, particularly in Air pollutant problem, Biochemical Oxygen Demand (BOD) problem, and Phytoplankton problem. Keywords: Fuzzy c-means clustering; SFCM; Radial basis function network; point symmetry distance; forecasting.

A spatial interpolation method based on radial basis function networks incorporating a semivariogram model

Based on the combination of the radial basis function network (RBFN) and the semivariogram, a spatial interpolation method, named improved RBFN, is proposed in this paper. To evaluate the interpolation accuracy of the proposed method, reference surfaces with prescribed semivariograms of different sills and scale parameters are generated. The proposed method as well as two existing methods (ordinary kriging and standard RBFN) is then used in the restoration of these reference surfaces. Among three interpolation methods, the proposed method has the highest interpolation accuracy regardless of the arrangement of sample points. The proposed method is performing well especially when the variance of the reference surface is large. An application of the proposed method to the estimation of the spatial distribution of rainfall also shows that the proposed method can estimate more precisely as compare to the other two existing methods. The proposed method is recommended as an alternative to the existing methods, because it has a clear principle and a simple structure. In addition, it provides more flexibility adjusted with stochastic property.

An electronic nose and modular radial basis function network classifiers for recognizing multiple fragrant materials

An electronic nose with 16 TGS sensors is developed for effectively recognizing numerous kinds of odors. In order to solve large-sample, multi-class and high-dimensional classification problems, this paper proposes a type of modular radial basis function (RBF) network classifiers, in which every module consists of a single-layer RBF network and a single-layer perceptron. The method for optimally determining the number, locations and widths of RBF kernels and the target values of Gaussian activation functions is gone into details. The presented adaptive algorithm, which only propagates error one layer backwards, has much lower computational complexity than the back-propagation algorithm used in multilayer perceptrons. The electronic nose with the modular adaptive RBF neural network classifiers is able to reach the recognition rate of 96.67% for 21 kinds of simple and complex fragrant materials. The experimental result for the extend training set, which consists of 4050 samples and 84 classes, shows that the modular RBF networks as well as the adaptively learning algorithm have faster convergence rate, higher classification accuracy, larger probability to get optimal structures, and better ability to reach global minimum points, compared with the standard RBF networks and multilayer perceptrons. Therefore, the presented modular RBF networks are quite suitable for large sample problems.

Constructing a query-able radial basis function artificial neural network.

Artificial neural networks will be more widely accepted as standard engineering tools if their reasoning process can be made less opaque. This paper describes NetQuery, an explanation mechanism that extracts meaningful explanations from trained Radial Basis Function (RBF) networks. RBF networks are well suited for explanation generation because they contain a set of locally tuned units. Standard RBF networks are modified to identify dependencies between the inputs, to be sparsely connected, and to have an easily interpretable output layer. Given these modifications, the network architecture can be used to extract "Why?" and "Why not?" explanations from the network in terms of excitatory and inhibitory in-puts and their linear relationships, greatly simplified by a run-time pruning algorithm. These query results are validated by creating an expert system based on the explanations. NetQuery is also able to inform a user about a possible change in category for a given pattern by responding to a "How can I...?" query. This kind of query is extremely useful when analyzing the quality of a pattern set.

Pruning and Extraction of Qualitative Fault Diagnosis Rules from a Neural Network

Artificial neural networks have been shown to be an effective tool for process fault diagnosis. However, a main criticism is that details of how fault diagnosis decisions are made are embedded in the complex input-output mapping performed by a network, and are in general difficult to obtain. In this paper, statistical techniques and relationships between fuzzy systems and standard radial basis function networks, are exploited to prune a trained network and to extract qualitative rules that explain the network operation for fault diagnosis. The procedures are demonstrated in an application to the diagnosis of a range of both soft and hard faults in a simulated, complex, chemical process.

Radial basis function networks, regression weights, and the expectation-maximization algorithm

We propose a modified radial basis function (RBF) network in which the regression weights are used to replace the constant weights in the output layer. It is shown that the modified RBF network can reduce the number of hidden units significantly. A computationally efficient algorithm, known as the expectation-maximization (EM) algorithm, is used to estimate the parameters of the regression weights. A salient feature of this algorithm is that it decomposes a complicated multiparameter optimization problem into L separate small-scale optimization problems, where L is the number of hidden units. The superior performance of the modified RB network over the standard RBF network is illustrated by computer simulations.

On the persistency of excitation in radial basis function network identification of nonlinear systems

Considers radial basis function (RBF) network approximation of a multivariate nonlinear mapping as a linear parametric regression problem. Linear recursive identification algorithms applied to this problem are known to converge, provided the regressor vector sequence has the persistency of excitation (PE) property. The main contribution of this paper is formulation and proof of PE conditions on the input variables. In the RBF network identification, the regressor vector is a nonlinear function of these input variables. According to the formulated condition, the inputs provide PE, if they belong to domains around the network node centers. For a two-input network with Gaussian RBF that have typical width and are centered on a regular mesh, these domains cover about 25% of the input domain volume. The authors further generalize the proposed solution of the standard RBF network identification problem and study affine RBF network identification that is important for affine nonlinear system control. For the affine RBF network, the author formulates and proves a PE condition on both the system state parameters and control inputs.

A neural network architecture that computes its own reliability

Artificial neural networks (ANNs) have been used to construct empirical nonlinear models of process data. Because network models are not based on physical theory and contain nonlinearities, their predictions are suspect when extrapolating beyond the range of the original training data. With multiple correlated inputs, it is difficult to recognize when the network is extrapolating. Furthermore, due to non-uniform distribution of the training examples and noise over the domain, the network may have local areas of poor fit even when not extrapolating. Standard measures of network performance give no indication of regions of locally poor fit or possible errors due to extrapolation. This paper introduces the “validity index network” (VI-net), an extension of radial basis function networks (RBFN), that calculates the reliability and the confidence of its output and indicates local regions of poor fit and extrapolation. Because RBFNs use a composition of local fits to the data, they are readily adapted to predict local fitting accuracy. The VI-net can also detect novel input patterns in classification problems, provided that the inputs to the classifier are real values. The reliability measures of the VI-net are implemented as additional output nodes of the underlying RBFN. Weights associated with the reliability nodes are given analytically based on training statistics from the fitting of the target function, and thus the reliability measures can be added to a standard RBFN with no additional training effort.