Adaptive Basis Function Research Articles

Iterative adaptive RBF methods for detection of edges in two-dimensional functions

Radial basis functions have gained popularity for many applications including numerical solution of partial differential equations, image processing, and machine learning. For these applications it is useful to have an algorithm which detects edges or sharp gradients and is based on the underlying basis functions. In our previous research, we proposed an iterative adaptive multiquadric radial basis function method for the detection of local jump discontinuities in one-dimensional problems. The iterative edge detection method is based on the observation that the absolute values of the expansion coefficients of multiquadric radial basis function approximation grow exponentially in the presence of a local jump discontinuity with fixed shape parameters but grow only linearly with vanishing shape parameters. The different growth rate allows us to accurately detect edges in the radial basis function approximation. In this work, we extend the one-dimensional iterative edge detection method to two-dimensional problems. We consider two approaches: the dimension-by-dimension technique and the global extension approach. In both cases, we use a rescaling method to avoid ill-conditioning of the interpolation matrix. The global extension approach is less efficient than the dimension-by-dimension approach, but is applicable to truly scattered two-dimensional points, whereas the dimension-by-dimension approach requires tensor product grids. Numerical examples using both approaches demonstrate that the two-dimensional iterative adaptive radial basis function method yields accurate results.

Cross-Entropy Optimization of Control Policies With Adaptive Basis Functions

This paper introduces an algorithm for direct search of control policies in continuous-state discrete-action Markov decision processes. The algorithm looks for the best closed-loop policy that can be represented using a given number of basis functions (BFs), where a discrete action is assigned to each BF. The type of the BFs and their number are specified in advance and determine the complexity of the representation. Considerable flexibility is achieved by optimizing the locations and shapes of the BFs, together with the action assignments. The optimization is carried out with the cross-entropy method and evaluates the policies by their empirical return from a representative set of initial states. The return for each representative state is estimated using Monte Carlo simulations. The resulting algorithm for cross-entropy policy search with adaptive BFs is extensively evaluated in problems with two to six state variables, for which it reliably obtains good policies with only a small number of BFs. In these experiments, cross-entropy policy search requires vastly fewer BFs than value-function techniques with equidistant BFs, and outperforms policy search with a competing optimization algorithm called DIRECT.

Radial basis function network based on time variant multi-objective particle swarm optimization for medical diseases diagnosis

This paper proposes an adaptive evolutionary radial basis function (RBF) network algorithm to evolve accuracy and connections (centers and weights) of RBF networks simultaneously. The problem of hybrid learning of RBF network is discussed with the multi-objective optimization methods to improve classification accuracy for medical disease diagnosis. In this paper, we introduce a time variant multi-objective particle swarm optimization (TVMOPSO) of radial basis function (RBF) network for diagnosing the medical diseases. This study applied RBF network training to determine whether RBF networks can be developed using TVMOPSO, and the performance is validated based on accuracy and complexity. Our approach is tested on three standard data sets from UCI machine learning repository. The results show that our approach is a viable alternative and provides an effective means to solve multi-objective RBF network for medical disease diagnosis. It is better than RBF network based on MOPSO and NSGA-II, and also competitive with other methods in the literature.

Continuous-action reinforcement learning with fast policy search and adaptive basis function selection

As an important approach to solving complex sequential decision problems, reinforcement learning (RL) has been widely studied in the community of artificial intelligence and machine learning. However, the generalization ability of RL is still an open problem and it is difficult for existing RL algorithms to solve Markov decision problems (MDPs) with both continuous state and action spaces. In this paper, a novel RL approach with fast policy search and adaptive basis function selection, which is called Continuous-action Approximate Policy Iteration (CAPI), is proposed for RL in MDPs with both continuous state and action spaces. In CAPI, based on the value functions estimated by temporal-difference learning, a fast policy search technique is suggested to search for optimal actions in continuous spaces, which is computationally efficient and easy to implement. To improve the generalization ability and learning efficiency of CAPI, two adaptive basis function selection methods are developed so that sparse approximation of value functions can be obtained efficiently both for linear function approximators and kernel machines. Simulation results on benchmark learning control tasks with continuous state and action spaces show that the proposed approach not only can converge to a near-optimal policy in a few iterations but also can obtain comparable or even better performance than Sarsa-learning, and previous approximate policy iteration methods such as LSPI and KLSPI.

Modelling cardiovascular physiological signals using adaptive Hermite and wavelet basis functions

This study presented a unified perspective of adaptive basis functions to compare Hermite decomposition and wavelet transform for the analysis of cardiovascular physiological signals. Three different algorithms were presented to carry out physiological signal modelling with adaptive Hermite basis functions (HBFs), orthonormal wavelet basis functions (OWBFs) and adaptive wavelet basis functions (AWBFs). The modelling with OWBFs is computationally efficient. However, the concomitant restrictions in mathematics make OWBFs not optimal for compact modelling. In contrast, the optimised AWBFs can model cardiovascular physiological signals compactly with the cost of losing orthonormality. It not only sacrifices the fast implementation but also degrades AWBFs in discriminant analysis. In summary, merely HBFs achieve a balanced performance in compact modelling and discriminant analysis.

A comparison of subset selection and adaptive basis function construction for polynomial regression model building

A comparison of subset selection and adaptive basis function construction for polynomial regression model buildingThe approach of subset selection in polynomial regression model building assumes that the chosen fixed full set of predefined basis functions contains a subset that is sufficient to describe the target relation sufficiently well. However, in most cases the necessary set of basis functions is not known and needs to be guessed - a potentially non-trivial (and long) trial and error process. In our previous research we considered an approach for polynomial regression model building which is different from the subset selection - letting the regression model building method itself construct the basis functions necessary for creating a model of arbitrary complexity without restricting oneself to the basis functions of a predefined full model. The approach is titled Adaptive Basis Function Construction (ABFC). In the present paper we compare the two approaches for polynomial regression model building - subset selection and ABFC - both theoretically and empirically in terms of their underlying principles, computational complexity, and predictive performance. Additionally in empirical evaluations the ABFC is compared also to two other well-known regression modelling methods - Locally Weighted Polynomials and Multivariate Adaptive Regression Splines.

An iterative adaptive multiquadric radial basis function method for the detection of local jump discontinuities

In [J.-H. Jung, Appl. Numer. Math. 57 (2007) 213–229], an adaptive multiquadric radial basis function method has been proposed for the reconstruction of discontinuous functions. Utilizing the vanishing shape parameters near the local jump discontinuity, the adaptive method considerably reduces the Gibbs oscillations and enhances convergence. In this paper, a new jump discontinuity detection method is developed based on the adaptive method. The global maximum of the expansion coefficients, λ i , exists at the strongest jump discontinuity and its magnitude increases exponentially with the number of the center points, N. The adaptive method, however, dynamically reduces its magnitude to O ( N ) once applied. The global maximum of λ i then exists at the next strongest jump discontinuity and its magnitude is exponentially large. In this way, the local jump discontinues are successively detected with the adaptive method applied iteratively. Numerical examples are provided using the piecewise analytic functions and the numerical solution of the shock interaction equations. Numerical results verify that the proposed method is efficient and accurate in finding local jump discontinuities.

Adaptive Selection and Rearrangement of Wavelet Packets for Quad-Tree Image Coding

Embedded zero-tree image coding in wavelet domain has drawn a lot of attention. Among noteworthy algorithms is the set partitioning in hierarchical trees (SPIHT). Typically, most of images' energy is concentrated in low frequency subbands. For an image with textures, however many middle-high frequency wavelet coefficients are likely to become significant in the early passes of SPIHT; thus the coding results are often insufficient. Middle and high frequency subbands of images may demand further decompositions using adaptive basis functions. As wavelet packet transform offers a great diversity of basis functions, we propose a quad-tree based adaptive wavelet packet transform to construct adaptive wavelet packet trees for zero-tree image coding. Experimental results show that coding performances can be significantly improved especially for fingerprints images.

A Faster Method of Moments Solution to Double Layer Formulation of Acoustic Scattering

In this work, the acoustic scattering problem based oil double layer formulation is solved with a novel numerical technique using method of moment's solution. A new set of basis functions, namely, Edge based Adaptive Basis Functions (EABF) are defined in the method of moment's solution procedure. The geometry of the body is divided into triangular patches and basis functions are defined oil the edges. Since the double layer formulation involves the evaluation of the hyper-singular integral, the edge based adaptive basis functions are used to make the solution faster. The matrix equations are derived for the double layer formulation. The edge based adaptive basis functions used in this work generates a diagonal moment matrix and hence do not need any matrix inversion to be carried out. The scattering cross section of the canonical shapes is used to validate the numerical solution developed and the plots are presented.

An adaptive recurrent radial basis function network tracking controller for a two-dimensional piezo-positioning stage

An adaptive recurrent radial basis function network (ARRBFN) tracking controller for a two-dimensional piezo-positioning stage is proposed in this study. First, a mathematical model that represents the dynamics of the two-dimensional piezo-positioning stage is proposed. In this model, a hysteresis friction force that describes the hysteresis behavior of one-dimensional motion is used; and a nonconstant stiffness with the cross-coupling dynamic due to the effect of bending of a lever mechanism in x and y axes also is included. Then, according to the proposed mathematical model, an ARRBFN tracking controller is proposed. In the proposed ARRBFN control system, a recurrent radial basis function network (RRBFN) with accurate approximation capability is used to approximate an unknown dynamic function. The adaptive learning algorithms that can learn the parameters of the RRBFN on line are derived using Lyapunov stability theorem. Moreover, a robust compensator is proposed to confront the uncertainties, including approximation error, optimal parameter vectors, higher-order terms in Taylor series. To relax the requirement of the value of the lumped uncertainty in the robust compensator, an adaptive law is investigated to estimate the lumped uncertainty. Using the proposed control scheme, the position tracking performance is substantially improved and the robustness to uncertainties, including hysteresis friction force and cross-coupling stiffness, can be obtained as well. The tracking performance and the robustness to external load of the proposed ARRBFN control system are illustrated by some experimental results.

Adaptive wavelet modelling of a nested 3 factor experimental design in NIR chemometrics

The objective of this study was to investigate the effects of some commonly used sample preparation procedures, including overnight freezing, and the type of homogeniser on the near-infrared (NIR) spectra of red grape homogenates. Homogenates ( n = 284) of three red grape varieties were prepared using one of three types of homogenisers after one of two types of short term storage (fresh and overnight freezing) and then scanned in a FOSS NIRSystems6500 instrument (400–2500 nm). The NIR spectral data were then analysed using various discrimination techniques, namely Penalized Discriminant Analysis (PDA), Multivariate Adaptive Regression Splines discriminant analysis (MARS-DA) and Random Forests (RF) yielding correct classification rates (CCR) of 63.4%, 58.6% and 45.6%, respectively. To improve the CCR of the discrimination models, feature extraction from the NIR spectral data was performed using an adaptive discrete wavelet transformation (DWT). The DWT algorithm employs an adaptive wavelet basis function that maximizes the discrimination between the different combinations of homogenisers, storage and grape varieties. The results after adaptive DWT on the NIR spectra resulted in CCRs of 99.93%, 99.2% and 76.4% for PDA, MARS-DA and RF, respectively. Further analysis of adaptive DWT PDA via MANOVA revealed significant differences in the main and interaction effects of the three treatments, which were then associated with specific regions within the NIR spectrum.

Analysis of linear arrays using the adaptive basis functions/diagonal moment matrix technique

An efficient algorithm is proposed for the analysis of large finite arrays using the adaptive basis functions/diagonal moment matrix technique. Adaptive basis functions constructed using clusters spanning over an array element are used to generate a highly diagonally dominant moment matrix. The physical interpretation of the constructed set of adaptive basis functions is discussed. The new matrix equation is solved iteratively in a way that only the significant mutual impedances are considered. The proposed algorithm is applied to a linear array of bow-tie antennas, and results are compared to those obtained using the direct moment method solution and exhibit very good agreement. The relative computational time improves as the array size increases compared to the conventional moment method solution. A speedup factor of more than 100 is achieved for an array of 32 elements.

A neural network tool for analyzing trends in rainfall

Rainfall, like all other natural phenomena is highly unpredictable. Traditionally, principal component analysis and spectral analysis are used to understand trends in rainfall over long periods. In this paper, we present a new method that appears to be better than existing methods to understand the long-term behavior of rainfall phenomena. Using a case study on the rainfall in Kerala State, the southern part of Indian Peninsula, we show that a new kind of neural network known as the adaptive basis function network is a promising tool for climatic studies, especially rainfall analysis. The paper also reveals that in spite of the fluctuations resulting from the nonlinearity in the system, the trends in the rainfall pattern in Kerala state have remained unaffected over the past 87 years from 1893 to 1980. We also successfully filter out the chaotic part of the system and illustrate that its effects are marginal over long-term predictions.

Application of adaptive basis functions for a diagonal moment matrix solution of arbitrarily shaped three-dimensional conducting body problems

We present the application of specially constructed adaptive basis functions that generate a diagonal matrix in the method of moments solution procedure for the calculation of scattered electromagnetic fields from arbitrarily shaped conducting bodies excited by a plane electromagnetic wave. The arbitrary body is modeled using planar triangular patches. The crucial step in the solution procedure is the construction of the adaptive basis functions to generate the diagonal matrix. This task is accomplished with the help of well-known RWG basis functions. The solution thus obtained is very efficient, accurate, and applicable to truly arbitrary bodies. Several numerical examples are presented to validate the new method.

Adaptive basis function for artificial neural networks

It is shown that modifying the sigmoidal basis function of a multi-layer feedforward artificial neural network using a control parameter improves the network's ability to learn. The modification is rendered by a gradient descent algorithm similar to the back-propagation. In doing so, the method retains all the goodies of the sigmoidal function and causes the ANN to approximate the decision function faster and also with better accuracy.

Detecting Sensor Faults for a Chemical Reactor

For time-varying processes a fixed neural network (NN) model trained off-line cannot be used for on-line fault detection, as the NN cannot model the process dynamics change and therefore the modelling error will increase. When sensor faults are detected, a parallel model should be used for multi-step ahead prediction. In this case the time-varying effects will result in modelling error diverging. An on-line fault detection strategy using adaptive parallel radial basis function network (RBFN) is proposed in this paper. The RBFN is trained on-line using the recursive orthogonal least squares (ROLS) algorithm until a fault is detected. An error index is formulated to distinguish the dynamics change from the faults. The method is applied to a three-input three-output chemical process to detect the simulated faults on the real data. The results indicate the feasibility of the method and the applicability to the real dynamic processes.

Application of adaptive basis functions for a diagonal moment‐matrix solution of two‐dimensional field problems—TE case

AbstractIn this work, we present the application of specially constructed adaptive basis functions to generate a diagonal matrix in the method‐of‐moments solution procedure for the calculation of a scattered electromagnetic field from arbitrarily shaped two‐dimensional cylinders excited by a transverse electric incident electromagnetic wave. Several numerical examples are also presented to validate the new method. © 2001 John Wiley & Sons, Inc. Microwave Opt Technol Lett 29: 270–273, 2001.

Building Adaptive Basis Functions with a Continuous Self-OrganizingMap

This paper introduces CSOM, a continuous version of the Self-Organizing Map (SOM). The CSOM network generates maps similar to those created with the original SOM algorithm but, due to the continuous nature of the mapping, CSOM outperforms the SOM on function approximation tasks. CSOM integrates self-organization and smooth prediction into a single process. This is a departure from previous work that required two training phases, one to self-organize a map using the SOM algorithm, and another to learn a smooth approximation of a function. System performance is illustrated with three examples.

Reconstruction of Monthly SST in the Tropical Pacific Ocean during 1868–1993Using Adaptive Climate Basis Functions

Abstract An EOF-based reconstruction of monthly SST anomaly fields is calculated for the years 1868–1993 in the tropical Pacific Ocean. The EOFs are computed from optimally interpolated SST anomaly fields from November 1981 to October 1995 from the same region. These are used as a functional basis set for projecting SST anomalies from the Comprehensive Ocean–Atmosphere Data Set (COADS) to produce a smooth, spatially complete reconstruction. The optimal number of EOF modes for each month’s reconstruction is chosen using large-scale error analysis: a modern subset of the COADS SST is subsampled according to the distribution of observations in the historical month being reconstructed, forming a testbed on which to evaluate the reconstruction based on a variable number of modes. The test bed is least squares fit to the basis EOFs. The difference fields between the reconstruction and a smoothed version of the test bed is found. The total variance of difference fields (residual) is minimized with respect to the...

Adaptive tree-structured self-generating radial basis function network and its performance evaluation

Several algorithms have been proposed to identify a large scale system, such as the neuro-fuzzy GMDH, and the fuzzy modeling using a fuzzy neural network, As another approach, Sanger proposed a tree-structured adaptive network. But in Sanger's network, it is not clear how to determine the initial disposition of bases and the number of bases in each subtree. We propose a nonlinear modeling method called the adaptive tree-structured self-generating radial basis function network (ATree0RBFN). In ATree-RBFN, we take the maximum absolute error (MAE) selection method in order to improve Sanger's model. We combine Sanger's tree-structured adaptive network for an overall model structure with the MAE selection method for a subtree identification problem. In ATree-RBFN, the tuning parameters are not only the coefficients but also the centers and widths of bases, and a subtree can be generated under all leaf nodes. Then, the input-outpu data can be divided into the training data set and the checking data set, and an element of inputs in each subtree is selected according to the corresponding error value from the checking data set. We also demonstrate the effectiveness of the proposed method by solving several numerical examples.