Linear Approximation Of Nonlinear Function Research Articles

A Simultaneous Training and Input Selection Algorithm for Classification Problems Using Piecewise Approximations

Heart disease diagnosis using few measurements is a challenging an important task considering the increasing population. Artificial Neural Networks (ANNs) are promising mathematical architectures once the training is performed in an elegant manner to avoid theoretical challenges related to high nonlinearity, nonconvexity using few input variables to ensure generalization capability. This study shows the impact of the piecewise linear approximation of nonlinear functions in ANN architecture and training problem to benefit from the mixed integer linear problem formulation for the simultaneous input selection and training to obtain mixed integer programming based ANN (MIP-ANN). Proposed formulation is further tailored through linking constraints to remove the connections from the eliminated inputs to favor parameter identifiability. A publicly available dataset is considered as a case study of whose results are also compared to traditional ANN with all inputs (FC-ANN) and a relatively more straightforward but common input selection method (SKB-ANN). The results provide a comparable performance despite significant reduction in the input space in addition to significant computational and theoretical advantages thanks to advanced formulation.

Speaker verification from mixture of speech and non-speech audio signals via using pole distribution of piecewise linear predictive coding coefficients

This paper examines a method of speaker verification from mixture of speech and non-speech audio signals via using pole distribution of speech signals. The poles of speech signal derived from the all-pole speech production model are obtained via a neural net called bagging CAN2 (competitive associative net 2) for learning efficient piecewise linear approximation of nonlinear functions. We show an analysis that poles of the mixture of two audio signals are expected to show three typical phenomena. Namely, with the change of the weight of the mixture, there will be (1) a sudden change of the number of poles, (2) a gradual change of the number of poles, and (3) novel poles not in the original signals. By means of experiments, we show that there are poles exhibiting those phenomena, and they are considered to affect the performance on speaker verification. As a performance measure of speaker verification, recall shows large decrease with the change of speech signal from unmixed to mixed, especially for the mixture with violin signal as well as with speech signal. Furthermore, precision has larger value than recall for the same dataset of mixture of audio signals. Finally, we show the usefulness of the present method in further researches.

Performance improvement via bagging in probabilistic prediction of chaotic time series using similarity of attractors and LOOCV predictable horizon

Recently, we have presented a method of probabilistic prediction of chaotic time series. The method employs learning machines involving strong learners capable of making predictions with desirably long predictable horizons, where, however, usual ensemble mean for making representative prediction is not effective when there are predictions with shorter predictable horizons. Thus, the method selects a representative prediction from the predictions generated by a number of learning machines involving strong learners as follows: first, it obtains plausible predictions holding large similarity of attractors with the training time series and then selects the representative prediction with the largest predictable horizon estimated via LOOCV (leave-one-out cross-validation). The method is also capable of providing average and/or safe estimation of predictable horizon of the representative prediction. We have used CAN2s (competitive associative nets) for learning piecewise linear approximation of nonlinear function as strong learners in our previous study, and this paper employs bagging (bootstrap aggregating) to improve the performance, which enables us to analyze the validity and the effectiveness of the method.

The Best Piecewise Linearization of Nonlinear Functions

In this paper, we propose a method for finding the best piecewise linearization of nonlinear functions. For this aim, we try to obtain the best approximation of a nonlinear function as a piecewise linear function. Our method is based on an optimization problem. The optimal solution of this optimization problem is the best piecewise linear approximation of nonlinear function. Finally, we examine our method to some examples.

An intelligent UFastSLAM with MCMC move step

FastSLAM is a framework for simultaneous localization and mapping (SLAM). However, FastSLAM algorithm has two serious drawbacks, namely the linear approximation of nonlinear functions and the derivation of the Jacobian matrices. For solving these problems, UFastSLAM has been recently proposed. However, UFastSLAM is inconsistent over time due to the loss of particle diversity that is caused mainly by the particle depletion in the resampling step and incorrect a priori knowledge of process and measurement noises. To improve consistency, intelligent UFastSLAM with Markov chain Monte Carlo (MCMC) move step is proposed. In the proposed method, the adaptive neuro-fuzzy inference system supervises the performance of UFastSLAM. Furthermore, the particle impoverishment caused by resampling is restrained after the resample step with MCMC move step. Simulations and experiments are presented to evaluate the performance of algorithm in comparison with UFastSLAM. The results show the effectiveness of the proposed method.

A NOVEL ADAPTIVE NEURO-FUZZY UNSCENTED KALMAN FILTER FOR SLAM

Extended Kalman filter (EKF) has been used as a popular choice to solve simultaneous localization and mapping (SLAM) problem. However, SLAM algorithm based on EKF-SLAM has two serious drawbacks, namely the linear approximation of nonlinear functions and the calculation of Jacobin matrices. For solving these problems, SLAM algorithm based on unscented Kalman filter (UKF-SLAM) has been recently proposed. However, the performance of the UKF-SLAM and thus the quality of the estimation depends on the correct a priori knowledge of process and measurement noise covariance matrices respectively denoted by Qk and Rk. Imprecise knowledge of these statistics can cause significant degradation in performance. This article proposes the development of an adaptive neuro-fuzzy UKF (ANFUKF) for SLAM. The Adaptive neuro-fuzzy attempts to estimate the elements of Rk matrix in the UKF-SLAM algorithm at each sampling instant when measurement updating step is carried out. The adaptive neuro-fuzzy inference system (ANFIS) supervises the performance of the UKF-SLAM with the aim of reducing the mismatch between the theoretical and actual covariance of the innovation sequences. The free parameters of ANFIS are trained using the steepest gradient descent (GD) to minimize the differences of the actual value of the covariance of the residual with its theoretical value as much as possible. The simulation results show the effectiveness of the proposed algorithm.

Improved Piecewise Linear Approximation of Nonlinear Functions in Hybrid Control

AbstractThe paper addresses the issue of the PWA approximation of nonlinear functions in hybrid modeling. We propose an original approach to PWA approximation of common nonlinear functions of one and more variables by orthogonal activation function based neural network (OAF NN). Complexity of the PWA system is the basic limitation in model predictive control design. We present a universal linearization method for generating PWA model of a common nonlinear system from process data. The presented method gives good results in precision in proportion to the number of linearization points. The method allows linearization of a nonlinear dynamical system in few number of linearization points for the purpose of hybrid modeling and model predictive control. The proposed method was successfully verified on many case studies for SISO and MIMO systems.

Converting general nonlinear programming problems into separable programming problems with feedforward neural networks

In this paper we present a method for converting general nonlinear programming (NLP) problems into separable programming (SP) problems by using feedforward neural networks (FNNs). The basic idea behind the method is to use two useful features of FNNs: their ability to approximate arbitrary continuous nonlinear functions with a desired degree of accuracy and their ability to express nonlinear functions in terms of parameterized compositions of functions of single variables. According to these two features, any nonseparable objective functions and/or constraints in NLP problems can be approximately expressed as separable functions with FNNs. Therefore, any NLP problems can be converted into SP problems. The proposed method has three prominent features. (a) It is more general than existing transformation techniques; (b) it can be used to formulate optimization problems as SP problems even when their precise analytic objective function and/or constraints are unknown; (c) the SP problems obtained by the proposed method may highly facilitate the selection of grid points for piecewise linear approximation of nonlinear functions. We analyze the computational complexity of the proposed method and compare it with an existing transformation approach. We also present several examples to demonstrate the method and the performance of the simplex method with the restricted basis entry rule for solving SP problems.

Converting general nonlinear programming problems into separable programming problems with feedforward neural networks

In this paper we present a method for converting general nonlinear programming (NLP) problems into separable programming (SP) problems by using feedforward neural networks (FNNs). The basic idea behind the method is to use two useful features of FNNs: their ability to approximate arbitrary continuous nonlinear functions with a desired degree of accuracy and their ability to express nonlinear functions in terms of parameterized compositions of functions of single variables. According to these two features, any nonseparable objective functions and/or constraints in NLP problems can be approximately expressed as separable functions with FNNs. Therefore, any NLP problems can be converted into SP problems. The proposed method has three prominent features. (a) It is more general than existing transformation techniques; (b) it can be used to formulate optimization problems as SP problems even when their precise analytic objective function and/or constraints are unknown; (c) the SP problems obtained by the proposed method may highly facilitate the selection of grid points for piecewise linear approximation of nonlinear functions. We analyze the computational complexity of the proposed method and compare it with an existing transformation approach. We also present several examples to demonstrate the method and the performance of the simplex method with the restricted basis entry rule for solving SP problems.