Linear Function Approximation Research Articles

Improved Piecewise Linear Approximation of Nonlinear Functions in Hybrid Control

AbstractThe paper addresses the issue of the PWA approximation of nonlinear functions in hybrid modeling. We propose an original approach to PWA approximation of common nonlinear functions of one and more variables by orthogonal activation function based neural network (OAF NN). Complexity of the PWA system is the basic limitation in model predictive control design. We present a universal linearization method for generating PWA model of a common nonlinear system from process data. The presented method gives good results in precision in proportion to the number of linearization points. The method allows linearization of a nonlinear dynamical system in few number of linearization points for the purpose of hybrid modeling and model predictive control. The proposed method was successfully verified on many case studies for SISO and MIMO systems.

Numerical analysis of composite STEEL-CONCRETE SECTIONS using integral equation of Volterra

AbstractThe paper presents analysis of the stress and deflections changes due to creep in statically determinate composite steel-concrete beam. The mathematical model involves the equation of equilibrium, compatibility and constitutive relationship, i.e. an elastic law for the steel part and an integral-type creep law of Boltzmann — Volterra for the concrete part. On the basis of the theory of the viscoelastic body of Arutyunian-Trost-Bažant for determining the redistribution of stresses in beam section between concrete plate and steel beam with respect to time “t”, two independent Volterra integral equations of the second kind have been derived. Numerical method based on linear approximation of the singular kernal function in the integral equation is presented. Example with the model proposed is investigated.The creep functions is suggested by the model CEB MC90-99 and the “ACI 209R-92 model. The elastic modulus of concrete Ec(t) is assumed to be constant in time ‘t’. The obtained results from the both models are compared.

Inverse Optimality Design for Biological Movement Systems

AbstractThis paper proposes an inverse optimality design method for nonlinear deterministic system and nonlinear stochastic system with multiplicative and additive noises. The new method is developed based on the general Hamilton-Jacobi-Bellman(HJB) equation, and it constructs an estimated cost function using a linear function approximator—Gaussian Radial Basis function, which can recover the original performance criterion for which the given control law is optimal. The performance of the algorithm is illustrated on scalar systems and a complex biomechanical control problem involving a stochastic model of the human arm.

On a concept to avoid electromechanical locking in a piezoelectric finite beam element

AbstractWithin the development process of piezoelectric sensor and actuator applications, numerical tools are essential. These tools solve approximately the electromechanical boundary value problems received from physical descriptions. Since many piezoelectric components are generally thin rod like structures, a piezoelectric finite beam element is presented in this contribution which can be used effectively to analyze a wide range of piezoelectric devices. The mechanical strains and the electric field are coupled by the constitutive relations. This results in incompatibilities within the numerical approximations in case of bending dominated problems, which can be compared to those causing the well‐known shear locking in purely mechanical problems. This effect occurs in standard finite element formulations, where the mechanical and electrical degrees of freedom are derived from standard kinematic assumptions. The present element formulation is based on the classical Timoshenko beam theory, extended by a piezoelectric part. Each finite element node contains eight degrees of freedom: three displacements, three rotations and the electrical potential on the top and the bottom of the beam. The related fields are interpolated with linear approximation functions. In order to overcome the described problem of incompatible approximation spaces, a mixed multi‐field variational approach is introduced, wherein six independent fields are considered. These are the displacements, strains, stresses, electric potential, electric field and the dielectric displacements. It allows for 3D constitutive equations and approximations of the strains and the electric field independent of the linear interpolation functions. Throughout the beam's cross section, a set of special polynomial approaches for strains and electric field components is proposed. This leads to well‐balanced approximation functions regarding coupling of electrical and mechanical fields. Thus, electromechanical locking does not occur as is demonstrated by means of several numerical examples. (© 2010 Wiley‐VCH Verlag GmbH & Co. KGaA, Weinheim)

Robust high performance reinforcement learning through weighted k-nearest neighbors

The aim of this paper is to present (jointly) a series of robust high performance (award winning) implementations of reinforcement learning algorithms based on temporal-difference learning and weighted k - nearest neighbors for linear function approximation. These algorithms, named k NN‐TD ( λ ) methods, where rigorously tested at the Second and Third Annual Reinforcement Learning Competitions (RLC2008 and RCL2009) held in Helsinki and Montreal respectively, where the k NN‐TD ( λ ) method (JAMH team) won in the PolyAthlon 2008 domain, obtained the second place in 2009 and also the second place in the Mountain-Car 2008 domain showing that it is one of the state of the art general purpose reinforcement learning implementations. These algorithms are able to learn quickly, to generalize properly over continuous state spaces and also to be robust to a high degree of environmental noise. Furthermore, we describe a derivation of k NN‐TD ( λ ) algorithm for problems where the use of continuous actions have clear advantages over the use of fine grained discrete actions: the Ex 〈 a 〉 reinforcement learning algorithm.

The effect of numerical integration on the finite element approximation of linear functionals

In this paper, we have studied the effect of numerical integration on the finite element method based on piecewise polynomials of degree k, in the context of approximating linear functionals, which are also known as “quantities of interest”. We have obtained the optimal order of convergence, $${\mathcal{O}(h^{2k})}$$, of the error in the computed functional, when the integrals in the stiffness matrix and the load vector are computed with a quadrature rule of algebraic precision 2k − 1. However, this result was obtained under an increased regularity assumption on the data, which is more than required to obtain the optimal order of convergence of the energy norm of the error in the finite element solution with quadrature. We have obtained a lower bound of the error in the computed functional for a particular problem, which indicates the necessity of the increased regularity requirement of the data. Numerical experiments have been presented indicating that over-integration may be necessary to accurately approximate the functional, when the data lack the increased regularity.

On the quantum and randomized approximation of linear functionals on function spaces

We deal with quantum and randomized algorithms for approximating a class of linear continuous functionals. The functionals are defined on a Holder space of functions f of d variables with r continuous partial derivatives, the rth derivative being a Holder function with exponent ?. For a certain class of such linear problems (which includes the integration problem), we define algorithms based on partitioning the domain of f into a large number of small subdomains, and making use of the well-known quantum or randomized algorithms for summation of real numbers. For N information evaluations (quantum queries in the quantum setting), we show upper bounds on the error of order N ?(?+1) in the quantum setting, and N ?(?+1/2) in the randomized setting, where ? = (r + ?)/d is the regularity parameter. Hence, we obtain for a wider class of linear problems the same upper bounds as those known for the integration problem. We give examples of functionals satisfying the assumptions, among which we discuss functionals defined on the solution of Fredholm integral equations of the second kind, with complete information about the kernel. We also provide lower bounds, showing in some cases sharpness of the obtained results, and compare the power of quantum, randomized and deterministic algorithms for the exemplary problems.

Oscillatory pulses in FitzHugh-Nagumo type systems with cross-diffusion

We study FitzHugh-Nagumo type reaction-diffusion systems with linear cross-diffusion terms. Based on an analytical description using piecewise linear approximations of the reaction functions, we completely describe the occurrence and properties of wavy pulses, patterns of relevance in several biological contexts, in two prototypical systems. The pulse wave profiles arising in this treatment contain oscillatory tails similar to those in travelling fronts. We find a fundamental, intrinsic feature of pulse dynamics in cross-diffusive systems--the appearance of pulses in the bistable regime when two fixed points exist.

Variational Bayes for a Mixed Stochastic/Deterministic Fuzzy Filter

This study, under the variational Bayes (VB) framework, infers the parameters of a Takagi-Sugeno fuzzy filter having deterministic antecedents and stochastic consequents. The aim of this study is to take advantages of the VB framework to design fuzzy-filtering algorithms, which include an automated regularization, incorporation of statistical noise models, and model-comparison capability. The VB method can be easily applied to the linear-in-parameters models. This paper applies the VB method to the nonlinear fuzzy filters without using Taylor expansion for a linear approximation of some nonlinear function. It is assumed that the nonlinear parameters (i.e., antecedents) of the fuzzy filter are deterministic, while linear parameters are stochastic. The VB algorithm, by maximizing a strict lower bound on the data evidence, makes the approximate posterior of linear parameters as close to the true posterior as possible. The nonlinear deterministic parameters are tuned in a way to further increase the lower bound on data evidence. The VB paradigm can be used to design an algorithm that automatically selects the most-suitable fuzzy filter out of the considered finite set of fuzzy filters. This is done by fitting the observed data as a stochastic combination of the different Takagi-Sugeno fuzzy filters such that the individual filters compete with one another to model the data.

Piecewise linear approximations of multivariate functions: A multiresolution-based compression algorithm suitable for circuit implementation

This paper is concerned with a multiresolution approach to the piecewise-linear approximation of multivariate nonlinear continuous functions. The proposed technique has no restrictions on the number of variables the functions depend on, and is based on the use of piecewise-linear “hat” functions. The approximation levels are related to nested function spaces. The multiresolution approach allows one to define a simple model reduction strategy that is based on a proper error definition. The interest in the piecewise-linear approximations and in the hat functions is motivated by the simplicity of their circuit implementations. The efficiency of the method is tested via two benchmark examples, one of which concerns the approximation of the vector field of a nonlinear dynamical system.

Exploring QSAR of hydroxyphenylureas as antioxidants using physicochemical and electrotopological state atom parameters

In the present study, free radical scavenging activity of 36 substituted hydroxyphenylurea derivatives was subjected to classical quantitative structure–activity relationship (QSAR) analyses using physicochemical (hydrophobicity and molar refractivity) and electrotopological state atom parameters. For the development of the QSAR models, statistical techniques such as stepwise multiple linear regression and genetic function approximation (GFA) were used. The developed models indicate an important contribution of the phenolic hydroxyl group of hydroxyphenylureas, apart from that of the urea moiety and piperazine nucleus of the side chain, to the free radical scavenging activity. The presence of substituents at the phenyl ring influences the electron density distribution over the phenolic ring system and modulates the activity. Hydrophobicity is found to contribute positively to the free radical scavenging activity. Based on internal validation (Q 2), external validation () and overall validation criteria (), a GFA model with spline options was found to be the best model (Q 2 = 0.957, = 0.966, = 0.914).

Continuous-action reinforcement learning with fast policy search and adaptive basis function selection

As an important approach to solving complex sequential decision problems, reinforcement learning (RL) has been widely studied in the community of artificial intelligence and machine learning. However, the generalization ability of RL is still an open problem and it is difficult for existing RL algorithms to solve Markov decision problems (MDPs) with both continuous state and action spaces. In this paper, a novel RL approach with fast policy search and adaptive basis function selection, which is called Continuous-action Approximate Policy Iteration (CAPI), is proposed for RL in MDPs with both continuous state and action spaces. In CAPI, based on the value functions estimated by temporal-difference learning, a fast policy search technique is suggested to search for optimal actions in continuous spaces, which is computationally efficient and easy to implement. To improve the generalization ability and learning efficiency of CAPI, two adaptive basis function selection methods are developed so that sparse approximation of value functions can be obtained efficiently both for linear function approximators and kernel machines. Simulation results on benchmark learning control tasks with continuous state and action spaces show that the proposed approach not only can converge to a near-optimal policy in a few iterations but also can obtain comparable or even better performance than Sarsa-learning, and previous approximate policy iteration methods such as LSPI and KLSPI.

A Convergent Online Single Time Scale Actor Critic Algorithm

Actor-Critic based approaches were among the first to address reinforcement learning in a general setting. Recently, these algorithms have gained renewed interest due to their generality, good convergence properties, and possible biological relevance. In this paper, we introduce an online temporal difference based actor-critic algorithm which is proved to converge to a neighborhood of a local maximum of the average reward. Linear function approximation is used by the critic in order estimate the value function, and the temporal difference signal, which is passed from the critic to the actor. The main distinguishing feature of the present convergence proof is that both the actor and the critic operate on a similar time scale, while in most current convergence proofs they are required to have very different time scales in order to converge. Moreover, the same temporal difference signal is used to update the parameters of both the actor and the critic. A limitation of the proposed approach, compared to results available for two time scale convergence, is that convergence is guaranteed only to a neighborhood of an optimal value, rather to an optimal value itself. The single time scale and identical temporal difference signal used by the actor and the critic, may provide a step towards constructing more biologically realistic models of reinforcement learning in the brain.

Sharp Front Capturing Method for Carbon Dioxide Plume Propagation during Injection into a Deep Confined Aquifer

An in-depth understanding of patterns and evolutional characteristics of carbon dioxide plume has significant implications to optimize operational strategies and manage carbon dioxide during and after injection into saline aquifers for long-term geological storage. A sharp-interface mathematical model that describes the displacement process and its interfacial dynamics of carbon dioxide−brine flow in a deep confined saline aquifer was proposed for predicting the propagation of the carbon dioxide plume. On the basis of a conservative level set method (LSM), the governing equation of interfacial dynamics for capturing time-dependent fronts of the carbon dioxide plume was established. It features a unified relationship between the properties of fluids and the saturations in both the single-phase fluid regions, and the interfacial region in the computational domain was derived by the approximation of linear fraction functions. The model equations numerically solved under different operational conditions after...

Piecewise linear approximation of functions of two variables in MILP models

We consider three easy-to-implement methods for the piecewise linear approximation of functions of two variables. We experimentally evaluate their approximation quality, and give a detailed description of how the methods can be embedded in a MILP model. The advantages and drawbacks of the three methods are discussed on numerical examples.

Efficient MILP formulations for the simultaneous optimal peptide tag design and downstream processing synthesis

AbstractNovel and efficient linear formulations are developed for the problem of simultaneously performing an optimal synthesis of chromatographic protein purification processes, and the concomitant selection of peptide purification tags, that result in a maximal process improvement. To this end, two formulations are developed for the solution of this problem: (1) a model that minimizes both the number of chromatographic steps in the final purification process flow sheet and the composition of the tag, by use of weighted objectives, while satisfying minimal purity requirements for the final product; and (2) a model that attempts to find the maximal attainable purity under constraints on the maximum number of separation techniques and tag size. Both models are linearized using a previously developed strategy for obtaining optimal piecewise linear approximations of nonlinear functions. Proposed are models to two case studies based on protein mixtures with different numbers of proteins. Results show that the models are capable of solving to optimality all the implemented cases with computational time requirements of under 1 s, on average. The results obtained are further compared with previous nonlinear and linear models attempting to solve the same problem, and, thus, show that the approach represents significant gains in robustness and efficiency. © 2009 American Institute of Chemical Engineers AIChE J, 2009

Convergence Results for Some Temporal Difference Methods Based on Least Squares

We consider finite-state Markov decision processes, and prove convergence and rate of convergence results for certain least squares policy evaluation algorithms of the type known as LSPE(lambda ). These are temporal difference methods for constructing a linear function approximation of the cost function of a stationary policy, within the context of infinite-horizon discounted and average cost dynamic programming. We introduce an average cost method, patterned after the known discounted cost method, and we prove its convergence for a range of constant stepsize choices. We also show that the convergence rate of both the discounted and the average cost methods is optimal within the class of temporal difference methods. Analysis and experiment indicate that our methods are substantially and often dramatically faster than TD(lambda), as well as more reliable.

Boolean Approximation in Periodic Hilbert Spaces

Babuska introduced the concept of periodic Hilbert spaces in studying optimal approximation of linear functionals. We used these spaces to study the approximation properties of trigonometric interpolation and periodic spline interpolation.

Controlling chaos through local knowledge

We propose a duopoly game where quantity-setting firms have incomplete information about the demand function. At each time step, they solve a profit maximization problem assuming a linear local approximation of the demand function. In particular, we construct an example using the well known Puu’s model with isoelastic demand function and constant marginal costs. An explicit form of the dynamical system that describes the time evolution of the duopoly game with boundedly rational players is given. The main result is the global stability of the system.

Experimental analysis of eligibility traces strategies in temporal difference learning

Temporal difference (TD) learning is a model-free reinforcement learning technique, which adopts an infinite horizon discount model and uses an incremental learning technique for dynamic programming. The state value function is updated in terms of sample episodes. Utilising eligibility traces is a key mechanism in enhancing the rate of convergence. TD(λ) represents the use of eligibility traces by introducing the parameter λ. However, the underlying mechanism of eligibility traces with an approximation function has not been well understood, either from theoretical point of view or from practical point of view. The TD(λ) method has been proved to be convergent with local tabular state representation. Unfortunately, proving convergence of TD(λ) with function approximation is still an important open theoretical question. This paper aims to investigate the convergence and the effects of different eligibility traces. In this paper, we adopt Sarsa(λ) learning control algorithm with a large, stochastic and dynamic simulation environment called SoccerBots. The state value function is represented by a linear approximation function known as tile coding. The performance metrics generated from the simulation system can be used to analyse the mechanism of eligibility traces.