Separation conditions and criteria for uniform approximation of input-output maps

Abstract

We consider multidimensional shift-invariant input–output maps G from a relatively compact set of functions S to a set of real-valued functions, and we give criteria under which these maps can be uniformly approximated arbitrarily well using a certain structure consisting of a not-necessarily linear dynamic part followed by a non-linear memoryless section that may contain sigmoids or radial basis functions, etc. The dynamic part is comprised of a finite number of dynamic maps h1,…,hj drawn from a set H of maps that satisfy a certain continuity condition. In our results certain separation conditions, of the kind associated with the Stone–Weierstrass theorem, play a prominent role. Here they emerge as criteria for approximation, and not just sufficient conditions under which an approximation exists. In particular, one of the theorems given is a result to the effect that universal approximation can be achieved using the structure we consider if and only if the set H satisfies the separation condition that (hu1)(0)≠(hu2)(0) for some h∈H whenever u1,u2∈cl(S) and u1≠u2 (where cl(S) denotes the closure of S). This holds even if the elements of H are not linear. © 1998 John Wiley & Sons, Ltd.