Least-squares Learning Research Articles

A note on least-squares learning mechanism

Abstract Marcet and Sargent (1989a,b) recently applied the approach of Ljung (1977) to analyze the convergence of the least-squares learning mechanism. The purpose of this short paper is to point out and to correct two types of errors in Marcet and Sargent's analysis.

Least-Squares Learning and the Stability of Equilibria with Externalities

This paper studies the stability of competitive equilibria in a model of aggregate employment when the representative agent uses a least-squares forecasting procedure. It is shown that the Pareto inferior low employment steady state is always unstable under least-squares learning, even if it is stable under perfect foresight. The high employment steady state is stable under learning if and only if it is saddle point stable under perfect foresight. This weakens multiple equilibrium theories of coordination failure that purport to explain persistently high unemployment. The Pareto superior high employment steady state will be the focal point of individual forecasting.

Fuzzy basis functions, universal approximation, and orthogonal least-squares learning

Fuzzy systems are represented as series expansions of fuzzy basis functions which are algebraic superpositions of fuzzy membership functions. Using the Stone-Weierstrass theorem, it is proved that linear combinations of the fuzzy basis functions are capable of uniformly approximating any real continuous function on a compact set to arbitrary accuracy. Based on the fuzzy basis function representations, an orthogonal least-squares (OLS) learning algorithm is developed for designing fuzzy systems based on given input-output pairs; then, the OLS algorithm is used to select significant fuzzy basis functions which are used to construct the final fuzzy system. The fuzzy basis function expansion is used to approximate a controller for the nonlinear ball and beam system, and the simulation results show that the control performance is improved by incorporating some common-sense fuzzy control rules.

A note on least-squares learning procedures and classification by neural network models

Neural network models are considered as mathematical classifiers whose inputs comprise random variables generated according to arbitrary stationary class distributions, and the implication of learning based on minimization of sum-square classification error over a training set of these observations for which class assignments are absolutely determined is addressed. Expectations for network outputs in such cases are weighted least-squares approximations to a posteriori probabilities for the classes, which justifies interpretation of network outputs as indicating degree of confidence in class membership. The author demonstrates this with a straightforward proof in which class probability densities are regarded as primitives and which for simplicity does not rely on probability theory or statistics. The author cites more detailed results giving conditions for consistency of the estimators and discusses some issues relating to the suitability of neural network models and back-propagation training for approximation of conditional probabilities in classification tasks.

Convergence of Least-Squares Learning in Environments with Hidden State Variables and Private Information

We study the convergence of recursive least-squares learning schemes in economic environments in which there is private information. The presence of private information leads to the presence of hidden state variables from the viewpoint of particular agents. By applying theorems of Ljung, we extend some of our earlier results to characterize conditions under which a system governed by leastsquares learning will eventually converge to a rational expectations equilibrium. We apply insights from the learning results to formulate and compute the equilibrium of a version of Townsend's model.