Least-squares Learning Research Articles

The anatomy of sentiment-driven fluctuations

We show that sentiments - self-fulfilling changes in beliefs that are orthogonal to fundamentals - can drive persistent aggregate fluctuations under rational expectations in a beauty-contest game. Such fluctuations can occur even in the absence of exogenous aggregate fundamental shocks. Moreover, sentiments alter the volatility and persistence of aggregate outcomes in response to fundamental shocks. We provide (i) necessary conditions under which sentiments can affect aggregate outcomes in equilibrium and (ii) conditions under which sentiments drive persistent fluctuations and when they only affect aggregate outcomes contemporaneously. We also show that sentiment equilibria are stable under least-squares learning while the fundamental equilibrium is not.

A semi-supervised linear-nonlinear prediction system for tumbler strength of iron ore sintering process with imbalanced data in multiple working modes

An iron ore sintering is a process that provides qualified sinter for the blast-furnace iron-making process. The tumbler strength is an important physical quality index of the sinter. Precise prediction of the tumbler strength is essential for solving the problem of how to improve it. In this study, a semi-supervised prediction system was devised for the tumbler strength of an iron ore sintering process. First, the process was briefly described with an analysis of the process characteristics. The characteristics of the existence of imbalanced data in multiple working modes, the lack of labeled samples, and the coexistence of linear–nonlinear components were taken into consideration for building the system. Then, the configuration of the prediction system was devised based on the characteristics. The system consists of three parts: working-modes decomposition based on a Gaussian mixture model (GMM) considering the existence multiple working modes, a GMM based just-in-time learning for nearest-samples selection in the relevant working modes considering the existence of imbalanced data, and the development of a semi-supervised linear–nonlinear least-square learning network considering the existence of the linear–nonlinear component and lack of labeled samples. Finally, comparisons of simulation results using actual run process data with scarce and abundant labeled samples verified the effectiveness of the proposed system. In addition, results of industrial application also verified the effectiveness of the prediction system.

Heterogeneous expectations, forecasting behaviour and policy experiments in a hybrid Agent-based Stock-flow-consistent model

This paper presents a hybrid agent-based stock-flow-consistent model featuring heterogeneous banks, purposely built to examine the effects of variations in banks’ expectations formation and forecasting behaviour and to conduct policy experiments with a focus on monetary and prudential policy. The model is initialised to a deterministic stationary state and a subset of its free parameters are calibrated empirically in order to reproduce characteristics of UK macro-time-series data. Experiments carried out on the baseline focus on the expectations formation and forecasting behaviour of banks through allowing banks to switch between forecasting strategies and having them engage in least-squares learning. Overall, simple heuristics are remarkably robust in the model. In the baseline, which represents a relatively stable environment, the use of arguably more sophisticated expectations formation mechanisms makes little difference to simulation results. In a modified version of the baseline representing a less stable environment alternative heuristics may in fact be destabilising. To conclude the paper, a range of policy experiments is conducted, showing that an appropriate mix of monetary and prudential policy can considerably attenuate the macroeconomic volatility produced by the model.

From CATS to CAOS: Fiscal Multipliers and Agents’ Expectations in a Macroeconomic Agent-Based Model

This paper uses a macroeconomic agent-based model building on Delli Gatti et al. (2011) to investigate the influence of agents’ expectations and consumption choices on government expenditure multipliers. Following a thorough investigation of the size of the multiplier in the pre-existing baseline model, a modification is introduced, allowing agents to engage in inter-temporal optimization of consumption subject to a budget constraint which is based on estimates of future income. Compared to the baseline, the fiscal multiplier is strongly affected by this alternative consumption behavior, becoming significantly smaller. In a further step, agents’ beliefs about the effects of government expenditure shocks are explicitly introduced. In the case of exogenously imposed beliefs coupled either with adaptation of individual beliefs or switching behavior between different types of beliefs, it is shown that both optimistic and pessimistic expectations can be temporarily self-fulfilling and either increase or decrease the value of the multiplier. Both forms of belief dynamics also allow for the incorporation of announcement effects of fiscal policy. In a final experiment, agents are allowed to engage in least-squares learning in order to gain an estimate of the effect of government expenditure shocks on future income. It is shown that under least squares learning, beliefs are ‘rational’ insofar as they lead to broadly correct predictions on average. The paper hence contributes to addressing aspects of the Lucas critique as applied to macro-ABMs, since agents react systematically (and reasonably) to announcements of changes in fiscal policy.

A semi-supervised linear–nonlinear least-square learning network for prediction of carbon efficiency in iron ore sintering process

An iron ore sintering is a large energy-consuming process. The energy mainly comes from the combustion of carbon. Improving the carbon efficiency is beneficial to cost saving and environmental protection. The carbon efficiency has to be predicted before it can be improved. A semi-supervised linear–nonlinear least-square learning network (LLLN) was devised based on the process characteristics for the prediction of the carbon efficiency. First, a new comprehensive carbon ratio (CCR) that takes into account the coke residual was proposed for estimating the carbon efficiency. Then, the process characteristics that are concerned in building the model were presented. They are the existence of linear–nonlinear component and limited labeled samples. After that, a semi-supervised LLLN (SS-LLLN) approach that takes into account the process characteristics was presented for the prediction of the CCR. Last, actual run data was collected to verify the effectiveness of the proposed method. The error distribution, accuracy, and overfitness of an extreme learning machine (ELM), a semi-supervised ELM, an LLLN and an SS-LLLN were compared, which shows the effectiveness of the SS-LLLN.

Error analysis of the moving least-squares regression learning algorithm with β-mixing and non-identical sampling

We consider the regression learning based on the moving least-squares framework for the polynomially β-mixing samples. The rigorous error analysis is carried out by using the independent-blocks technique in the sample error estimates. We derive the satisfactory learning rate that can be arbitrarily close to the best rate , when the sequence of marginal distributions converges exponentially to some marginal distribution in the dual of a Hölder space.

Dehydration modeling of Cordyceps militaris in mid-infrared-assisted convection drying system: Using low-field nuclear magnetic resonance with the aid of ELM and PLSR

The changes in moisture content and shrinkage ratio of Cordyceps militaris during mid-infrared-assisted convection drying (MIRCD) with different drying temperatures (40, 50, and 60 °C) and velocities of airflow (1 and 2 ms−1) were studied. The relationship between low-field nuclear magnetic resonance (LF-NMR) information and moisture content/shrinkage ratio was modeled using partial least-squares regression (PLSR) and extreme learning machine (ELM). Results indicated that the influence of drying temperature was more pronounced than that of air flow velocity. Both types of models showed good predictive ability with R2>0.90. The ELM models exhibited superior predictive performance than that of the PLSR models.

Convergence rate for the moving least-squares learning with dependent sampling

We consider the moving least-squares (MLS) method by the regression learning framework under the assumption that the sampling process satisfies the α-mixing condition. We conduct the rigorous error analysis by using the probability inequalities for the dependent samples in the error estimates. When the dependent samples satisfy an exponential α-mixing, we derive the satisfactory learning rate and error bound of the algorithm.

Water Leakage Detection for Complex Pipe Systems Using Hybrid Learning Algorithm Based on ANFIS Method

In most city water distribution systems, a considerable amount of water is lost because of leaks occurring in pipes. Moreover, an unobservable fluid leakage fault that may occur in a hazardous industrial system, such as nuclear power plant cooling process or chemical waste disposal, can cause both environmental and economical disasters. This situation generates crucial interest for industry and academia due to the financial cost related with public health risks, environmental responsibility, and energy efficiency. In this paper, to find a reliable and economic solution for this problem, adaptive neuro fuzzy inference system (ANFIS) method which consists of backpropagation and least-squares learning algorithms is proposed for estimating leakage locations in a complex water distribution system. The hybrid algorithm is trained with acceleration, pressure, and flow rate data measured through the sensors located on some specific points of the complex water distribution system. The effectiveness of the proposed method is discussed comparing the results with the current methods popularly used in this area.

Error analysis of the moving least-squares method with non-identical sampling

ABSTRACTWe derive the convergence rate of the moving least-squares learning algorithm for regression under the assumption that the samples are drawn from a non-identical sequence of probability measures. The error analysis is carried out by analysing the drift error and using the probability inequalities for the non-identical sampling. When the sequence of marginal distributions converges exponentially to marginal distribution in the dual of a Hölder space, we obtain the satisfactory capacity dependent error bounds of the algorithm that can be arbitrarily close to the rate .

Learning to live in a liquidity trap

The Taylor rule in combination with the zero lower bound on nominal rates has been shown to create an unintended liquidity-trap equilibrium. The relevance of this equilibrium has been challenged on the basis that it is not stable under least-square learning. In this paper, we show that the liquidity-trap equilibrium is stable under social learning. The learning mechanism we employ includes three realistic elements: mutation, crossover, and tournaments. We show that agents can learn to have pessimistic sentiments about the central bank’s ability to generate price growth, giving rise to a stochastically stable environment characterized by deflation and stagnation.This article is part of a Special Issue entitled “Fiscal and Monetary Policies”.

Force and torque parameter estimation for a 4-pole hybrid electromagnet by ANFIS hybrid learning algorithm

In this study, a force and torque estimation method based on an adaptive neuro-fuzzy inference system (ANFIS) has been developed to get rid of multiple integral calculations of air gap coefficients that cause time delay for magnetic levitation control applications. During magnetic levitation applications that contain a 4-pole hybrid electromagnet, multiple integral calculations have to be done for obtaining air gap permanence parameters, and these parameters are needed to calculate force and torque parameters that are produced by the poles of the hybrid electromagnet, which means, if time delay occurs for calculation of permanence parameters, actual force and actual torque values that are produced by hybrid electromagnet's poles cannot be exactly known; thus, the advantage of having an exact model of the system gets lost and, as a result, the controller's performance goes down. To address a solution, an ANFIS using a hybrid learning algorithm consisting of backpropagation and least-squares learning methods is proposed to estimate force and torque parameters using training data already obtained using multiple integral calculations before.

Forecast combination, non-linear dynamics, and the macroeconomy

This paper introduces the concept of a Forecast Combination Equilibrium to model boundedly rational agents who combine a menu of different forecasts in a way that mimics the behavior of actual forecasters. The equilibrium concept is consistent with rational expectations under certain conditions, while also permitting multiple, distinct, self-fulfilling equilibria, many of which are stable under least-squares learning. The equilibrium concept is applied to a Lucas-type monetary model and to a Fisherian monetary model with a Taylor rule. The existence of multiple equilibria is shown to depend on the aggressiveness of monetary policy in both models. In the latter, a more aggressive response to inflation is required in the Taylor rule than is typically found in this class of model to ensure a unique and learnable equilibrium. Real-time learning simulations with a constant gain illustrate some appealing properties of this approach including time-varying volatility and sharp movements in inflation, similar to actual data, while assuming only i.i.d. random shocks.

The stability of learning prior to an anticipated change in the target inflation rate

I examine the stability of least-squares learning in a flexible-price endowment economy when there is an anticipated change in the inflation target. Monetary policy follows a Taylor rule and a change in the target is announced several periods in advance. I show that highly plausible parameter values may lead to short-run instability and that the conditions for the stability of learning are more demanding soon after the policy change is announced. Also, the farther in advance the change is announced the narrower is the range of policy parameters that generate stability. This is because with a long transition, any estimation error is projected farther into the future and its effects are therefore magnified. These results suggest that advance communication of a change in the inflation target may destabilise inflation in the short run; when the central bank changes its inflation target, a brief transition may be preferable.

Complex Learning in Bio-plausible Memristive Networks.

The emerging memristor-based neuromorphic engineering promises an efficient computing paradigm. However, the lack of both internal dynamics in the previous feedforward memristive networks and efficient learning algorithms in recurrent networks, fundamentally limits the learning ability of existing systems. In this work, we propose a framework to support complex learning functions by introducing dedicated learning algorithms to a bio-plausible recurrent memristive network with internal dynamics. We fabricate iron oxide memristor-based synapses, with well controllable plasticity and a wide dynamic range of excitatory/inhibitory connection weights, to build the network. To adaptively modify the synaptic weights, the comprehensive recursive least-squares (RLS) learning algorithm is introduced. Based on the proposed framework, the learning of various timing patterns and a complex spatiotemporal pattern of human motor is demonstrated. This work paves a new way to explore the brain-inspired complex learning in neuromorphic systems.

Intelligent Dynamic Acquisition Learning Fuzzy Modeling Systems Design

Technologies of fuzzy knowledge-based discovery can be exploited to extract appropriate behaviors for identified modeling problems. A proposed novel data acquisition learning algorithm (DALA) is dynamically applied to examine the initial recognitions of the training data set, where it can automatically establish the number of cluster centers and their associated positions. The characteristics of the training data set are dynamically mined by the DALA, and the primary architecture of the fuzzy system is initially represented with the collected information. Because the prototypes of the collected training data are determined with DALA, the initial populations of the particle swarm optimization (PSO) are generated and distributed around them. The hybrid PSO and recursive least-squares (RLS) learning schemes referred to as RPSO are applied to design an appropriate fuzzy modeling system. The approximation of the proposed fuzzy modeling system shows that it automatically determines suitable fuzzy membership functions and resolves the local optimal problem of identifying appropriate parameters for fuzzy systems to swiftly approach desired functions. A comparison between computer simulations and other modeling methods shows the excellent performance of the dynamic acquisition learning fuzzy modeling system in explaining nonlinear and inverted pendulum function approximations. Simulation results show that the generated fuzzy system with the DALA scheme can be adapted to balance the pole position for various initial conditions. The proposed control rules swiftly recover the sudden and large imported noise.

Adaptive learning and distributional dynamics in an incomplete markets model

Recent research shows that several DSGE models provide a closer fit to the data under adaptive learning. This paper extends this research by introducing adaptive learning in the model of Krusell and Smith (1998) with uninsurable idiosyncratic risks and aggregate uncertainty. A first contribution of this paper establishes that the equilibrium of this framework is stable under least-squares learning. The second contribution consists of showing that bounded rationality enhances the ability of this model to match the distribution of income in the US. Learning increases significantly the Gini coefficients because of the opposite effects on consumption of the capital-rich and of the capital-poor agent. The third contribution is an empirical exercise that shows that learning can account for increases in the income Gini coefficient of up to 25% in a period of 28 years. Overall, these findings suggest that adaptive learning has important distributional repercussions in this class of models.

Parallelized Online Regularized Least-Squares for Adaptive Embedded Systems

The authors introduce a machine learning approach based on parallel online regularized least-squares learning algorithm for parallel embedded hardware platforms. The system is suitable for use in real-time adaptive systems. Firstly, the system can learn in online fashion, a property required in real-life applications of embedded machine learning systems. Secondly, to guarantee real-time response in embedded multi-core computer architectures, the learning system is parallelized and able to operate with a limited amount of computational and memory resources. Thirdly, the system can predict several labels simultaneously. The authors evaluate the performance of the algorithm from three different perspectives. The prediction performance is evaluated on a hand-written digit recognition task. The computational speed is measured from 1 thread to 4 threads, in a quad-core platform. As a promising unconventional multi-core architecture, Network-on-Chip platform is studied for the algorithm. The authors construct a NoC consisting of a 4x4 mesh. The machine learning algorithm is implemented in this platform with up to 16 threads. It is shown that the memory consumption and cache efficiency can be considerably improved by optimizing the cache behavior of the system. The authors’ results provide a guideline for designing future embedded multi-core machine learning devices.

Learning about Risk and Return: A Simple Model of Bubbles and Crashes

This paper demonstrates that an asset pricing model with least-squares learning can lead to bubbles and crashes as endogenous responses to the fundamentals driving asset prices. When agents are risk-averse they need to make forecasts of the conditional variance of a stock's return. Recursive updating of both the conditional variance and the expected return implies several mechanisms through which learning impacts stock prices. Extended periods of excess volatility, bubbles, and crashes arise with a frequency that depends on the extent to which past data is discounted. A central role is played by changes over time in agents' estimates of risk. (JEL D81, D83, E32, G01, G12)

Concentration estimates for the moving least-square method in learning theory

In this paper we apply a concentration technique to improve the convergence rates for a moving least-square learning algorithm for regression. The concentration technique allows us to obtain a sharper bound for the sample error.