Local Models for data-driven learning of control policies for complex systems

Abstract

An approach based on local learning, relying on Nadaraya–Watson models (NWMs), is introduced for the problem of deriving an automatic controller able to exploit data collected during the operation of some complex plant or system by a reference teacher (e.g., a human operator). Such learning approach is particularly useful when the system is too complex to be modeled accurately and/or the task cannot be easily formalized by a cost function, a situation which rules out classic approaches based, e.g., on dynamic programming. Here it is proved that local models are a suitable solution for a real-time employment, since they allow to incorporate new information directly and efficiently without the need of offline training, and new data immediately reflect in improvement of performance. To this purpose, convergence analysis of the method is provided, also considering the case where the reference controller introduces random variations in the training data. Finally, a simulation test, concerning the control of a mechanical system, is provided to showcase the use of local models in an applicative scenario.