An empirical approach to optimal self-control

Abstract

This paper describes an application of the empirical modeling of natural phenomena to the optimal self-control of an autonomous system in a chaotic environment. The system consists of a network of sensors, a modeler, a controller, a plant, and a utility estimator. The modeler contains a self-organizing neural network and a conditional average estimator. An empirical model, which incorporates the influences from the environment, the system response and the utility, is formed in the modeler during training. The sensors provide signals representing the joint state of the environment and the system, while the utility estimator transforms these signals into a utility signal. A vector comprising the joint state, the control, and the utility variable is then utilized in a self-organized adaptation of prototype vectors. During adaptation, samples of the control variable are generated either randomly or by a reinforcement procedure, while during application the optimal control variable is estimated by a conditional average taken over the prototype vectors. The control variable drives the plant, and improves its performance. The method is demonstrated, using as examples the optimal selection of cutting depth in a chaotic manufacturing process, the self-stabilization of a randomly influenced system, and reversing a vehicle.