Efficient computation of optimal open-loop controls for stochastic systems

Abstract

Optimal control is a prominent approach in robotics and movement neuroscience, among other fields of science. Methods for deriving optimal choices of action have been classically devised either in deterministic or stochastic settings. Here, we consider a setting in-between that retains the stochastic aspect of the controlled system but assumes deterministic open-loop control actions. The rationale stems from observations about the neural control of movement which highlighted that relatively stable behaviors can be achieved without feedback circuitry, via open-loop motor commands adequately tuning the mechanical impedance of the neuromusculoskeletal system. Yet, effective methods for deriving optimal open-loop controls for stochastic systems are lacking overall. This work presents a continuous-time approach based on statistical linearization techniques for the efficient computation of optimal open-loop controls for a broad class of stochastic optimal control problems. We first show that non-trivial departure from the optimal solutions of classical deterministic and stochastic approaches may arise for simple synthetic examples, thereby stressing the originality of the framework. We then exemplify its potential relevance to the planning of biological movement by showing that a well-known phenomenon in motor control, referred to as muscle co-contraction, occurs naturally. More generally, this stochastic optimal control framework may be suited to other fields where the design of optimal open-loop actions is relevant.