Dynamic programming in the presence of indeterminacy

Abstract

480 Indeterminacy must be addressed in optimization in many scientific and technical fields, such as the reg ulation of production processes and the creation of new materials and technologies. The required out comes are optimal parameter values, optimal trajecto ries toward the goal, and often models of the process (response surfaces). Traditional dynamic program ming permits the derivation of optimal trajectories, but only where a model of the process exists—that is, only in conditions of complete determinacy. Optimi zation methods of search type permit the determina tion of optimal parameter values but do not provide optimal trajectories for their attainment. Such meth ods do not provide means of deriving a model of the process. To derive optimal parameter values, optimal trajec tories, and a model of the process, we propose integra tion of dynamic programming, evolutionary planning (search based optimization), and adaptive identifica tion. The proposed method is based on a successive iterative approach: formulation of experiments within a small region of the response surface; construction of a mathematical model in that region; extrapolation of the model to the whole response surface; identifica tion of the optimal trajectory on the extrapolated sur face; determination of the trajectory within a small specified vicinity; subsequent repetition of these oper ations for a new region of the response surface in the direction of motion of the optimal trajectory; and con tinuation until the optimal trajectories and optimal parameters have been established. So as to be specific, we consider examples in which the response surface is a smooth single extremum sur face (Fig. 1a) or has a more complex configuration with gullies (Fig. 1b). First we select the coordinates of the initial point A0 in Fig. 2. Around this point, we construct a regular simplex and conduct experiments to determine the values of the response surface function. From the results, by the least squares method, we construct a model of the form y = a1x1 + a2x2 + a0. The model is extended over the whole region of investigation. Then, by dynamic programming, we determine the optimal trajectory from point A1 to point A0 in the plane y I = ⎯1.9x1 – 1.9x2 + 1.8. The model is extended to a distance specified by the simplex increment (points 2 and 3 in Fig. 3a). The resulting trajectory is fixed in region I (seg ment A0–A2 in Fig. 2). Around point A2, which is the final point of the trajectory, we again construct a regu Dynamic Programming in the Presence of Indeterminacy