Using Simulated Annealing Algorithm to Solve the Optimal Control Problem

Abstract

A lot of research has been done in Automatic Control Systems during the last decade and more recently in discrete control systems due to the popular use of powerful personal computers. This work presents an approach to solve the Discrete-Time Time Invariant Linear Quadratic (LQ) Optimal Control problem which minimizes a specific performance index (either minimum time and/or minimum energy). The design approach presented in this paper transforms the LQ problem into a combinatorial optimization problem. The Simulated Annealing (SA) algorithm is used to carry out the optimization. Simulated Annealing is basically an iterative improvement strategy augmented by a criterion for occasionally accepting configurations with higher values of the performance index (Malhorta et al., 1991; Martinez-Alfaro & Flugrad, 1994; Martinez-Alfaro & Ulloa-Perez, 1996; Rutenbar, 1989). Given a performance index J(z) (analog to the energy of the material) and an initial configuration z0, the iterative improvement solution is seeked by randomly perturbing z0. The Metropolis algorithm (Martinez-Alfaro & Flugrad, 1994; Martinez-Alfaro & Ulloa-Perez, 1996; Rutenbar, 1989) was used for acceptance/rejection of the perturbed configuration. In this design approach, SA was used to minimized the performance index of the LQ problem and as result obtaining the values of the feedback gain matrix K that make stable the feedback system and minimize the performance index of the control system in state space representation (Ogata, 1995). The SA algorithm starts with an initial feedback gain matrix K and evaluates the performance index. The current K is perturbed to generate another Knew and the performance index is evaluated. The acceptance/rejection criteria is based on the Metropolis algorithm. This procedure is repeated under a cooling schedule. Some experiments were performed with first through third order plants for Regulation and Tracking, Single Input Single Output (SISO) and Multiple Input Multiple Output (MIMO) systems. Matlab and Simulink were used as simulation software to carry out the experiments. The parameters of the SA algorithm (perturbation size, initial temperature, number ofMarkov chains, etc.) were specially tunned for each plant. Additional experimentswere performedwith non-conventional performance indices for tracking problems (Steffanoni Palacios, 1998) where characteristics like maximum overshoot max(y(k)− r(k)), manipulation softness index |u(k + 1)− u(k)|, output softness index |y(k+ 1)− y(k)|, and the error magnitude |r(k)− y(k)|. 1