Selection of learning parameters for CMAC-based adaptive critic learning

Abstract

The CMAC-based adaptive critic learning structure consists of two CMAC modules: the action and the critic ones. Learning occurs in both modules. The critic module learns to evaluate the system status. It transforms the system response, usually some occasionally provided reinforcement signal, into organized useful information. Based on the knowledge developed in the critic module, the action module learns the control technique. One difficulty in using this scheme lies on selection of learning parameters. In our previous study on the CMAC-based scheme, the best set of learning parameters were selected from a large number of test simulations. The picked parameter values are not necessarily adequate for generic cases. A general guideline for parameter selection needs to be developed. In this study, the problem is investigated. Effects of parameters are studied analytically and verified by simulations. Results provide a good guideline for parameter selection.