On the study of embedding fuzzy concept into reinforcement learning schemes

Abstract

The paper is to study how to correctly embed fuzzy concept into reinforcement learning schemes with the box‐type structure. In fact, precious work has already proposed the idea of incorporating fuzzy concept into reinforcement learning systems. However, in that approach only two state variables were considered. In our implementation, when four state variables were used with the fuzzification process, the results show a complete disaster. After analyzing the behaviors of the approach, three problems have been identified. The first one is the credit assignment problem about the learning process in the use of the fuzzification process. Since the considered system is of bang‐bang control, all fired boxes may not all agree on the fired action. Thus, in the learning process, such a credit assignment problem must be taken into account. The second one is the weight domination problem. In the early stage, some learned boxes even with very small firing strengths might dominate the action generation process because other boxes are unlearned. Finally, the use of fuzzy schemes in generating the predicted evaluation under the temporal difference mechanism seems to be a negative factor in our example. The modifications to overcome those problems were also proposed in the paper. The results have shown the correctness of our modifications.