A Novel Behavioral Strategy for RoboCode Platform Based on Deep Q-Learning

Hakan Kayakoku,Erkan Bostanci,Ihsan Tolga Medeni,Deepti Mishra,Mehmet Serdar Guzel,Kalyana C Veluvolu

doi:10.1155/2021/9963018

Abstract

This paper addresses a new machine learning-based behavioral strategy using the deep Q-learning algorithm for the RoboCode simulation platform. According to this strategy, a new model is proposed for the RoboCode platform, providing an environment for simulated robots that can be programmed to battle against other robots. Compared to Atari Games, RoboCode has a fairly wide set of actions and situations. Due to the challenges of training a CNN model for such a continuous action space problem, the inputs obtained from the simulation environment were generated dynamically, and the proposed model was trained by using these inputs. The trained model battled against the predefined rival robots of the environment (standard robots) by cumulatively benefiting from the experience of these robots. The comparison between the proposed model and standard robots of RoboCode Platform was statistically verified. Finally, the performance of the proposed model was compared with machine learning based-customized robots (community robots). Experimental results reveal that the proposed model is mostly superior to community robots. Therefore, the deep Q-learning-based model has proven to be successful in such a complex simulation environment. It should also be noted that this new model facilitates simulation performance in adaptive and partially cluttered environments.

Highlights

In the last decade, studies on machine learning and robotics have shown notable improvements compared to previous years with the evolution of technology and the demands of the sector
Within the scope of the mentioned article, seven popular Atari 2600 games were trained with images obtained from the arcade games through the convolutional neural network using the deep reinforcement learning model, and agents that can play 7 games with very high scores were obtained [10]. is study has been revolutionary in terms of satisfactory results and the use of reinforced learning and convolutional neural networks. e concept mentioned in the study was later used in various fields such as image processing, computer vision, robotics. e aforementioned study motivates authors to adapt the deep Q-learning algorithm in a more complex simulation environment, involving multiagent systems and battling strategies
It has been proved that the proposed model is mostly superior to other RoboCode community robots against predefined robots defined in the RoboCode simulation environment, as shown in Figure 11. e only exception is the “Net” robot which produces a higher score against the “Corners” robot

Summary

A Novel Behavioral Strategy for RoboCode Platform Based on Deep Q-Learning

Hakan Kayakoku ,1 Mehmet Serdar Guzel ,2 Erkan Bostanci ,3 Ihsan Tolga Medeni ,4 and Deepti Mishra 5. Is paper addresses a new machine learning-based behavioral strategy using the deep Q-learning algorithm for the RoboCode simulation platform. According to this strategy, a new model is proposed for the RoboCode platform, providing an environment for simulated robots that can be programmed to battle against other robots. E comparison between the proposed model and standard robots of RoboCode Platform was statistically verified. The performance of the proposed model was compared with machine learning basedcustomized robots (community robots). Erefore, the deep Q-learning-based model has proven to be successful in such a complex simulation environment. It should be noted that this new model facilitates simulation performance in adaptive and partially cluttered environments

Introduction

RoboCode and Background

Methodology