Q-Learning Based Failure Detection and Self-Recovery Algorithm for Multi-Robot Domains

Hatice Hilal Ezercan Kayir

doi:10.5755/j01.eie.25.2.23197

Abstract

Task allocation is the essential part of multi-robot coordination researches and it plays a significant role to achieve desired system performance. Uncertainties in multi-robot systems’ working environment due to nature of them are the major hurdle for perfect coordination. When learning-based task allocation approaches are used, firstly robots learn about their working environment and then they benefit from their experiences in future task allocation process. These approaches provide useful solutions as long as environmental conditions remain unchanged. If permanent changes in environment characteristics or some failure in multi-robot system occur undesirably e.g. in disaster response which is a good example to represent such cases, the previously-learned information becomes invalid. At this point, the most important mission is to detect the failure and to recover the system initial learning state. For this purpose, Q-learning based failure detection and self-recovery algorithm is proposed in this study. According to this approach, multi-robot system checks whether these variations permanent, then recover the system to learning state if it is required. So, it provides dynamic task allocation procedure having great advantages against unforeseen situations. The experimental results verify that the proposed algorithm offer efficient solutions for multi-robot task allocation problem even in systemic failure cases. DOI: 10.5755/j01.eie.25.2.23197

Highlights

In recent years, multi-robot systems (MRS) have become more interested in a lot of areas varying from small indoor applications like home or office serving, museum guiding to more complex and sometimes dangerous fields such as search-and-rescue, fire fighting, underwater researches, mining, etc
The key issue to benefit from these advantages and to reach desired system performance in MRS is that the multi-robot coordination should be done precisely and accurately [2]
The proposed algorithm is realized on a heterogeneous MRS with six robots (R1, R2, R3, R4, R5, R6,) capable of executing five different tasks (T1, T2, T3, T4, T5)

Summary

INTRODUCTION

Multi-robot systems (MRS) have become more interested in a lot of areas varying from small indoor applications like home or office serving, museum guiding to more complex and sometimes dangerous fields such as search-and-rescue, fire fighting, underwater researches, mining, etc. In the studies mentioned above, proposed approaches use instant decisions or actions of robots [13] or they require to model the uncertainties [10], [14] This is not the case in real applications because of the nondeterministic features of environments especially in disaster areas [15], [16]. A catastrophic failure of systems, i.e. some faulty robots may be out-of-order permanently, causes irretrievable decrease in system performance [13] It is a major problem for real-time MRS applications that to detect such failure cases and to adapt robots’ decision-making and acting mechanisms. The novelty of this paper is that the proposed algorithm provides an adaptive task allocation procedure against dynamic system structure and it ensures a great advance in system performance even in disaster cases by detecting the systemic or environmental failures.

Q-LEARNING THEORY

PROBLEM STATEMENT

APPLICATION AND RESULTS

CONCLUSIONS