Effect of Sensor Sensitivity on Autonomous Aerial Vehicle Target Localization

Abstract

In recent years, Unmanned Aerial Systems (UASs) development and application have achieved remarkable growth. With the advancement of technology, UASs nowadays feature more advanced autonomous capabilities than ever before. In order to achieve autonomous behavior, intelligent systems are required to be incorporated to support system learning, control and decision-making. With these capabilities, UASs can learn from their past experiences, through interacting with the task environment to adapt their behavior to enhance their future performance. Machine learning is one of the most commonly used techniques for UASs to acquire knowledge from their experience, and research in this area is still developing. In this study, Reinforcement Learning (RL) algorithms were used on autonomous aerial systems to achieve adaptive behavior and decision-making capabilities. The effects of UAS sensor sensitivity, as modeled through Signal Detection Theory (SDT), on the ability of RL algorithms to accomplish a target localization task were investigated. Three levels of sensor sensitivity were simulated and compared to the results of the same system using a perfect sensor, with the consideration of two RL algorithms, namely, Temporal Difference (TD) and Monte Carlo (MC) methods. Target localization and identification task were used as the test bed, and a hierarchical architecture was developed with two distinct agents. Mission performance was analyzed using multiple metrics, including episodic reward and the time taken to locate all targets. Statistical analyses were carried out to detect significant differences in the comparison of steady-state behavior of different factors. Results were discussed, and future research direction was given at the end of the paper.