Thermal, Multispectral, and RGB Vision Systems Analysis for Victim Detection in SAR Robotics

Abstract

Technological advancements have facilitated the development of sophisticated vision systems, integrating optical sensors with artificial vision and machine learning techniques to create applications in different fields of robotics. One such field is Search and Rescue (SAR) robotics, which has historically played a significant role in assisting brigades following post-disaster events, particularly in exploration phases and, crucially, in victim identification. The importance of employing these systems in victim identification lies in their functionality under challenging conditions, enabling the capture of information across different light spectrum ranges (RGB, Thermal, Multispectral). This article proposes an innovative comparative analysis that scrutinizes the advantages and limitations of three sensor types in victim detection. It explores contemporary developments in the state-of-the-art and proposes new metrics addressing critical aspects, such as functionality in specific scenarios and the analysis of environmental disturbances. For the indoor and outdoor testing phase, a quadrupedal robot has been equipped with these cameras. The primary findings highlight the individual contributions of each sensor, particularly emphasizing the efficacy of the infrared spectrum for the thermal camera and the Near Infrared and Red Edge bands for the multispectral camera. Ultimately, following system evaluations, detection precisions exceeding 92% and 86%, respectively, were achieved.