Forest Fire Segmentation via Temporal Transformer from Aerial Images

Abstract

Forest fires are among the most critical natural tragedies threatening forest lands and resources. The accurate and early detection of forest fires is essential to reduce losses and improve firefighting. Conventional firefighting techniques, based on ground inspection and limited by the field-of-view, lead to insufficient monitoring capabilities for large areas. Recently, due to their excellent flexibility and ability to cover large regions, unmanned aerial vehicles (UAVs) have been used to combat forest fire incidents. An essential step for an autonomous system that monitors fire situations is first to locate the fire in a video. State-of-the-art forest-fire segmentation methods based on vision transformers (ViTs) and convolutional neural networks (CNNs) use a single aerial image. Nevertheless, fire has an inconsistent scale and form, and small fires from long-distance cameras lack salient features, so accurate fire segmentation from a single image has been challenging. In addition, the techniques based on CNNs treat all image pixels equally and overlook global information, limiting their performance, while ViT-based methods suffer from high computational overhead. To address these issues, we proposed a spatiotemporal architecture called FFS-UNet, which exploited temporal information for forest-fire segmentation by combining a transformer into a modified lightweight UNet model. First, we extracted a keyframe and two reference frames using three different encoder paths in parallel to obtain shallow features and perform feature fusion. Then, we used a transformer to perform deep temporal-feature extraction, which enhanced the feature learning of the fire pixels and made the feature extraction more robust. Finally, we combined the shallow features of the keyframe for de-convolution in the decoder path via skip-connections to segment the fire. We evaluated empirical outcomes on the UAV-collected video and Corsican Fire datasets. The proposed FFS-UNet demonstrated enhanced performance with fewer parameters by achieving an F1-score of 95.1% and an IoU of 86.8% on the UAV-collected video, and an F1-score of 91.4% and an IoU of 84.8% on the Corsican Fire dataset, which were higher than previous forest fire techniques. Therefore, the suggested FFS-UNet model effectively resolved fire-monitoring issues with UAVs.