Smoke detection in video using convolutional neural networks and efficient spatio-temporal features

Abstract

Fire detection in its early stages is of a great importance in different environmental related applications. Among the visual signs of fire, smoke appears earlier than the flames in many cases, and quickly reaches the environment. Thus, it can be used for early detection of fire using machine vision techniques. Existing approaches have tried to do it either by traditional machine learning methods applying various combinations of color, texture, and motion features, or by deep learning-based methods that can automatically capture the smoke features from images. However, the former approaches face number of challenges due to the transparent nature of smoke and its variant visual appearance in different environments, and the later ones are not able to capture motion characteristics of smoke, so, their efficiency in various environmental conditions is still problematic and often cause false alarms. In this study, we aim to introduce a hybrid approach that is based on deep learning, spatio, and spatio-temporal characteristics of the smoke. Doing so, we use all the strengths of these techniques to detect the smoke as accurately as possible. The proposed method consists of four stages: 1) moving pixels are extracted from input images by an efficient motion detection scheme; 2) the extracted moving areas are given individually to a tailored convolutional neural network to identify candidate smoke regions; 3) an efficient combination of spatial and spatio-temporal features is extracted from each candidate region based on the distinct characteristics of smoke; 4) a support vector machine classifier is used to further classify real smoke from non-smoke regions using the extracted features. The proposed method is implemented using Python programming language, and extensive experiments conducted on “Visor”, “Bilkent”, and “Yuan” benchmark datasets which show it has high performance and accuracy. The results also indicate that its reliability is far better than the competitors in terms of false alarm rate. The average accuracy and false positive rates obtained on 10 testing videos are 97.63% and 3.8%, respectively. Also, the proposed method is able to detect both white and black smokes which, to our best knowledge, has not been addressed in any of the related researches.