Smoke Recognition Research Articles

A satellite imagery smoke detection framework based on the Mahalanobis distance for early fire identification and positioning

Wildfires negatively affect the atmosphere and ecological environment. The rapid identification of smoke is helpful for early fire detection and positioning, which are significant for fire early warning, fire point tracing, and atmospheric environment monitoring. The purpose of this research is the establishment of a smoke detection framework with which to carry out smoke identification, concentration inversion and the extraction of the smoke concentration center to realize fire source identification and positioning. The spectral characteristics and variation pattern of smoke were first studied and analyzed based on a physical correlation model and laboratory experiments. Moreover, the spectral variation of the vegetation background was measured by the Mahalanobis distance (MD), and MD-based smoke identification and concentration inversion were carried out. Then, the extraction of the smoke concentration center and fire source positioning were realized based on the Laplace operator. Finally, the application and verification of the proposed method were carried out on spaceborne data of forest smoke in Daxing’anling, China, and British Columbia, Canada. The results show that: (1) At the significance level α = 0.1%, the overall accuracy of smoke recognition based on satellite images was 91.30%, and the Kappa coefficient was 81.69%. (2) The retrieved smoke concentration was in line with the visual interpretation results. (3) The fire point location error was 23.05 ± 4.14 m (less than 2 pixels). The results indicate that the proposed MD-based smoke detection model can effectively realize smoke pixel identification and concentration inversion. The proposed smoke concentration center identification method can accurately locate the fire source and provide positioning services to trace the source of wildfires in forest fire emergencies.

An adaptive frame selection network with enhanced dilated convolution for video smoke recognition

Recognizing smoke from surveillance videos is crucial to achieve robust fire detection due to the rich temporal cues in video data. However, the slow-moving smoke makes it difficult to learn complete motion information of smoke. In addition, the widely-used max/average pooling operations in smoke recognition models may lead to the loss of spatial detail information. To solve the above issues, we propose an adaptive frame selection network (AFSNet) with enhanced dilated convolution for video smoke recognition. Specifically, first, to reduce information redundancy for learning discriminative feature representations, we propose an adaptive frame selection convolution to automatically select the most useful frames in the image sequence. Second, to utilize all the pixels for reducing the loss of detail information and carrying over global information, we propose an enhanced dilated convolution by considering the pixels with high responses as well as the local region to learn the contribution of each pixel to the output automatically. Finally, a feature extraction module without any average and max pooling operations is designed to learn multi-scale, context, and spatiotemporal information simultaneously. Experiments show that our model achieves the highest detection rate of 0.9673 and the lowest false alarm rate of 0.0316 on the SRSet dataset, and achieves the highest F1-score accuracy of 0.85, 0.86, and 0.91 on the three subsets of the RISE dataset, respectively.

Learning Discriminative Feature Representation for Estimating Smoke Density of Smoky Vehicle Rear

Estimating the smoke density from a single image of the smoky vehicle rear is significant for smoke level estimation and smoky vehicle recognition. However, this is a highly ill-posed problem. To solve it, this paper presents a novel smoke density estimation network (SDENet). First, to reduce the susceptibility of deeper convolutions towards the smoke scale variant and enhance feature diversity, we propose the attention multi-scale encoding blocks based on multi-scale blocks and the convolutional block attention. The multi-scale block exploits the multi-scale features at a granular level within a single basic block. The attention makes our model pay more attention to specific regions which are beneficial to smoke density estimation. Second, the extracted features for smoke recognition may contain background interference information due to the smoke translucency, so we propose semantic-guided feature selection blocks to progressively select smoke-relevant features and suppress background interference features from the encoded feature via global high-level semantic information for learning more discriminative features. Finally, to make the learned features more adaptive to smoke feature resolution and visual appearance, we design attention gate decoding blocks to fuse different features via gate blocks, which enhance the features at spatial locations and channel locations where the features are essential for smoke density estimation. Extensive experiments on smoke density estimation show that our model achieves the best performance among existing methods.

Few-Shot Fine-Grained Forest Fire Smoke Recognition Based on Metric Learning.

To date, most existing forest fire smoke detection methods rely on coarse-grained identification, which only distinguishes between smoke and non-smoke. Thus, non-fire smoke and fire smoke are treated the same in these methods, resulting in false alarms within the smoke classes. The fine-grained identification of smoke which can identify differences between non-fire and fire smoke is of great significance for accurate forest fire monitoring; however, it requires a large database. In this paper, for the first time, we combine fine-grained smoke recognition with the few-shot technique using metric learning to identify fire smoke with the limited available database. The experimental comparison and analysis show that the new method developed has good performance in the structure of the feature extraction network and the training method, with an accuracy of 93.75% for fire smoke identification.

Online detection and source tracing of crop straw burning

The carbon dioxide, sulfur dioxide, and metal ions produced by straw burning can severely pollute the atmosphere; thus, online detection and traceability for straw burning is very important. However, to our best knowledge, there is no comprehensive system that can satisfy online detection, classification, and traceability due to the challenging online detection and traceability of straw burning. In this paper, a new system based on laser-induced breakdown spectroscopy (LIBS) and machine learning is developed, and this developed system is employed for the first time in online detection and traceability of straw combustion. Four different types of straw are selected and the straw burning smoke is monitored online using this developed system. The analysis of straw smoke spectra shows that there are Fe, Mn, and Ba heavy metal spectra in the smoke spectra. By comparing the smoke spectra of different types of straw, the characteristic spectral lines with large differences are selected and dimensionality reduction is performed by linear discriminant analysis algorithm. Then, combined with random forest to achieve classification, the final smoke recognition accuracy reaches 87.0%. Straw ash is then used as a reference analysis and the same operation is performed on it. Mn, Ba, and Li heavy metal spectral lines are found in the spectra of ash, and the final recognition accuracy is 92.6%. The innovative and developed system based on LIBS and machine learning is fast, online, and in situ and has far-reaching application prospects in the environment.

Flame and smoke recognition on smart edge using deep learning

Flame and smoke recognition on smart edge using deep learning

3DVSD: An end-to-end 3D convolutional object detection network for video smoke detection

In addition to static features, dynamic features are also important for smoke recognition. 3D convolution can extract temporal and spatial information from video sequences. Currently, for video smoke detection, 3D convolution is usually used as a tool for secondary judgment of the detection results of single frame approaches. In this work, an end-to-end object detection neural network based on 3D convolution for video smoke detection, named 3DVSD, is proposed for the first time. The network captures moving objects from the input video sequences by the dynamic feature extraction part first and then inputs the feature tensor to the static feature extraction part for recognition and localization, which makes full use of the spatiotemporal features of smoke and improves the reliability of the algorithm. In addition, a time-series smoke video dataset for network training is proposed. The proposed algorithm is compared with other related studies. The experimental results demonstrated that the 3DVSD is promising with an accuracy rate of 99.54%, a false alarm rate of 1.11%, and a missed detection rate of 0.14%, and meets the requirements of real-time detection.

CENet: A Channel-Enhanced Spatiotemporal Network With Sufficient Supervision Information for Recognizing Industrial Smoke Emissions

Vision-based industrial smoke emission recognition technology can identify smoke emissions and provide a visual evidence for humans to pursue environmental justice. However, the existing methods still face the issues of low detection rates (DRs) and high false alarm rates (FARs) due to the insufficient supervision information and limited feature representation capability. To solve these issues, this article presents a channel-enhanced spatiotemporal network (CENet) with sufficient supervision information for recognizing industrial smoke emissions. First, to provide sufficient supervision information for learning discriminative feature representation, we propose a new loss function by collaboratively taking the binary category, pixel-level smoke density, and background information as supervision information and use them in the network final layer as well as the network middle layers to guide model training. Second, to solve the deficiencies of max/average pooling and convolution operations in feature extraction, we propose channel-enhanced modules, including channel-enhanced pooling (CEPool), channel-enhanced convolution (CEConv), and channel-enhanced upsampling (CEUpsample) to learn high-response values of bright features as well as the low-response values with discriminative features. The channel-enhanced modules selectively enhance the learned features with large amount of information and suppress those useless features. Third, we propose a two-stage method based on spatiotemporal information extraction module (SIEM) and smoke recognition module (SRM), which are designed to learn spatiotemporal information between input frames and that between smoke density frames, respectively. Extensive experiments show that CENet achieves the best performance among the existing smoke recognition methods.

Lightweight Smoke Recognition Based on Deep Convolution and Self-Attention

Deep convolutional networks have better smoke recognition performance. However, a lightweight network model and high recognition accuracy cannot be balanced when deployed on hardware with limited computing resources such as edge computing. Based on this background, we propose a novel smoke recognition network that combines convolutional networks (CNN) and self-attention. The core ideas of this framework are as follows: (1) Combine the depthwise convolution and asymmetric convolution of large convolution kernels to construct a lightweight CNN model, and realize multiscale extraction of feature information with slight model complexity. (2) Combined with the self-attention in transformer, a skip-connection branch is designed, which improves the feature extraction capability of the backbone network through parallel processing and fusion of feature map information. (3) Fusion multicomponent discrete cosine transform (DCT) is used to compress channel information and expand the ability of global average pooling (GAP) to aggregate feature maps. The proposed DCT-GAP improves the accuracy of the network without adding additional computational costs. Experimental results show that the proposed CSANet achieves an average accuracy of over 98.3% with 238 M FLOPs and 5.8 M parameters on the homemade smoke dataset, outperforming state-of-the-art competitors.

Situations of Passive Smoking by Urine Cotinine Levels and Recognition of Passive Smoking in High School Students -Effectiveness of Smoking Prevention Education Using a Video Made by High School Students

We investigated a situation of passive smoking and its damaging effects among high school students. Urine cotinine concentration was measured and quantified. Additionally, we evaluated the awareness of passive smoking and smoking regulations in high school students, and the educational effect on passive smoking using a questionnaire survey and educational videos produced by high school students. We conducted a self-reporting questionnaire survey with high school students before and after watching the video produced by the high school students. We gathered the scores of the Kano Social Nicotine Dependence Questionnaire (KTSND) and awareness of smoking restrictions. Consent was obtained through the questionnaire before watching the video and collecting urine samples. Urine cotinine concentrations from 54 samples were evaluated and indicated within the low value. The KTSND score significantly decreased for those who responded to both questionnaires, after watching the video. Furthermore, analysis of the KTSND questionnaire items showed a significant decrease in scores for lifestyle, stress, and smoking location. This suggests that the video produced in this study has a certain amount of educational effect on passive smoking and that the student-led educational method is effective. The survey using the KTSND revealed that there were some students who were not exposed to passive smoking, but instead had high smoking tolerance. Going forward, it will be necessary to promote education on passive smoking and smoking prevention by incorporating the video lecture and urine cotinine concentration was measured, as in this study, to encourage behavior that decreases passive smoking among high school students.

Acoustic Wave Based Forest Fire Extinguisher and Detection using Deep Learning

Apart from causing tragic loss of lives and valuable natural and individual properties including thousands of hectares of forest and hundreds of houses, forest fires are a great menace to ecologically healthy grown forests and protection of the environment. Every year, thousands of forest fire across the globe cause disasters beyond measure and description. This issue has been the research interest for many years; there are a huge amount of very well studied solutions available out there for testing or even ready for use to resolve this problem. Forest and urban fires have been and still are serious problem for many countries in the world. Currently, there are many different solutions to detect the forest fires. People are using sensors to detect the fire. But this case is not possible for large acres of forest. In this paper, we discuss a new approach for fire detection, in which modern technologies are used. In particular, we propose a platform that Artificial Intelligence. The computer vision methods for recognition and detection of smoke and fire, based on the still images or the video input from the cameras. Deep learning method “convolution neural network “for finding the amount of smoke and fire. The accuracy is based on the algorithm which we are going to use and the datasets and splitting them into train set and test set.

Convolution Optimization in Fire Classification

Early alert fire and smoke detection systems are crucial for daily and security management decision-making. Recent literature approaches are based on Deep Learning (DL) models. Efficient models are required for hardware-constrained systems, such as mobile devices, embedded systems, and robotics achieving high performance at low power consumption. For this research, we designed a novel specific-purpose model for fire and smoke recognition using still images and the study of state-of-the-art convolution techniques to improve the trade-off between accuracy and complexity. In this regard, the literature suggests that the inverted residual block, the depthwise and octave convolution techniques, reduces the model’s size and computation requirements working well by themselves. In this work, we propose the KutralNext architecture, an efficient model for single- and multi-label fire and smoke recognition tasks. Additionally, a more efficient architecture KutralNext+, demonstrates that those convolution techniques achieve a better trade-off combined, reaching an 84.36% average test accuracy in FireNet, FiSmo, and FiSmoA fire datasets. The KutralSmoke and FiSmo fire and smoke datasets attained an 81.53% average test accuracy. Furthermore, a previous fire and smoke recognition model considered, FireDetection, KutralNext uses 59% fewer parameters, and KutralNext+ requires 97% fewer flops and is 4x faster.

Video-based Fire Smoke Detection Using Temporal-spatial Saliency Features

In this paper, we propose a video based spatial-temporal convolutional neural network for fire smoke recognition. The model concatenates the appearance features and the motion features followed by a convolution layer to implement spatial-temporal feature fusion. To reduce the influence of background of no-smoke, we use an attention module to capture salience features from the input image. Experiments on the self-created dataset show that the presented method is valid, which achieves a detection rate of 97.5% and accuracy rate of 96.8%.

PDAM–STPNNet: A Small Target Detection Approach for Wildland Fire Smoke through Remote Sensing Images

The target detection of smoke through remote sensing images obtained by means of unmanned aerial vehicles (UAVs) can be effective for monitoring early forest fires. However, smoke targets in UAV images are often small and difficult to detect accurately. In this paper, we use YOLOX-L as a baseline and propose a forest smoke detection network based on the parallel spatial domain attention mechanism and a small-scale transformer feature pyramid network (PDAM–STPNNet). First, to enhance the proportion of small forest fire smoke targets in the dataset, we use component stitching data enhancement to generate small forest fire smoke target images in a scaled collage. Then, to fully extract the texture features of smoke, we propose a parallel spatial domain attention mechanism (PDAM) to consider the local and global textures of smoke with symmetry. Finally, we propose a small-scale transformer feature pyramid network (STPN), which uses the transformer encoder to replace all CSP_2 blocks in turn on top of YOLOX-L’s FPN, effectively improving the model’s ability to extract small-target smoke. We validated the effectiveness of our model with recourse to a home-made dataset, the Wildfire Observers and Smoke Recognition Homepage, and the Bowfire dataset. The experiments show that our method has a better detection capability than previous methods.

Evaluation of Deep Learning Models for Smoking Recognition with Smartwatch and Smartphone Sensors

Smartwatches and smartphones are extensively used in human activity recognition, particularly for step counting and daily sports applications, thanks to the motion sensors integrated into these devices. Machine learning algorithms are often utilized to process sensor data and classify the activities. There are many studies that explore the use of traditional classification algorithms in activity recognition, however, recently, deep learning approaches are also receiving attention. In this paper, we use a dataset that particularly consists of smoking-related activities and explores the recognition performance of three deep learning architectures, namely Long-Short Term Memory (LSTM)}, Recurrent Neural Networks (RNN) and Convolutional Neural Networks (CNN). We evaluate their performances according to different hyperparameters, different sensor types and device types. The results show that the performance of LSTM is much higher than that of CNN and RNN. Moreover, the use of magnetometer and gyroscope together with accelerometer data improves the performance. Use of data from smartphone sensors also enhances the performance results and the final accuracy with the best parameter combinations is observed to be 98%.

Co-occurrence balanced time series classification for the semi-supervised recognition of surgical smoke.

Automatic recognition and removal of smoke in surgical procedures can reduce risks to the patient by supporting the surgeon. Surgical smoke changes its visibility over time, impacting the vision depending on its amount and the volume of the body cavity. While modern deep learning algorithms for computer vision require large amounts of data, annotations for training are scarce. This paper investigates the use of unlabeled training data with a modern time-based deep learning algorithm. We propose to improve the state of the art in smoke recognition by enhancing a image classifier based on convolutional neural networks with a recurrent architecture thereby providing temporal context to the algorithm. We enrich the training with unlabeled recordings from similar procedures. The influence of surgical tools on the smoke recognition task is studied to reduce a possible bias. The evaluations show that smoke recognition benefits from the additional temporal information during training. The use of unlabeled data from the same domain in a semi-supervised training procedure shows additional improvements reaching an accuracy of 86.8%. The proposed balancing policy is shown to have a positive impact on learning the discrimination of co-occurring surgical tools. This study presents, to the best of our knowledge, the first use of a time series algorithm for the recognition of surgical smoke and the first use of this algorithm in the described semi-supervised setting. We show that the performance improvements with unlabeled data can be enhanced by integrating temporal context. We also show that adaption of the data distribution is beneficial to avoid learning biases.

Project RISE: Recognizing Industrial Smoke Emissions

Industrial smoke emissions pose a significant concern to human health. Prior works have shown that using Computer Vision (CV) techniques to identify smoke as visual evidence can influence the attitude of regulators and empower citizens to pursue environmental justice. However, existing datasets are not of sufficient quality nor quantity to train the robust CV models needed to support air quality advocacy. We introduce RISE, the first large-scale video dataset for Recognizing Industrial Smoke Emissions. We adopted a citizen science approach to collaborate with local community members to annotate whether a video clip has smoke emissions. Our dataset contains 12,567 clips from 19 distinct views from cameras that monitored three industrial facilities. These daytime clips span 30 days over two years, including all four seasons. We ran experiments using deep neural networks to establish a strong performance baseline and reveal smoke recognition challenges. Our survey study discussed community feedback, and our data analysis displayed opportunities for integrating citizen scientists and crowd workers into the application of Artificial Intelligence for Social Impact.

Video smoke detection with domain knowledge and transfer learning from deep convolutional neural networks

Video smoke detection (VSD) is a prospective and effective solution for fire detection in spacious buildings and forests. Most of the deep learning based VSD model are end-to-end models, and the intrinsical knowledge of the smoke, such as the motion and color which are obvious and effective for detection have not been utilized effectively. In order to improve the detection rate and reduce the false positive rate of the VSD systems, domain knowledge of smoke was used to segment suspected smoke regions in a video frame firstly in this detection framework. Then a deep neural network was designed to extract the features of smoke regions and distinguish smoke regions from all suspected region. For the intrinsical source domain and target domain, source task and target task, transfer learning of Alex-net, Inception V3 and ResNet pre-trained had been used to distinguish smoke regions in this work. A dataset composed of smoke images and other common objects was established to train the smoke recognition model. The F-beta value can be as high as 0.99. Experiments show that domain knowledge is important for smoke detection in the preliminary stage. Meanwhile, instance-transfer learning and parameter-transfer learning can be potentially effective solutions for VSD.

Joint Network Smoke Recognition based on Channel Attention Mechanism

Aiming at the problems that traditional fire smoke recognition methods in a low recognition accuracy, a fusion network based on VGG16 is proposed, which use channel attention mechanism and contain Dense Blocks network to extract smoke features. To avoid the loss of smoke features, channel attention mechanism in backbone network is automatically to learn the importance of feature in this network. The experiment results show that the accuracy of this network is 3.0% higher than VGG16 neural network, and which is effective and feasible in smoke recognition tasks.

Visualization study on the effect of ambient wind on smoke layer height in chamber fires under natural smoke exhaust condition

Ambient wind often affects the smoke movement and natural smoke exhaust performance in enclosure fires. In this study, six wind speeds and twelve kinds of vent combinations are designed. 72 groups of experiments are conducted to investigate the coupling effects of vent combination and ambient wind speed during quasi-steady state of natural smoke extraction in chamber fires. Compared to the traditional Particle Image Velocimetry (PIV) method, a novel particle image flow field visualization (PIFFV) technique is proposed, and a smoke recognition algorithm for the determination of smoke layer height is developed based on two-dimensional OTSU method. Experimental results show that using the PIFFV method and the proposed algorithm, smoke layer height can be determined more accurately, in comparison to the N-percentage method. When the height of outlet is higher than that of inlet, the worst smoke control effect is found. When the air inlet is located on the windward side, the effect of natural smoke extraction is better under a reasonable wind speed than that in a windless condition. However, when the height of outlet is lower than that of the windward inlet, the optimal smoke control effect is achieved without ambient wind. In addition, when the inlet is arranged on the sideward side, ambient wind has a limited influence on smoke layer height with a lower outlet. Results of this study can provide references for the design of smoke control and ventilation system in room fires.