Fire and Gun Detection Based on Sematic Embeddings

Abstract

It is critical that real-time gun and fire detection from video be accurate to protect life, property and the environment. Recent advances in deep machine learning have greatly improved detection accuracy in this domain. In this paper, a semantic embedding-based method is developed for zero-shot gun and fire detection. Using a pre-trained Contrastive Language-Image Pre-Training (CLIP) model, input images and arbitrary texts can be mapped to semantic vectors and their similarity can be computed. By defining object classes using the semantic vector of each classes’ description, highly accurate object detection accuracy can be achieved without training any new model. Evaluation of this method on public domain FireNet and IMFDB datasets demonstrates fire and gun detection accuracy of 99.8% and 97.3%, respectively, which significantly outperforms state of the art FireNet and you look only once (YOLO) algorithms. Semantic embedding enables open set semantic search in video and simplifies deploying and maintaining object detection applications.