Exploring deep learning-based content-based video retrieval with Hierarchical Navigable Small World index and ResNet-50 features for anomaly detection

Abstract

Surveillance video can capture a variety of realistic scenarios for providing security in both private and public spaces. The prediction of anomalies promptly is essential. However, existing methods are time-consuming due to marking conflicting clips in training videos. Therefore, this paper proposes a CBVR (Content-Based Video Retrieval) framework for detecting anomalies in surveillance videos. The proposed method leverages deep learning techniques and the HNSW (Hierarchical Navigable Small World) index to achieve efficient and accurate video retrieval. To preprocess the video data, frames are resized and normalized, and the dataset is augmented with data transformations such as rotations, translations, and flips to improve model generalization. The best results are offered while evaluating with the UCSD Ped2 dataset which produces the MAP score of 94.25%, and recall, precision, and F1-scores of 96.15%, 98.03%, and 92.63% respectively. The results proved that maximizing the number of layers in the HNSW index enhances the MAP score, suggesting that deeper layers are more effective at retrieving relevant results. Overall, the proposed CBVR framework provides an efficient and accurate method for anomaly detection in surveillance videos. It has potential applications in various domains, including video summarization, recommendation, and surveillance, among others.