Detection of Abandoned and Stolen Objects Based on Dual Background Model and Mask R-CNN

Hyeseung Park,Youngbok Joo,Seungchul Park

doi:10.1109/access.2020.2990618

Abstract

Dual background models have been widely used for detecting stationary objects in video surveillance systems. However, there is a problem that both abandoned and stolen objects are equally detected as stationary objects, making it difficult to distinguish them. Another problem is the ghost region created by shadow shift or light changes, which makes the discrimination issue more complicated. In this paper, we present an efficient method to distinguish abandoned objects, stolen objects, and ghost regions in the surveillance video. This method contains two main strategies: the first one is the dual background model for extracting candidate stationary objects, the second one is object segmentation based on mask regions with CNN features (Mask R-CNN) for providing the object mask information. The basic idea is: given a candidate stationary object from the background model, it is checked whether a corresponding segmented object exists in the current video frame or the previous background frame to take into account the current and past situations. And the final state of the candidate stationary object is determined by considering various situations through the comparative analysis technique presented in this paper. The proposed algorithm has qualitatively experimented with our own dataset focusing on the discrimination issue, which generated satisfactory results. Therefore, it is expected to be widely applied to automatic detection of stolen objects as well as abandoned objects in open environments such as exhibition halls and public parks where existing intrusion detection-based security services are difficult to be deployed.

Highlights

In recent years, researches on intelligent video surveillance system analyzing video automatically without continuous observation by humans have been actively conducted to provide methods for detecting and notifying the occurrence of specific events such as intrusion, loitering, abandonment, crime, and fire detection
Researches on existing intelligent video surveillance systems that rely on the foreground analysis generated by the background subtraction have a problem that abandoned objects look like stolen objects and ghost regions
We presented a novel algorithm based on traditional image processing techniques and artificial intelligence technology to precisely distinguish abandoned objects, stolen objects, and ghost regions

Summary

Introduction

Researches on intelligent video surveillance system analyzing video automatically without continuous observation by humans have been actively conducted to provide methods for detecting and notifying the occurrence of specific events such as intrusion, loitering, abandonment, crime, and fire detection. Intelligent surveillance systems can reduce human errors by lowering the dependency on humans. They can improve the response time by generating alarms as soon as events occur [1], [2]. Systems based only on traditional computer vision technology have suffered limitations in accuracy or performance, recent advances in artificial intelligence have opened practical ways for improvement in various applications. We propose a novel approach for detecting abandoned and stolen objects based on the conventional background subtraction and an artificial intelligence technology, i.e., Mask R-CNN. The background subtraction technique creates dynamic backgrounds and foregrounds in real-time [3]. The primary way to detect abandoned and stolen objects is to analyze the foreground and to select stationary objects.

Objectives

Methods

Conclusion