Detecting humans carrying objects

Abstract

Surveillance cameras may be a tool for solving crimes, but what about using them to prevent or stop criminals or terrorists? Although computerized video-based monitoring would seem to be the obvious answer, algorithms that can recognize suspicious activities and individuals have proven highly difficult to devise. Examples include a terrorist carrying a suicide bomb or a military target holding a heavy weapon. Traditional approaches to this problem are based on markers or feature points extracted from the human body, which is impractical for low-resolution images and moving platforms.1, 2 Using artificial intelligence,3 we are developing a computer monitoring system that can analyze human motion under challenges such as low-resolution and real-time processing. We proceed by examining the cyclic property of motion and present algorithms to classify humans in videos according to their gait patterns. When a person’s limbs are unencumbered, gait movements are symmetrical. Represented graphically, they form a twisted helical pattern called a double-helical signature (DHS) that resembles a figure 8. The pattern is changed by any activity that disturbs the symmetry, such as carrying a package. By defining these signatures, our system can recognize unique characteristics in human gait and automatically detect asymmetries (see Figure 1). In the proposed model, an image is decomposed into ‘X-t’ slices (where ‘t’ is the time axis). The motion of the limbs is represented as a pair of kinematic chains oscillating out of phase. The hands maintain the center of gravity above the point of contact and minimize the energy to balance the body during bipedal leg swing. We expect that the presence of a sufficiently heavy object will (at least in the hand regions) distort the DHS pattern. Because of the compactness of the method, we need only look at one slice to understand the arm articulation. We have studied three activities—natural walking, carrying an Figure 1. (top) Original frames and silhouette in X-Y. (middle) Activity sequence in X-Y-t. (bottom) Selected 2D slices containing DHS in X-t. t: Time axis.