An occlusion‐robust tracking algorithm based on a spatiotemporal Markov random field model

Abstract

AbstractThe tracking of vehicles on images is important as a baseline technology for applying ITS image processing to the detection of abnormal phenomena such as accidents. However, up to now one of the most difficult problems in vehicle tracking has been occlusion, which creates a difficult environment for realizing stable tracking. This has been particularly true in our research, because our objects of study are large intersections in which an average of 20 vehicles are present at the same time, of various sizes and shapes, and so on. The various motions of these vehicles also generate various conditions under which occlusion can occur. In tracking under these circumstances, it is necessary to use a paradigm different from those used previously, which assume a model of rectilinear motion, largely vacant intersections, predictions based on straight‐line trajectories, and specific vehicle shapes. To solve this problem, we developed an algorithm that makes use of the Markov random field (MRF) model, generalized beyond a single image to include temporal sequences of images, and evaluated its degree of accuracy based on spatiotemporal correlation of textures from one image to the one immediately following it and the connection of loci followed by the moving bodies. The results were further optimized by stochastic relaxation of the energy distributions exhibited by images from this space‐time MRF, which makes the tracking robust against various forms of occlusion. We applied this space‐time MRF to image data taken over about 25 minutes, tracking 3214 vehicles moving through an intersection under conditions of heavy congestion. Our results showed that the tracking was successful with a 99% probability when the vehicles being tracked generated no occlusions, while for the 541 cars that did cause occlusions the probability of successfully tracking multiple cars and distinguishing the separate vehicles through the occlusions was 95%. Because this algorithm uses information obtained from black‐and‐white images only, and can be implemented without any prior hypotheses about the vehicle's shape etc., it is useful over long periods of time and under a broad range of conditions. For this reason, we anticipate that it will be effective for detecting accidents and other anomalies in traffic at intersections. © 2003 Wiley Periodicals, Inc. Electron Comm Jpn Pt 3, 86(9): 73–86, 2003; Published online in Wiley InterScience (www. interscience.wiley.com). DOI 10.1002/ecjc.10110