Real Video Sequences Research Articles

Directional People Counter Based on Head Tracking

This paper presents an application for counting people through a single fixed camera. This system performs the count distinction between input and output of people moving through the supervised area. The counter requires two steps: detection and tracking. The detection is based on finding people's heads through preprocessed image correlation with several circular patterns. Tracking is made through the application of a Kalman filter to determine the trajectory of the candidates. Finally, the system updates the counters based on the direction of the trajectories. Different tests using a set of real video sequences taken from different indoor areas give results ranging between 87% and 98% accuracies depending on the volume of flow of people crossing the counting zone. Problematic situations, such as occlusions, people grouped in different ways, scene luminance changes, etc., were used to validate the performance of the system.

Stopped object detection by learning foreground model in videos.

The automatic detection of objects that are abandoned or removed in a video scene is an interesting area of computer vision, with key applications in video surveillance. Forgotten or stolen luggage in train and airport stations and irregularly parked vehicles are examples that concern significant issues, such as the fight against terrorism and crime, and public safety. Both issues involve the basic task of detecting static regions in the scene. We address this problem by introducing a model-based framework to segment static foreground objects against moving foreground objects in single view sequences taken from stationary cameras. An image sequence model, obtained by learning in a self-organizing neural network image sequence variations, seen as trajectories of pixels in time, is adopted within the model-based framework. Experimental results on real video sequences and comparisons with existing approaches show the accuracy of the proposed stopped object detection approach.

The Performance of MANET Routing Protocols for Scalable Video Communication

Mobile Ad-hoc Networks (MANETs) operate without infrastructure where nodes can move randomly. Therefore, routing in MANETs is a challenging task. In this paper we evaluate the performance of three important MANET routing protocols: Ad hoc On-Demand Distance Vector (AODV), Dynamic Source Routing (DSR) and Optimized Link State Routing (OLSR) when employed to forward scalable video contents. AODV and DSR are reactive protocols in that routing paths are established once needed. On the other hand, OLSR is a proactive routing protocol where routing information is exchanged and maintained continuously. The goal of the performance evaluation in this study is to assess the performance of AODV, DSR and OLSR in communicating scalable video contents. In the simulation part of this paper, a real video sequence is communicated where the characteristics and quality of the video decoded at receiver nodes are evaluated. NS2 along with extensions and other evaluation frameworks have been used to assess the performance of the MANET routing protocols when used for scalable video communication. The framework allows starting from a raw video that is encoded, packetized, transmitted through a network topology and collected at receiver to be decoded, played, and evaluated. Delay and timing constraints are taken into consideration when decoding the received video packets.

Tracking Method in Consideration of Existence of Similar Object around Target Object

Tracking methods based on the particle filter uses frequently the appearance information of the target object to calculate the likelihood. The method using it often fails in tracking when the target object intersects with other objects with similar appearances. We propose a new approach for tracking objects with similar patterns in a video sequence taken by a moving camera. The proposed method based on the particle filter is robust to the intersection with other objects. Two state transition functions are defined for robust tracking. The method changes the function depending on the situation. In addition, the likelihood is calculated by using four factors which are the information of the color, the velocity, the distance between the objects and the values calculated by the probability background model. The method detects objects which are similar to the target object and which exist around the target object. This prevents the method from tracking other object mistakenly. Results are demonstrated by experiments using real video sequences.

Bernoulli filter for joint detection and tracking of an extended object in clutter

The problem is joint detection and tracking of a non-point or extended moving object, characterised by multiple feature points, which can result in detections. Owing to imperfect detection, only some of the feature points are detected and in addition, false alarms [or clutter] can also be present. Standard tracking techniques assume point objects, that is at most one detection per object, and hence are not adequate for this problem. This study presents a principled theoretical solution in the form of the Bayes filter, referred to as the Bernoulli filter for an extended object. The derivation follows the random set filtering framework introduced by Mahler. The filter is implemented approximately as a particle filter and subsequently applied both to simulated data and a real video sequence.

Image and Video Restorations via Nonlocal Kernel Regression

A nonlocal kernel regression (NL-KR) model is presented in this paper for various image and video restoration tasks. The proposed method exploits both the nonlocal self-similarity and local structural regularity properties in natural images. The nonlocal self-similarity is based on the observation that image patches tend to repeat themselves in natural images and videos, and the local structural regularity observes that image patches have regular structures where accurate estimation of pixel values via regression is possible. By unifying both properties explicitly, the proposed NL-KR framework is more robust in image estimation, and the algorithm is applicable to various image and video restoration tasks. In this paper, we apply the proposed model to image and video denoising, deblurring, and superresolution reconstruction. Extensive experimental results on both single images and realistic video sequences demonstrate that the proposed framework performs favorably with previous works both qualitatively and quantitatively.

Optimized pre-processing input plane GPU implementation of an optical face recognition technique using a segmented phase only composite filter

The key outcome of this work is to propose and validate a fast and robust correlation scheme for face recognition applications. The robustness of this fast correlator is ensured by an adapted pre-processing step for the target image allowing us to minimize the impact of its (possibly noisy and varying) amplitude spectrum information. A segmented composite filter is optimized, at the very outset of its fabrication, by weighting each reference with a specific coefficient which is proportional to the occurrence probability. A hierarchical classification procedure (called two-level decision tree learning approach) is also used in order to speed up the recognition procedure. Experimental results validating our approach are obtained with a prototype based on GPU implementation of the all-numerical correlator using the NVIDIA GPU GeForce 8400GS processor and test samples from the Pointing Head Pose Image Database (PHPID), e.g. true recognition rates larger than 85% with a run time lower than 120ms have been obtained using fixed images from the PHPID, true recognition rates larger than 77% using a real video sequence with 2 frame per second when the database contains 100 persons. Besides, it has been shown experimentally that the use of more recent GPU processor like NVIDIA-GPU Quadro FX 770M can perform the recognition of 4 frame per second with the same length of database.

A fast image registration approach based on SIFT key-points applied to super-resolution

An accurate image registration is a fundamental stage in many image processing problems. In this paper, a new and fast registration approach based on scale invariant feature transform (SIFT) key-points, under Euclidean transformation model, is proposed. The core idea of the proposed method is estimation of rotation angle and vertical and horizontal shifts using averaging of differences of SIFT key-point pairs locations. The method is simple but requires some tuning modules for accurate estimation. Orientation modification and compensation and shift compensation are some of the proposed modules. The proposed method is fast, about ive times faster than RANSAC method for model parameters estimation. The accuracy of the proposed method is compared with some popular registration methods. Various comparisons have been done with LIVE database images with known motion vectors. The experimental results over two real video sequences show the high performance of the proposed algorithm in a super-resolution application.

Fuzzy spatial constraints and ranked partitioned sampling approach for multiple object tracking

While particle filters are now widely used for object tracking in videos, the case of multiple object tracking still raises a number of issues. Among them, a first, and very important, problem concerns the exponential increase of the number of particles with the number of objects to be tracked, that can make some practical applications intractable. To achieve good tracking performances, we propose to use a Partitioned Sampling method in the estimation process with an additional feature about the ordering sequence in which the objects are processed. We call it Ranked Partitioned Sampling, where the optimal order in which objects should be processed and tracked is estimated jointly with the object state. Another essential point concerns the modeling of possible interactions between objects. As another contribution, we propose to represent these interactions within a formal framework relying on fuzzy sets theory. This allows us to easily model spatial constraints between objects, in a general and formal way. The association of these two contributions was tested on typical videos exhibiting difficult situations such as partial or total occlusions, and appearance or disappearance of objects. We show the benefit of using conjointly these two contributions, in comparison to classical approaches, through multiple object tracking and articulated object tracking experiments on real video sequences. The results show that our approach provides less tracking errors than those obtained with the classical Partitioned Sampling method, without the need for increasing the number of particles.

Trajectory clustering in CCTV traffic videos using probability product kernels with hidden Markov models

This article presents a method for clustering the trajectories obtained by tracking vehicles in traffic videos, recorded from CCTV cameras in public spaces. The proposed method employs a model-based approach in which (1) each trajectory (position and velocity) is modelled using a hidden Markov model (HMM), and (2) the distance between two trajectories is computed as the probabilistic similarity between HMMs, by means of the probability product kernel. Experiments on a set of real traffic video sequences reveal very good results of the proposed approach, outperforming two non-trivial baselines. To the best of the authors' knowledge, this approach is novel for trajectory grouping in traffic videos.

Robust multiple-vehicle tracking via adaptive integration of multiple visual features

Abstract This article presents a robust approach to tracking multiple vehicles with integration of multiple visual features. The observation is modeled by democratic integration strategies according to the reliability of the information in the current multi-visual features to adjust their weights. The appearance model is also embedded in a particle filter (PF) tracking framework. Furthermore, we propose a new model updating algorithm based on the PF. In order to avoid incorrect results caused by "model drift" introduced into the observation model, model updating should only be controlled in a reliable manner, and the rate of updating is based on reliability. This article also presents the experiments using a real video sequence to verify the proposed method.

Scalable Video Multicast With Adaptive Modulation and Coding in Broadband Wireless Data Systems

Future mobile broadband networks are characterized with high data rate and improved coverage, which will enable real-time video multicast and broadcast services. Scalable video coding (SVC), combined with adaptive modulation and coding schemes (MCS) and wireless multicast, provides an excellent solution for streaming video to heterogeneous wireless devices. By choosing different MCSs for different video layers, SVC can provide good video quality to users in good channel conditions while maintaining basic video quality for users in bad channel conditions. A key issue to apply SVC to wireless multicast streaming is to choose appropriate MCS for each video layer and to determine the optimal resource allocation among multiple video sessions. We formulate this problem as total utility maximization, subject to the constraint of available radio resources. We prove that the formulated problem is NP-hard and propose an optimal, two-step dynamic programming solution with pseudo-polynomial time complexity. Simulation results show that our algorithm offers significant improvement on the video quality over a naive algorithm and an adapted greedy algorithm, especially in the scenarios with multiple real video sequences and limited radio resources.

Content-Based 3-D Mosaics for Representing Videos of Dynamic Urban Scenes

We propose a content-based 3-D mosaic (CB3M) representation for long video sequences of 3-D and dynamic urban scenes captured by a camera on a mobile platform. In the first phase, a set of parallel-perspective (pushbroom) mosaics with varying viewing directions is generated to capture both the 3-D and dynamic aspects of the scene under the camera coverage. In the second phase, a segmentation-based stereo matching algorithm is applied to extract parametric representations of the color, structure and motion of the dynamic and/or 3-D objects in urban scenes, where a lot of planar surfaces exist. Multiple pairs of stereo mosaics are used for facilitating reliable stereo matching, occlusion handling, accurate 3-D reconstruction, and robust moving target detection. CB3M is a highly compressed visual representation for a dynamic 3-D scene, and has object contents of both 3-D and motion information. Experimental results are given for various real video sequences of large-scale 3-D scenes.

3D video visualization employing wavelet multilevel decomposition

This study analyzed the implementation and performance of a framework that can be efficiently applied to three-dimensional (3D) video sequence visualization. The proposed algorithm is based on wavelets and wavelet atomic functions used in the computation of disparity maps. The proposed algorithm employs wavelet multilevel decomposition and 3D visualization via color anaglyphs synthesis. Simulations were run on synthetic images, synthetic video sequences, and real-life video sequences. Results shows that this novel approach performs better in depth and spatial perception tasks compared to existing methods, both in terms of objective criteria such as quantity of bad disparities and similarity structural index measure and the more subjective measure of human vision.

Security System and Surveillance Using Real Time Object Tracking and Multiple Cameras

In this paper we propose multiple cameras using real time tracking for surveillance and security system. It is extensively used in the research field of computer vision applications, like that video surveillance, authentication systems, robotics, pre-stage of MPEG4 image compression and user inter faces by gestures. The key components of tracking for surveillance system are extracting the feature, background subtraction and identification of extracted object. Video surveillance, object detection and tracking have drawn a successful increased interest in recent years. A object tracking can be understood as the problem of finding the path (i.e. trajectory) and it can be defined as a procedure to identify the different positions of the object in each frame of a video. Based on the previous work on single detection using single stationary camera, we extend the concept to enable the tracking of multiple object detection under multiple camera and also maintain a security based system by multiple camera to track person in indoor environment, to identify by my proposal system which consist of multiple camera to monitor a person. Present study mainly aims to provide security and detect the moving object in real time video sequences and live video streaming. Based on a robust algorithm for human body detection and tracking in videos created with support of multiple cameras.

A Joint Approach to Global Motion Estimation and Motion Segmentation From a Coarsely Sampled Motion Vector Field

In many content-based video processing systems, the presence of moving objects limits the accuracy of global motion estimation (GME). On the other hand, the inaccuracy of global motion parameter estimates affects the performance of motion segmentation. In this paper, we introduce a procedure for simultaneous object segmentation and GME from a coarsely sampled (i.e., block-based) motion vector (MV) field. The procedure starts with removing MV outliers from the MV field, and then performs GME to obtain an estimate of global motion parameters. Using these estimates, global motion is removed from the MV field, and moving region segmentation is performed on this compensated MV field. MVs in the moving regions are treated as outliers in the context of GME in the next round of processing. Iterating between GME and motion segmentation helps improve both GME and segmentation accuracy. Experimental results demonstrate the advantage of the proposed approach over state-of-the-art methods on both synthetic motion fields and MVs from real video sequences.

Visual SLAM Based on Rigid-Body 3D Landmarks

In current visual SLAM methods, point-like landmarks (As in Filliat and Meyer (Cogn Syst Res 4(4):243---282, 2003), we use this expression to denote a landmark generated by a point or an object considered as punctual.) are used for representation on maps. As the observation of each point-like landmark gives only angular information about a bearing camera, a covariance matrix between point-like landmarks must be estimated in order to converge with a global scale estimation. However, as the computational complexity of covariance matrices scales in a quadratic way with the number of landmarks, the maximum number of landmarks that is possible to use is normally limited to a few hundred. In this paper, a visual SLAM system based on the use of what are called rigid-body 3D landmarks is proposed. A rigid-body 3D landmark represents the 6D pose of a rigid body in space (position and orientation), and its observation gives full-pose information about a bearing camera. Each rigid-body 3D landmark is created from a set of N point-like landmarks by collapsing 3N state components into seven state components plus a set of parameters that describe the shape of the landmark. Rigid-body 3D landmarks are represented and estimated using so-called point-quaternions, which are introduced here. By using rigid-body 3D landmarks, the computational time of an EKF-SLAM system can be reduced up to 5.5%, as the number of landmarks increases. The proposed visual SLAM system is validated in simulated and real video sequences (outdoor). The proposed methodology can be extended to any SLAM system based on the use of point-like landmarks, including those generated by laser measurement.

Optimal Metric Projections for Deformable and Articulated Structure-from-Motion

This paper describes novel algorithms for recovering the 3D shape and motion of deformable and articulated objects purely from uncalibrated 2D image measurements using a factorisation approach. Most approaches to deformable and articulated structure from motion require to upgrade an initial affine solution to Euclidean space by imposing metric constraints on the motion matrix. While in the case of rigid structure the metric upgrade step is simple since the constraints can be formulated as linear, deformability in the shape introduces non-linearities. In this paper we propose an alternating bilinear approach to solve for non-rigid 3D shape and motion, associated with a globally optimal projection step of the motion matrices onto the manifold of metric constraints. Our novel optimal projection step combines into a single optimisation the computation of the orthographic projection matrix and the configuration weights that give the closest motion matrix that satisfies the correct block structure with the additional constraint that the projection matrix is guaranteed to have orthonormal rows (i.e. its transpose lies on the Stiefel manifold). This constraint turns out to be non-convex. The key contribution of this work is to introduce an efficient convex relaxation for the non-convex projection step. Efficient in the sense that, for both the cases of deformable and articulated motion, the proposed relaxations turned out to be exact (i.e. tight) in all our numerical experiments. The convex relaxations are semi-definite (SDP) or second-order cone (SOCP) programs which can be readily tackled by popular solvers. An important advantage of these new algorithms is their ability to handle missing data which becomes crucial when dealing with real video sequences with self-occlusions. We show successful results of our algorithms on synthetic and real sequences of both deformable and articulated data. We also show comparative results with state of the art algorithms which reveal that our new methods outperform existing ones.

Bootstrap optical flow confidence and uncertainty measure

We address the problem of estimating the uncertainty of optical flow algorithm results. Our method estimates the error magnitude at all points in the image. It can be used as a confidence measure. It is based on bootstrap resampling, which is a computational statistical inference technique based on repeating the optical flow calculation several times for different randomly chosen subsets of pixel contributions. As few as ten repetitions are enough to obtain useful estimates of geometrical and angular errors. For demonstration, we use the combined local–global optical flow method (CLG) which generalizes both Lucas–Kanade and Horn–Schunck type methods. However, the bootstrap method is very general and can be applied to almost any optical flow algorithm that can be formulated as a pixel-based minimization problem. We show experimentally on synthetic as well as real video sequences with known ground truth that the bootstrap method performs better than all other confidence measures tested.

Integration of Fuzzy Spatial Information in Tracking Based on Particle Filtering

In this paper, we propose a novel method to introduce spatial information in particle filters. This information may be expressed as spatial relations (orientation, distance, etc.), velocity, scaling, or shape information. Spatial information is modeled in a generic fuzzy-set framework. The fuzzy models are then introduced in the particle filter and automatically define transition and prior spatial distributions. We also propose an efficient importance distribution to produce relevant particles, which is dedicated to the proposed fuzzy framework. The fuzzy modeling provides flexibility both in the semantics of information and in the transitions from one instant to another one. This allows one to take into account situations where a tracked object changes its direction in a quite abrupt way and where poor prior information on dynamics is available, as demonstrated on synthetic data. As an illustration, two tests on real video sequences are performed in this paper. The first one concerns a classical tracking problem and shows that our approach efficiently tracks objects with complex and unknown dynamics, outperforming classical filtering techniques while using only a small number of particles. In the second experiment, we show the flexibility of our approach for modeling: Fuzzy shapes are modeled in a generic way and allow the tracking of objects with changing shape.