Proposed Appearance Model Research Articles

Learning spatial regularized correlation filters with response consistency and distractor repression for UAV tracking

Correlation filter-based trackers have made significant progress in visual object tracking for various types of unmanned aerial vehicle (UAV) applications due to their promising performance and efficiency. However, the boundary effect remains a challenging problem. Several methods enlarge search areas to handle this shortcoming but introduce more background noise, and the filter is prone to learn from distractors. To address this issue, we present spatial regularized correlation filters with response consistency and distractor repression. Specifically, a temporal constraint is introduced to reinforce the consistency across frames by minimizing the difference between consecutive correlation response maps. A dynamic spatial constraint is also integrated by exploiting the local maximum points of the correlation response produced during the detection phase to mitigate the interference from background distractions. The proposed appearance model can optimize the temporal and spatial constraints together with a spatial regularization weight simultaneously. Meanwhile, the proposed appearance model can be solved effectively based on the alternating direction method of multipliers algorithm. The spatial and temporal information concealed in the response maps is fully taken into consideration to boost overall tracking performance. Extensive experiments are conducted on a public UAV benchmark dataset with 123 challenging sequences. The experimental results and analysis demonstrate that the proposed method outperforms 12 state-of-the-art trackers in terms of both accuracy and robustness while efficiently operating in real time.

Video anomaly detection and localization based on appearance and motion models

In this paper, we present an approach to detect and localize anomalies in the surveillance videos. Precise detection, modeling the normality in a context and dealing with false alarms are the major challenges to cope with while performing detection and localization of anomalies. We propose appearance and motion models to detect an anomalous event at the frame level in a video. In the anomalous frames, we localize the anomaly using appearance and connected component analysis cues. The proposed appearance model utilizes the mask-rcnn for scene semantic analysis. The proposed motion model constructs frame descriptors based on the histograms of the intensity difference maps of the consecutive video frames. A one class support vector machine (OCSVM) is used to detect motion based anomalous event. In order to eliminate the false alarms of anomaly detection and localization emerging due to camera jitter and object movements in unlikely motion regions of a video frame, we exploit the connected component analysis. Experiments on the Avenue and UMN datasets reveal that the proposed approach achieves high accuracy and outperforms the existing methods.

Multiple Reliable Structured Patches for Object Tracking

It is essential to build the effective appearance model for object tracking in computer vision. Most object trackers can be roughly divided into two categories according to the appearance model: the bounding box model and the patch model. The bounding box model cannot handle shape deformation and occlusion of the non-rigid moving object effectively. The patch model is prone to be disturbed by complex backgrounds. In this paper, we propose a robust multi-structured-patch appearance model to represent the target for object tracking. The proposed appearance model is aimed to exploit and identify reliable patches that can be tracked effectively through the whole tracking process. According to attention mechanism in biological vision system, a coarse-to-fine strategy is usually used to search the target. Therefore, the proposed appearance model is represented by robust patches in different sizes, in which the bigger patches search the rough region of the target and the smaller patches estimate the accurate location. Experimental results on OTB100 dataset show that the proposed method outperforms state-of-the-art trackers.

Robust Visual Tracking Based on Relaxed Target Representation

Developing an effective target appearance model is a challenging task in visual tracking under the influences of complicated appearance variations. Many tracking algorithms use a linear combination of previous tracking results to represent a target candidate. In existing target representations, all of the feature elements of a target candidate have the same coding vector. With such type of target representations, robust tracking is not satisfactory when drastic appearance variations occur. In this work, we present a novel appearance model for visual tracking. The proposed appearance model considers the similarity and the distinctiveness of the feature elements of a target candidate. The feature elements should share some similarity to jointly represent a target pattern. We exploit the distinctiveness of feature elements to represent the different importance by introducing a weighted regularization term in the appearance model. A more stable and discriminative target representation is obtained. Superior performance on challenging sequences against state-of-the-art trackers show the robustness of the novel appearance model and the proposed tracker.

Robust UAV-based Tracking Using Hybrid Classifiers.

Visual object tracking is a key factor for unmanned aerial vehicles (UAVs). In this paper, we propose a robust and effective visual object tracking method with an appearance model based on the locally adaptive regression kernel (LARK). The proposed appearance model encodes the geometric structure of the target. The tracking problem is formulated as two binary classifiers via two support vector machines (SVMs) with online model update. The backward tracking which tracks the target in reverse of time is utilized to evaluate the accuracy and robustness of the two SVMs. The final locations are adaptively fused based on the results of the forward tracking and backward tracking validation. Several state-of-the-art tracking algorithms are evaluated on large scale benchmark datasets which include challenging factors such as heavy occlusion, pose variation, illumination variation and motion blur. Experimental results demonstrate that our method achieves appealing performance.

Single object tracking using particle filter framework and saliency-based weighted color histogram

Despite many years of research, object tracking remains a challenging problem, not only because of the variety of object appearances, but also because of the complexity of surrounding environments. In this research, we present an algorithm for single object tracking using a particle filter framework and color histograms. Particle filters are iterative algorithms that perform predictions in each iteration using particles, which are samples drawn from a statistical distribution. Color histograms are embedded in these particles, and the distances between histograms are used to measure likelihood between targets and observations. One downside of color histograms is that they ignore spatial information, which may produce tracking failure when objects appear that are similar in color. To overcome this disadvantage, we propose a saliency-based weighting scheme for histogram calculation. Given an image region, first its saliency map is generated. Next, its histogram is calculated based on the generated saliency map. Pixels located in salient regions have higher weights than those in others, which helps preserve the spatial information. Experimental results showed the efficiency of the proposed appearance model in object tracking under various conditions.

A hybrid occlusion free object tracking method using particle filter and modified galaxy based search meta-heuristic algorithm

An important problem in tracking methods is how to manage the changes in object appearance, such as illumination changes, partial/full occlusion, scale, and pose variation during the tracking process. In this paper, we propose an occlusion free object tracking method together with a simple adaptive appearance model. The proposed appearance model which is updated at the end of each time step includes three components: the first component consists of a fixed template of target object, the second component shows rapid changes in object appearance, and the third one maintains slow changes generated along the object path. The proposed tracking method not only can detect occlusion and handle it, but also it is robust against changes in the object appearance model. It is based on particle filter which is a robust technique in tracking and handles non-linear and non-Gaussian problems. We have also employed a meta-heuristic approach that is called Modified Galaxy based Search Algorithm (MGbSA), to reinforce finding the optimum state in the particle filter state space. The proposed method was applied to some benchmark videos and its results were satisfactory and better than results of related works.

Multiple cues-based active contours for target contour tracking under sophisticated background

In this paper, we propose a novel target contour tracking method under sophisticated background using the multiple cues-based active contour model. To locate the target position, a contour-based mean-shift tracker is designed which combines both color and texture information. To reduce the adverse impact of sophisticated background and also accelerate the curve motion, we propose a two-layer-based target appearance model that combines both discriminative pre-learned-based global layer and voting-based local layer. The proposed appearance model is able to extract rough target region from the complex background, which provides important target region information for our active contour model. We subsequently introduce a dynamical shape model to provide prior target shape information for more stable segmentation. To obtain accurate target boundaries, we design a new multiple cues-based active contour model which integrates with target edge, discriminative region, and shape information. The experimental results on 30 video sequences demonstrate that the proposed method outperforms other competitive contour tracking methods under various tracking environment.

Multi-human tracking using part-based appearance modelling and grouping-based tracklet association for visual surveillance applications

Although much progress has been made in multi-object tracking in recent decades due to its variety of applications including visual surveillance, traffic monitoring and medical image analysis, some difficult challenges such as the variation of object appearance and partial occlusion are still going on. In this work, we propose an effective multi-human tracking system called part-based appearance modelling and grouping-based tracklet association-based multi-human tracking (PAM-GTA-MHT). The proposed appearance model based on the upper body-centered multi-view human body part model can effectively resolve the drawback caused by inter-object occlusions and low camera positions. The grouping method embedded in global tracklet association can improve discriminability among targets with similar appearances when they are located sufficiently far away from each other. Thus, there is no need to compare all possible pairs of the detected targets in the tracklet association stage and thus it has the potential to enhance the tracking speed. We quantitatively evaluated the performance of our proposed approach on four challenging publicly available datasets and achieved a significant improvement compared to the state-of-the-art methods.

Fast Compressive Tracking.

It is a challenging task to develop effective and efficient appearance models for robust object tracking due to factors such as pose variation, illumination change, occlusion, and motion blur. Existing online tracking algorithms often update models with samples from observations in recent frames. Despite much success has been demonstrated, numerous issues remain to be addressed. First, while these adaptive appearance models are data-dependent, there does not exist sufficient amount of data for online algorithms to learn at the outset. Second, online tracking algorithms often encounter the drift problems. As a result of self-taught learning, misaligned samples are likely to be added and degrade the appearance models. In this paper, we propose a simple yet effective and efficient tracking algorithm with an appearance model based on features extracted from a multiscale image feature space with data-independent basis. The proposed appearance model employs non-adaptive random projections that preserve the structure of the image feature space of objects. A very sparse measurement matrix is constructed to efficiently extract the features for the appearance model. We compress sample images of the foreground target and the background using the same sparse measurement matrix. The tracking task is formulated as a binary classification via a naive Bayes classifier with online update in the compressed domain. A coarse-to-fine search strategy is adopted to further reduce the computational complexity in the detection procedure. The proposed compressive tracking algorithm runs in real-time and performs favorably against state-of-the-art methods on challenging sequences in terms of efficiency, accuracy and robustness.

Learning local appearances with sparse representation for robust and fast visual tracking.

In this paper, we present a novel appearance model using sparse representation and online dictionary learning techniques for visual tracking. In our approach, the visual appearance is represented by sparse representation, and the online dictionary learning strategy is used to adapt the appearance variations during tracking. We unify the sparse representation and online dictionary learning by defining a sparsity consistency constraint that facilitates the generative and discriminative capabilities of the appearance model. An elastic-net constraint is enforced during the dictionary learning stage to capture the characteristics of the local appearances that are insensitive to partial occlusions. Hence, the target appearance is effectively recovered from the corruptions using the sparse coefficients with respect to the learned sparse bases containing local appearances. In the proposed method, the dictionary is undercomplete and can thus be efficiently implemented for tracking. Moreover, we employ a median absolute deviation based robust similarity metric to eliminate the outliers and evaluate the likelihood between the observations and the model. Finally, we integrate the proposed appearance model with the particle filter framework to form a robust visual tracking algorithm. Experiments on benchmark video sequences show that the proposed appearance model outperforms the other state-of-the-art approaches in tracking performance.

Highly Nonrigid Object Tracking via Patch-Based Dynamic Appearance Modeling

A novel tracking algorithm is proposed for targets with drastically changing geometric appearances over time. To track such objects, we develop a local patch-based appearance model and provide an efficient online updating scheme that adaptively changes the topology between patches. In the online update process, the robustness of each patch is determined by analyzing the likelihood landscape of the patch. Based on this robustness measure, the proposed method selects the best feature for each patch and modifies the patch by moving, deleting, or newly adding it over time. Moreover, a rough object segmentation result is integrated into the proposed appearance model to further enhance it. The proposed framework easily obtains segmentation results because the local patches in the model serve as good seeds for the semi-supervised segmentation task. To solve the complexity problem attributable to the large number of patches, the Basin Hopping (BH) sampling method is introduced into the tracking framework. The BH sampling method significantly reduces computational complexity with the help of a deterministic local optimizer. Thus, the proposed appearance model could utilize a sufficient number of patches. The experimental results show that the present approach could track objects with drastically changing geometric appearance accurately and robustly.

Visual tracking via dynamic tensor analysis with mean update

The appearance model is an important issue in the visual tracking community. Most subspace-based appearance models focus on the time correlation between the image observations of the object, but the spatial layout information of the object is ignored. This paper proposes a robust appearance model for visual tracking which effectively combines the spatial and temporal eigen-spaces of the object in a tensor reconstruction way. In order to capture the variations in object appearance, an incremental updating strategy is developed to both update the eigen-space and mean of the object. Experimental results demonstrate that, compared with the state-of-the-art appearance models in the tracking literature, the proposed appearance model is more robust and effective.

Modeling human color perception under extended luminance levels

Display technology is advancing quickly with peak luminance increasing significantly, enabling high-dynamic-range displays. However, perceptual color appearance under extended luminance levels has not been studied, mainly due to the unavailability of psychophysical data. Therefore, we conduct a psychophysical study in order to acquire appearance data for many different luminance levels (up to 16,860 cd/m 2 ) covering most of the dynamic range of the human visual system. These experimental data allow us to quantify human color perception under extended luminance levels, yielding a generalized color appearance model. Our proposed appearance model is efficient, accurate and invertible. It can be used to adapt the tone and color of images to different dynamic ranges for cross-media reproduction while maintaining appearance that is close to human perception.

Human appearance modeling for matching across video sequences

We present an appearance model for establishing correspondence between tracks of people which may be taken at different places, at different times or across different cameras. The appearance model is constructed by kernel density estimation. To incorporate structural information and to achieve invariance to motion and pose, besides color features, an additional feature of path-length is used. To achieve illumination invariance, two types of illumination insensitive color features are discussed: brightness color feature and RGB rank feature. The similarity between a test image and an appearance model is measured by the information gain or Kullback–Leibler distance. To thoroughly represent the information contained in a video sequence with as little data as possible, a key frame selection and matching scheme is proposed. Experimental results demonstrate the important role of the path-length feature in the appearance model and the effectiveness of the proposed appearance model and matching method.

Face recognition under variable pose and illumination conditions using 3D facial appearance models

AbstractThis paper proposes a method in which an appearance model is constructed from 3D facial shape data that can describe the image variation due to the arbitrariness of the pose and the variation of the illumination, after which recognition is performed by fitting the model to facial images when the illumination conditions and the precise pose are unknown. The appearance model in the proposed method is constructed as follows. The illumination basis for an image of an arbitrary pose is derived from the geodesic illumination basis describing the brightness on the 3D object surface depending on the illumination variations, and an image with the same illumination condition is reproduced as the target image for the unknown illumination conditions. By optimizing the pose to minimize the reproduction error, model fitting with high accuracy is realized even if the exact pose is unknown. By experiment, the number of illumination samples needed in the calculation of the geodesic illumination basis is evaluated and it is verified that the proposed appearance model can describe arbitrary illumination variations in images with various poses. The robustness of the proposed pose optimization method against initial pose estimate is also evaluated. The effectiveness of the proposed method is demonstrated by a recognition experiment using 14,000 facial images taken in various situations, including extreme illumination variations such as backlighting, for a wide range of poses from frontal to 45° upward and 60° sidewise. © 2007 Wiley Periodicals, Inc. Syst Comp Jpn, 38(2): 57– 70, 2007; Published online in Wiley InterScience (www.interscience. wiley.com). DOI 10.1002/scj.20646