Local Appearance Model Research Articles

Robust Multi-Object Tracking With Local Appearance and Stable Motion Models

Robust Multi-Object Tracking With Local Appearance and Stable Motion Models

Robust coding in a global subspace model and its collaboration with a local model for visual tracking

The sparse representation-based trackers has attracted much attention in the research community due to its superior performance in spite of its computational complexity. But the assumption that the coding residual follows either the Gaussian or the Laplacian distribution may not accurately describe the coding residual in practical visual tracking scenarios. To deal with such issues as well as to improve the performance of the visual tracking, a novel generative tracker is proposed in a Bayesian inference framework by introducing robust coding (RC) into the PCA reconstruction. Also, it is proposed to collaborate the global and local PCA subspace appearance models to enhance the tracking performance. Further, a robust RC distance is proposed to differentiate the candidate samples from the subspace, and a novel observation likelihood is defined based on both global and local RC distances. In addition, a robust occlusion map generation and a novel appearance model update mechanism are proposed. The quantitative and qualitative performance evaluations on the OTB-50 and VOT2016 dataset demonstrate that the proposed method performs favorably against several methods based on particle filter framework.

Robust particle tracking via spatio-temporal context learning and multi-task joint local sparse representation

Particle filters have been proven very successful for non-linear and non-Gaussian estimation problems and extensively used in object tracking. However, high computational costs and particle decadency problem limit its practical application. In this paper, we present a robust particle tracking approach based on spatio-temporal context learning and multi-task joint local sparse representation. The proposed tracker samples particles according to the confidence map constructed by the spatio-temporal context information of the target. This sampling strategy can ameliorate problems of sample impoverishment and particle degeneracy, target state distribution to obtain robust tracking performance. In order to locate the target more accurately and be less sensitive to occlusion, the local sparse appearance model is adopted to capture the local and structural information of the target. Finally, the multi-task learning where the representations of particles are learned jointly is employed to further improve tracking performance and reduce overall computational complexity. Both qualitative and quantitative evaluations on challenging benchmark image sequences have demonstrated that the proposed tracking algorithm performs favorably against several state-of-the-art methods.

Dual‐scale weighted structural local sparse appearance model for object tracking

It is a great challenge to develop an effective appearance model for robust visual tracking due to various interfering factors, such as pose change, occlusion, background clutter etc. More and more visual tracking methods tend to exploit the local appearance model to deal with the above challenges. In this study, the authors present a simple yet effective weighted structural local sparse appearance model, which can better describe the target appearance information through patch‐based generative weight. To further improve the robustness of tracking, they implement this appearance model on two‐scale patches. The two derived appearance models are then combined to form a collaborative model to play their advantages. Extensive experiments on the tracking benchmark dataset show that the proposed method performs favourably against several state‐of‐the‐art methods.

Part-Based Background-Aware Tracking for UAV With Convolutional Features

In recent years, visual tracking is a challenging task in UAV applications. The standard correlation filter (CF) has been extensively applied for UAV object tracking. However, the CF-based tracker severely suffers from boundary effects and cannot effectively cope with object occlusion, which results in suboptimal performance. Besides, it is still a tough task to obtain an appearance model precisely with hand-crafted features. In this paper, a novel part-based tracker is proposed for the UAV. With successive cropping operations, the tracking object is separated into several parts. More specially, the background-aware correlation filters with different cropping matrices are applied. To estimate the translation and scale variation of the tracking object, a structured comparison, and a Bayesian inference approach are proposed, which jointly achieve a coarse-to-fine strategy. Moreover, an adaptive mechanism is used to update the local appearance model of each part with a Gaussian process regression method. To construct a better appearance model, features extracted from the convolutional neural network are utilized instead of hand-crafted features. Through extensive experiments, the proposed tracker reaches competitive performance on 123 challenging UAV image sequences and outperforms other 20 popular state-of-the-art visual trackers in terms of overall performance and different challenging attributes.

Visual tracking via context-aware local sparse appearance model

Most existing local sparse trackers are prone to drifting away as they do not make use of discriminative information of local patches. In this paper, we propose an effective context-aware local sparse appearance model to alleviate the drift problem caused by background clutter and occlusions. First, considering that different local patches should have different impacts on the likelihood computation, we present a novel Impact Allocation Strategy (IAS) with integration of the spatial-temporal context. Varying positive impact factors are adaptively assigned to different local patches based on their ability distinguishing the spatial context, which provides discriminative information to prevent the tracker from drifting. Furthermore, we exploit temporal context to introduce some historical information for more accurate locating. Second, we present a new patch-based dictionary update method being able to update each patch independently with the validation of effectiveness. On the one hand, we introduce sparsity concentration index to check whether the local patch to be updated is a valid local patch from the target object. On the other hand, spatial context is further employed to eliminate the effect of the background. Experimental results show the superiority and competitiveness of the proposed method on the benchmark data set compared to other state-of-the-art algorithms.

Object tracking with an effective composite similarity

How to achieve a robust performance remains an intractable problem with the various object tracking algorithms, due to some unfavorable factors, e.g., occlusion, appearance change, etc. A robust object-tracking approach is proposed based on a composite similarity measure. Based on the first several frames in a video sequence, an initial dictionary is constructed by searching for the best candidates via the kd-tree method. For a target candidate, a likelihood value is computed by weighting the sparse representation coefficients of local patches obtained via the adaptive structural local sparse appearance (ASLSA) model. In order to alleviate the effect of occlusion, a smoothing operation is designed in computing the likelihood value. Meanwhile, after obtaining the codebook by applying the k-means clustering algorithm on the randomly sampled positive and negative templates, a confidence value is computed, via the sparse collaborative model (SCM), for each of the candidates. Furthermore, a composite similarity measure is devised, which combines the likelihood value and the confidence value for each candidate. Finally, the best candidate is searched for, according to the composite similarity measure. Due to the complementarity of SCM and ASLSA, the proposed algorithm is a more robust approach to object tracking. Experimental results on several challenging sequences demonstrate the effectiveness and feasibility of the proposed method.

Visual tracking using structural local DCT sparse appearance model with occlusion detection

In this paper, a structural local DCT sparse appearance model with occlusion detection is proposed for visual tracking in a particle filter framework. The energy compaction property of the 2D-DCT is exploited to reduce the size of the dictionary as well as that of the candidate samples so that the computational cost of l1-minimization can be lowered. Further, a holistic image reconstruction procedure is proposed for robust occlusion detection and used for appearance model update, thus avoiding the degradation of the appearance model in the presence of occlusion/outliers. Also, a patch occlusion ratio is introduced in the confidence score computation to enhance the tracking performance. Quantitative and qualitative performance evaluations on two popular benchmark datasets demonstrate that the proposed tracking algorithm generally outperforms several state-of-the-art methods.

Facial Landmark Detection: A Literature Survey

The locations of the fiducial facial landmark points around facial components and facial contour capture the rigid and non-rigid facial deformations due to head movements and facial expressions. They are hence important for various facial analysis tasks. Many facial landmark detection algorithms have been developed to automatically detect those key points over the years, and in this paper, we perform an extensive review of them. We classify the facial landmark detection algorithms into three major categories: holistic methods, Constrained Local Model (CLM) methods, and the regression-based methods. They differ in the ways to utilize the facial appearance and shape information. The holistic methods explicitly build models to represent the global facial appearance and shape information. The CLMs explicitly leverage the global shape model but build the local appearance models. The regression-based methods implicitly capture facial shape and appearance information. For algorithms within each category, we discuss their underlying theories as well as their differences. We also compare their performances on both controlled and in the wild benchmark datasets, under varying facial expressions, head poses, and occlusion. Based on the evaluations, we point out their respective strengths and weaknesses. There is also a separate section to review the latest deep learning-based algorithms. The survey also includes a listing of the benchmark databases and existing software. Finally, we identify future research directions, including combining methods in different categories to leverage their respective strengths to solve landmark detection "in-the-wild".

Structural Regression Model Based Inverse Sparse Representation for Tracking Objects

In order to reduce the calculation cost and improve the robustness of appearance model, this paper presents an optimal object tracking method that consists of improved inverse sparse representation and global spatial envelope. First, partial least squares regression-based structural model is adopted, which easily facilitates target template sparsely represented by candidate dictionary. Furthermore, candidates with nonzero coefficients are easily selected as possible tracking results. Meanwhile, partial occlusion and slight appearance changes are effectively alleviated during the tracking process. Second, spatial envelope in the frequency domain is utilized to select the best candidate from the inverse sparse representation process. Multiple scales and orientations-based Gabor filters are established to obtain the Gist information, which keeps the potential structural attributes of local appearance models to tolerate appearance variation. In addition, the Bayesian inference framework is used to exploit candidate samples, and a simple model update scheme is employed to alleviate drifting caused by temporal varying multi-factors. The qualitative experimental results show that the proposed tracking algorithm provides a better performance in some dynamic scenes.

Local appearance modeling for objects class recognition

In this work, we propose a new formulation of the objects modeling combining geometry and appearance; it is useful for detection and recognition. The object local appearance location is referenced with respect to an invariant which is a geometric landmark. The appearance (shape and texture) is a combination of Harris–Laplace descriptor and local binary pattern (LBP), all being described by the invariant local appearance model (ILAM). We use an improved variant of LBP traits at regions located by Harris–Laplace detector to encode local appearance. We applied the model to describe and learn object appearances (e.g., faces) and to recognize them. Given the extracted visual traits from a test image, ILAM model is carried out to predict the most similar features to the facial appearance: first, by estimating the highest facial probability and then in terms of LBP histogram-based measure, by computing the texture similarity. Finally, by a geometric calculation the invariant allows to locate an appearance in the image. We evaluate the model by testing it on different face images databases. The experiments show that the model results in high accuracy of detection and provides an acceptable tolerance to the appearance variability.

Interactive Video Segmentation via Local Appearance Model

In numerous video segmentation algorithms, shape and color priors from previous frames are propagated to successive frames for processing. One prime issue of the existing algorithms is how priors are modeled and propagated effectively. This paper proposes a novel algorithm for accurate and robust foreground prediction via a local appearance model based on shape and color. In the shape estimation process, instead of performing global matching, a local search mechanism is developed to capture complex motions. In addition, global information is used to facilitate local matching for the final shape estimation. Color cues from multiple frames are used to estimate the foreground (background) pixel distribution in the current image. Furthermore, the contribution of each frame is weighted based on its reliability and discriminative strength. Based on the proposed local appearance model, a graph cut algorithm is used to generate segmentation results. The experimental results show that accurate video segmentation can be obtained by the proposed algorithm with fewer user interactions.

Object tracking via a cooperative appearance model

Object tracking has been wildly used in security monitoring, traffic control, medical imaging and other fields. Conventional algorithms design local and holistic appearance model for effectively tracking. However, the performance of algorithms decreases in the complex scenes, including deformation and background cluster, partial or full occlusion and so on. In this paper, an object-tracking algorithm via a cooperative appearance model is proposed. Considering the visual characteristics of human eyes, we propose the impact region with different impact factors. The impact regions are defined as the regions which play different effects in making decision. The pixels in the regions with different distance from the target center will have different importance. We divide the impact regions into the significant impact region, the sub-impact region, and the non-impact region. The cooperative appearance model uses local collaborative representation to rectify holistic representation with impact regions. In local representation, positive and negative dictionary are derived from the candidates of video frames. The candidates are segmented into non-overlapping sub-blocks, and the sub-block responses of each candidate are obtained based on a collaborative dictionary. In holistic representation, the candidates are represented sparsely to obtain the total reconstruction error. The tracking result is decided by combing the sub-block responses in local representation and the reconstruction error in holistic representation with impact regions. The experimental results show that the proposed algorithm has performed well on deformation and illumination variation, partial or full occlusion, scale variation and background cluster compared with the state-of-the-art algorithms.

Vision‐based wheelchair navigation using geometric AdaBoost learning

A novel training algorithm called geometric AdaBoost learning, which integrates the local-appearance models with the explicit shape model is proposed. The proposed algorithm employs a two-stage AdaBoost learning algorithm. The first-stage learning is performed to learn the local texture model within local image patches and to produce a confidence map. Based on the confidence values, the high-discriminative local patches are selected, and then the global context models between them are trained in the later stage using AdaBoost learning. The proposed algorithm is applied to wheelchair navigation, and the results demonstrate that it outperforms the state-of-the-art algorithms with improvements of 23.3 and 49% in terms of accuracy and speed.

Robust object tracking based on local discriminative sparse representation.

Despite much success in the application of sparse representation to object tracking, most of the existing sparse-representation-based tracking methods are still not robust enough for challenges such as pose variations, illumination changes, occlusions, and background distractions. In this paper, we propose a robust object-tracking algorithm via local discriminative sparse representation. The key idea in our method is to develop what we believe is a novel local discriminative sparse representation method for object appearance modeling, which can be helpful to overcome issues such as appearance variations and occlusions. Then a robust tracker based on the local discriminative sparse appearance model is proposed to track the object over time. Additionally, an online dictionary update strategy is introduced in our approach for further robustness. Experimental results on challenging sequences demonstrate the effectiveness and robustness of our proposed method.

Real-Time Object Tracking via Fusion of Global and Local Appearance Models

Real-Time Object Tracking via Fusion of Global and Local Appearance Models

Fully automatic segmentation of femurs with medullary canal definition in high and in low resolution CT scans

Femur segmentation can be an important tool in orthopedic surgical planning. However, in order to overcome the need of an experienced user with extensive knowledge on the techniques, segmentation should be fully automatic. In this paper a new fully automatic femur segmentation method for CT images is presented. This method is also able to define automatically the medullary canal and performs well even in low resolution CT scans.Fully automatic femoral segmentation was performed adapting a template mesh of the femoral volume to medical images. In order to achieve this, an adaptation of the active shape model (ASM) technique based on the statistical shape model (SSM) and local appearance model (LAM) of the femur with a novel initialization method was used, to drive the template mesh deformation in order to fit the in-image femoral shape in a time effective approach.With the proposed method a 98% convergence rate was achieved. For high resolution CT images group the average error is less than 1mm. For the low resolution image group the results are also accurate and the average error is less than 1.5mm.The proposed segmentation pipeline is accurate, robust and completely user free. The method is robust to patient orientation, image artifacts and poorly defined edges. The results excelled even in CT images with a significant slice thickness, i.e., above 5mm. Medullary canal segmentation increases the geometric information that can be used in orthopedic surgical planning or in finite element analysis.

Dual-scale structural local sparse appearance model for robust object tracking

Recently, sparse representation has been applied in object tracking successfully. However, the existing sparse representation captures either the holistic features of the target or the local features of the target. In this paper, we propose a dual-scale structural local sparse appearance (DSLSA) model based on overlapped patches, which can capture the quasi-holistic features and the local features of the target simultaneously. This paper first proposes two-scales structural local sparse appearance models based on overlapped patches. The larger-scale model is used to capture the structural quasi-holistic feature of the target, and the smaller-scale model is used to capture the structural local features of the target. Then, we propose a new mechanism to associate these two scale models as a new dual-scale appearance model. Both qualitative and quantitative analyses on challenging benchmark image sequences indicate that the tracker with our DSLSA model performs favorably against several state-of-the-art trackers.

Robust visual tracking based on generative and discriminative model collaboration

Effective object appearance model is one of the key issues for the success of visual tracking. Since the appearance of a target and the environment changes dynamically, the majority of existed visual tracking algorithms tend to drift away from targets. To address this issue, we propose a robust tracking algorithm by integrating the generative and discriminative model. The object appearance model is made up of generative target model and a discriminative classifier. For the generative target model, we adopt the weighted structural local sparse appearance model combining patch based gray value and Histogram of Oriented Gradients feature as the patch dictionary. By sampling positives and negatives, alignment-pooling features are obtained based on the patch dictionary through local sparse coding, then we use support vector machine to train the discriminative classifier. The proposed method is embedded into a Bayesian inference framework for visual tracking. A combined matching method is adopted to improve the proposal distribution of the particle filter. Moreover, in order to adapt the situation change, the patch dictionary and discriminative classifier are updated by incremental learning every five frames. Experimental results on some publicly available benchmarks of video sequences demonstrate the accuracy and effectiveness of our tracker.

Local subspace smoothness alignment for constrained local model fitting

Constrained local model (CLM) is a classic method for facial landmarks estimation. While the CLM enhances the well-known active shape model with discriminative local appearance models, its shape model is based on the point distribution model, which is essentially principal component analysis over the training facial shape vectors and hence the nonlinear manifold of facial shapes is not well embedded. In this paper, we propose a novel manifold learning method, i.e., local subspace smoothness alignment (LSSA), to address this issue. The LSSA approach smoothes the nonlinear structure directly in the original feature space, with a newly defined geometric measure for the curvature of the local structures. We then proceed to apply this method for face alignment, with an ensemble of correlated local subspaces derived from LSSA. The proposed method is demonstrated on both toy data and real-world datasets that it yields reasonable manifold embedding and leads to encouraging performance for face alignment even under difficult conditions.