Tracking Of Deformable Objects Research Articles

Deformable attention object tracking network based on cross-correlation

With the increasing popularity and widespread application of embedded devices, lightweight target tracking networks have emerged as a crucial research direction in the field of computer vision. In this paper, we propose a deformable attention object tracking network based on cross-correlation (DAM-LOTNet), which utilizes the lightweight MobileNetV3 network for efficient feature extraction. Aiming at the problem that lightweight networks cannot better capture target features and morphological changes, deformable attention modules are proposed to generate attention maps along the channel and spatial dimensions. Furthermore, a dual cross-correlation operation is proposed, aiming at fusing channel information and pixel-level information. Thus, the model obtains a more comprehensive feature representation and a more accurate similarity calculation capability while maintaining a lightweight framework. Finally, a feature enhancement network constructed by depthwise separable convolution was added to the classification and regression branches. The designed DAM-LOTNet has 0.552M parameters and 0.121G FLOPs. On the UAV123 dataset, DAM-LOTNet achieves state-of-the-art performance compared to other lightweight object tracking algorithms. Extensive experiments on visual tracking benchmarks (including VOT2018, VOT2019, OTB100, etc.) have demonstrated that DAM-LOTNet has extremely low model FLOPs and parameters compared to common deep learning-based trackers, and is essentially on par with these trackers in terms of performance.

Soft tissue monitoring of the surgical field: detection and tracking of breast surface deformations.

Deformable object tracking is common in the computer vision field, with applications typically focusing on nonrigid shape detection and usually not requiring specific 3D point localization.In surgical guidance however, accurate navigation is intrinsically linked to precise correspondence of tissue structure. This work presents a contactless, automated fiducial acquisition method using stereo video of the operating field to provide reliable fiducial localization for an image guidance framework in breast conserving surgery. On n=8 breasts from healthy volunteers, the breast surface was measured throughout the full range of arm motion in a supine mock-surgical position. Using hand-drawn inked fiducials, adaptive thresholding, and KAZE feature matching, precise three-dimensional fiducial locations were detected and tracked through tool interference, partial and complete marker occlusions, significant displacements and nonrigid shape distortions. Compared to digitization with a conventional optically tracked stylus, fiducials were automatically localized with 1.6 ± 0.5 mm accuracy and the two measurement methods did not significantly differ. The algorithm provided an average false discovery rate <0.1% with all cases' rates below 0.2%. On average, 85.6 ± 5.9% of visible fiducials were automatically detected and tracked, and 99.1 ± 1.1% of frames provided only true positive fiducial measurements, which indicates the algorithm achieves a data stream that can be used for reliable on-line registration. Tracking is robust to occlusions, displacements, and most shape distortions. This work-flow friendly data collection method provides highly accurate and precise three-dimensional surface data to drive an image guidance system for breast conserving surgery.

Particle Filters in Latent Space for Robust Deformable Linear Object Tracking

Tracking of deformable linear objects (DLOs) is important for many robotic applications. However, achieving robust and accurate tracking is challenging due to the lack of distinctive features or appearance on the DLO, the object's high-dimensional state space, and the presence of occlusion. In this letter, we propose a method for tracking the state of a DLO by applying a particle filter approach within a lower-dimensional state embedding learned by an autoencoder. The dimensionality reduction preserves state variation, while simultaneously enabling a particle filter to accurately track DLO state evolution with a practically feasible number of particles. Compared to previous works, our method requires neither running a high-fidelity physics simulation, nor manual designs of constraints and regularization. Without the assumption of knowing the initial DLO state, our method can achieve accurate tracking even under complex DLO motions and in the presence of severe occlusions.

Object Tracking in Video Using the TLD and CMT Fusion Model

Object tracking has been an attractive study topic in computer vision in recent years, thanks to the development of video monitoring systems. Tracking-Learning Detection (TLD), Compressive Tracking (CT), and Clustering of Static-Adaptive Correspondences for Deformable Object Tracking are some of the state-of-the-art methods for motion object tracking (CMT). We present a fusion model that combines TLD and CMT in this study. To restrict the calculation time of the CMT technique, the fusion TLD CMT model enhanced the TLD benefits of computation time and accuracy on t no deformable objects. The experimental results on the Vojir dataset for three techniques (TLD, CMT, and TLD CMT) demonstrated that our fusion proposal successfully trades off CMT accuracy for computing time.

Tracking of Deformable Objects Using Dynamically and Robustly Updating Pictorial Structures.

The problem posed by complex, articulated or deformable objects has been at the focus of much tracking research for a considerable length of time. However, it remains a major challenge, fraught with numerous difficulties. The increased ubiquity of technology in all realms of our society has made the need for effective solutions all the more urgent. In this article, we describe a novel method which systematically addresses the aforementioned difficulties and in practice outperforms the state of the art. Global spatial flexibility and robustness to deformations are achieved by adopting a pictorial structure based geometric model, and localized appearance changes by a subspace based model of part appearance underlain by a gradient based representation. In addition to one-off learning of both the geometric constraints and part appearances, we introduce a continuing learning framework which implements information discounting i.e., the discarding of historical appearances in favour of the more recent ones. Moreover, as a means of ensuring robustness to transient occlusions (including self-occlusions), we propose a solution for detecting unlikely appearance changes which allows for unreliable data to be rejected. A comprehensive evaluation of the proposed method, the analysis and discussing of findings, and a comparison with several state-of-the-art methods demonstrates the major superiority of our algorithm.

RGB-D Tracking and Optimal Perception of Deformable Objects

Addressing the perception problem of texture-less objects that undergo large deformations and movements, this article presents a novel RGB-D learning-free deformable object tracker in combination with a camera position optimisation system for optimal deformable object perception. The approach is based on the discretisation of the object’s visible area through the generation of a supervoxel graph that allows weighting new supervoxel candidates between object states over time. Once a deformation state of the object is determined, supervoxels of its associated graph serve as input for the camera position optimisation problem. Satisfactory results have been obtained in real time with a variety of objects that present different deformation characteristics.

Supplementary Virtual Keypoints of Weight-Based Correspondences for Occluded Object Tracking

In this paper, we propose a short-term tracking method, which is more robust than conventional methods for single-target tracking under occlusions. On the one hand, our method uses the features-supplemental points and optical flows for static-adaptive targets based on the online tracking algorithm, which clusters the static-adaptive correspondences for deformable object tracking. On the other hand, our method combines the locally matched target bounding box and the one from optical flows tracking using novel fuzzy weights, which are updated based on the changes in keypoints. In the proposed weighted-based keypoints matching tracker, we employ fast keypoints detection and partial descriptors, which makes it run in real time. We demonstrate the effectiveness of our method extensively on benchmark data sets that contain scenes of object deformation and occlusions. The results show that our algorithm performs more favorably against the competing tracking methods.

Online Deformable Object Tracking Based on Structure-Aware Hyper-Graph.

Recent advances in online visual tracking focus on designing part-based model to handle the deformation and occlusion challenges. However, previous methods usually consider only the pairwise structural dependences of target parts in two consecutive frames rather than the higher order constraints in multiple frames, making them less effective in handling large deformation and occlusion challenges. This paper describes a new and efficient method for online deformable object tracking. Different from most existing methods, this paper exploits higher order structural dependences of different parts of the tracking target in multiple consecutive frames. We construct a structure-aware hyper-graph to capture such higher order dependences, and solve the tracking problem by searching dense subgraphs on it. Furthermore, we also describe a new evaluating data set for online deformable object tracking (the Deform-SOT data set), which includes 50 challenging sequences with full annotations that represent realistic tracking challenges, such as large deformations and severe occlusions. The experimental result of the proposed method shows considerable improvement in performance over the state-of-the-art tracking methods.

Real-time tracking of deformable objects based on MOK algorithm

The traditional oriented FAST and rotated BRIEF (ORB) algorithm has problems of instability and repetition of keypoints and it does not possess scale invariance. In order to deal with these drawbacks, a modified ORB (MORB) algorithm is proposed. In order to improve the precision of matching and tracking, this paper puts forward an MOK algorithm that fuses MORB and Kanade-Lucas-Tomasi (KLT). By using Kalman, the object's state in the next frame is predicted in order to reduce the size of search window and improve the real-time performance of object tracking. The experimental results show that the MOK algorithm can accurately track objects with deformation or with background clutters, exhibiting higher robustness and accuracy on diverse datasets. Also, the MOK algorithm has a good real-time performance with the average frame rate reaching 90.8 fps.

Real-time tracking of deformable objects based on combined matching-and-tracking

Visual tracking is very challenging due to the existence of several sources of variations, such as partial occlusion, deformation, scale variation, rotation, and background clutter. A model-free tracking method based on fusing accelerated features using fast explicit diffusion in nonlinear scale spaces (AKAZE) and KLT features is presented. First, matching-keypoints are generated by finding corresponding keypoints from the consecutive frames and the object template, then tracking-keypoints are generated using the forward–backward flow tracking method, and at last, credible keypoints are obtained by AKAZE-KLT tracking (AKT) algorithm. To avoid the instability of a statistical method, the median method is adopted to compute the object’s location, scale, and rotation in each frame. The experimental results show that the AKT algorithm has strong robustness and can achieve accurate tracking especially under conditions of partial occlusion, scale variation, rotation, and deformation. The tracking performance shows higher robustness and accuracy in a variety of datasets and the average frame rate reaches 78 fps, showing good performance in real time.

Deformable Models and Optimal Mass Transportation for Processing of DCE-MRI 2D Images

Post-contrastographic Dynamic Contrast Enhancement (DCE) is a consolidated MRI technique to perform non-invasive analysis via Imaging. The dynamics perfusion of specific drugs, named Contrast Agents (CA), allows to highlight anomalies in tissues, since the vascular substrate is differently developed (hypervascularized) for diseases compared with healthy parenchyma. The goodness of the analysis procedure influences the inferred diagnosis: the more objective the analysis, the more accurate the prognosis and the therapies follow-up. Deformations and displacements affect organs and soft tissues and their correction is challenging for DCE analysis; the complexity lies also on artefacts of image registration process because of the backscattered image signal intensity varies over time due to the diffusion/perfusion phenomenon induced by the CA injection. In this work we present a combined approach based on the Active Contours framework and on the Optimal Mass Transportation for the automatic depicting and tracking of deformable objects. Active Contours provide the automatical contour's depiction of a deformable object by minimizing an energy cost functional; Optimal Mass Transportation allows to tracking a deformable object by minimizing the (object's) transport cost under the (object) mass conservation principle. We applied these methodologies to post-contrastographic DCE MR images' series and we are going to show the improvement in the goodness of DCE analysis compared with results gathered with other tracking strategies.

Online tracking of deformable objects under occlusion using dominant points

This paper deals with tracking of deformable objects in the presence of occlusion using dominant point representation of the boundary contour. A novel nonintegral time propagation model for propagating the dominant points is proposed. It uses an initial guess generated from a linear operation and an analytical conjugate gradient approach for online robust learning of the shape deformation and motion model. A scheme is presented to automatically detect and correct the region of large local deformation. In order to deal with occlusion, admissible restrictions on deformation and motion of the object are automatically determined. The proposed method overcomes the need of offline learning and learns the deformation and motion model of the object using very few initial frames of the input video. The performance of the method is demonstrated using varieties of videos of different objects.

Deformable Object Tracking Based on Lie Algebra

Dynamic deformation of object is a distinct problem in image-based tracking.A novel deformable object tracking method based on Lie algebra is presented.This method uses Gabor feature as target token,models deformation using affine Lie group,and optimizes parameters directly on manifold,which can be solved by exponential mapping between Lie group and its Lie algebra.We physically illustrate the essential of the geometric transformation in the tracking,then analyze the advantage of our method and give a direct proof of local quadratic convergence of the algorithm.The experimental results demonstrate that the Lie algebra based method makes significant improvements in convergence speed,tracking stability and accuracy in comparison to Euclidean based algorithms.

Direct Model-Based Tracking of 3D Object Deformations in Depth and Color Video

The tracking of deformable objects using video data is a demanding research topic due to the inherent ambiguity problems, which can only be solved using additional assumptions about the deformation. Image feature points, commonly used to approach the deformation problem, only provide sparse information about the scene at hand. In this paper a tracking approach for deformable objects in color and depth video is introduced that does not rely on feature points or optical flow data but employs all the input image information available to find a suitable deformation for the data at hand. A versatile NURBS based deformation space is defined for arbitrary complex triangle meshes, decoupling the object surface complexity from the complexity of the deformation. An efficient optimization scheme is introduced that is able to calculate results in real-time (25 Hz). Extensive synthetic and real data tests of the algorithm and its features show the reliability of this approach.

Prediction Based Collaborative Trackers (PCT): A Robust and Accurate Approach Toward 3D Medical Object Tracking

Robust and fast 3D tracking of deformable objects, such as heart, is a challenging task because of the relatively low image contrast and speed requirement. Many existing 2D algorithms might not be directly applied on the 3D tracking problem. The 3D tracking performance is limited due to dramatically increased data size, landmarks ambiguity, signal drop-out or complex nonrigid deformation. In this paper, we present a robust, fast, and accurate 3D tracking algorithm: prediction based collaborative trackers (PCT). A novel one-step forward prediction is introduced to generate the motion prior using motion manifold learning. Collaborative trackers are introduced to achieve both temporal consistency and failure recovery. Compared with tracking by detection and 3D optical flow, PCT provides the best results. The new tracking algorithm is completely automatic and computationally efficient. It requires less than 1.5 s to process a 3D volume which contains millions of voxels. In order to demonstrate the generality of PCT, the tracker is fully tested on three large clinical datasets for three 3D heart tracking problems with two different imaging modalities: endocardium tracking of the left ventricle (67 sequences, 1134 3D volumetric echocardiography data), dense tracking in the myocardial regions between the epicardium and endocardium of the left ventricle (503 sequences, roughly 9000 3D volumetric echocardiography data), and whole heart four chambers tracking (20 sequences, 200 cardiac 3D volumetric CT data). Our datasets are much larger than most studies reported in the literature and we achieve very accurate tracking results compared with human experts' annotations and recent literature.

パーティクルフィルタと適応ベクトル量子化による柔軟変形物体の追跡と特徴量抽出

パーティクルフィルタと適応ベクトル量子化による柔軟変形物体の追跡と特徴量抽出

Robust unsupervised extraction of vocal tract variables from midsagittal real-time magnetic resonance image sequences using region segmentation

The tracking of deformable objects in image sequences has been a topic of intensive research for many years, and many application-specific solutions have been proposed. In this work, we describe a method that was developed to robustly track the tissue structures of the human vocal tract in midsagittal, real-time magnetic resonance (MR) images. The goal of the algorithm is to fully automatically extract the vocal tract outline, the position of the articulators, and the tract variables to facilitate the study of the shaping of the vocal tract during speech production. The algorithm is unsupervised and requires only a one-time initialization step for a particular subject. Importantly, the tracking algorithm operates on the spatial frequency domain representation of the underlying images, and it is hence specifically fit to the data produced in the MR imaging process. The proposed method carries out a multiregion segmentation of the individual MR images using an anatomically informed model of the vocal tract whose fit to the observed image data is hierarchically optimized using an anatomically informed gradient descent procedure. The mathematical key components of the algorithm are the closed-form solution of the two-dimensional Fourier transform of a polygonal shape function and the design of alternative gradient descent flows for the iterative solution of an overdetermined nonlinear least squares optimization problem. Various examples of segmented real-time MR images and a summary of open challenges will be presented. [Work supported by NIH.]

A Deformable Object Tracking Algorithm Based on the Boundary Element Method that is Robust to Occlusions and Spurious Edges

The manipulation of deformable objects is an important problem in robotics and arises in many applications including biomanipulation, microassembly, and robotic surgery. For some applications, the robotic manipulator itself may be deformable. Vision-based deformable object tracking can provide feedback for these applications. Computer vision is a logical sensing choice for tracking deformable objects because the large amount of data that is collected by a vision system allows many points within the deformable object to be tracked simultaneously. This article introduces a template based deformable object tracking algorithm, based on the boundary element method, that is able to track a wide range of deformable objects. The robustness of this algorithm to occlusions and to spurious edges in the source image is also demonstrated. A robust error measure is used to handle the problem of occlusion and an improved edge detector based on the Canny edge operator is used to suppress spurious edges. This article concludes by quantifying the performance increase provided by the robust error measure and the robust edge detector. The performance of the algorithm is also demonstrated through the tracking of a sequence of cardiac MRI images.

Dynamical statistical shape priors for level set-based tracking

In recent years, researchers have proposed introducing statistical shape knowledge into level set-based segmentation methods in order to cope with insufficient low-level information. While these priors were shown to drastically improve the segmentation of familiar objects, so far the focus has been on statistical shape priors which are static in time. Yet, in the context of tracking deformable objects, it is clear that certain silhouettes (such as those of a walking person) may become more or less likely over time. In this paper, we tackle the challenge of learning dynamical statistical models for implicitly represented shapes. We show how these can be integrated as dynamical shape priors in a Bayesian framework for level set-based image sequence segmentation. We assess the effect of such shape priors "with memory" on the tracking of familiar deformable objects in the presence of noise and occlusion. We show comparisons between dynamical and static shape priors, between models of pure deformation and joint models of deformation and transformation, and we quantitatively evaluate the segmentation accuracy as a function of the noise level and of the camera frame rate. Our experiments demonstrate that level set-based segmentation and tracking can be strongly improved by exploiting the temporal correlations among consecutive silhouettes which characterize deforming shapes.

ACTIVE CONTOUR MODEL USING WAVELET MODULUS FOR OBJECT SEGMENTATION AND TRACKING IN VIDEO SEQUENCES

We propose an integrated wavelet-based framework of the active contour model (snake) for segmentation and motion tracking of deformable objects in video sequences. The input image frame is represented by means of wavelet transform. First, the moduli of wavelet transform coefficients are used in a multi-resolution motion estimation process to find the initial snake contour in the current frame. The presented multi-resolution motion estimation methods allow a larger movement of the tracked object than does traditional image-based motion estimation. Secondly, the wavelet transform modulus at each scale is considered in the energy function of the snake model. The snake computation is based on a coarse-to-fine scale continuation method. Application of the proposed methods to biological cell tracking is demonstrated in experiments.