Deep Learning Trackers Research Articles

Si-MSPDNet: A multiscale Siamese network with parallel partial decoders for the 3-D measurement of spines in 3D ultrasonic images.

Early screening and frequent monitoring effectively decrease the risk of severe scoliosis, but radiation exposure is a consequence of traditional radiograph examinations. Additionally, traditional X-ray images on the coronal or sagittal plane have difficulty providing three-dimensional (3-D) information on spinal deformities. The Scolioscan system provides an innovative 3-D spine imaging approach via ultrasonic scanning, and its feasibility has been demonstrated in numerous studies. In this paper, to further examine the potential of spinal ultrasonic data for describing 3-D spinal deformities, we propose a novel deep-learning tracker named Si-MSPDNet for extracting widely employed landmarks (spinous process (SP)) in ultrasonic images of spines and establish a 3-D spinal profile to measure 3-D spinal deformities. Si-MSPDNet has a Siamese architecture. First, we employ two efficient two-stage encoders to extract features from the uncropped ultrasonic image and the patch centered on the SP cut from the image. Then, a fusion block is designed to strengthen the communication between encoded features and further refine them from channel and spatial perspectives. The SP is a very small target in ultrasonic images, so its representation is weak in the highest-level feature maps. To overcome this, we ignore the highest-level feature maps and introduce parallel partial decoders to localize the SP. The correlation evaluation in the traditional Siamese network is also expanded to multiple scales to enhance cooperation. Furthermore, we propose a binary guided mask based on vertebral anatomical prior knowledge, which can further improve the performance of our tracker by highlighting the potential region with SP. The binary-guided mask is also utilized for fully automatic initialization in tracking. We collected spinal ultrasonic data and corresponding radiographs on the coronal and sagittal planes from 150 patients to evaluate the tracking precision of Si-MSPDNet and the performance of the generated 3-D spinal profile. Experimental results revealed that our tracker achieved a tracking success rate of 100% and a mean IoU of 0.882, outperforming some commonly used tracking and real-time detection models. Furthermore, a high correlation existed on both the coronal and sagittal planes between our projected spinal curve and that extracted from the spinal annotation in X-ray images. The correlation between the tracking results of the SP and their ground truths on other projected planes was also satisfactory. More importantly, the difference in mean curvatures was slight on all projected planes between tracking results and ground truths. Thus, this study effectively demonstrates the promising potential of our 3-D spinal profile extraction method for the 3-D measurement of spinal deformities using 3-D ultrasound data.

Drift-Proof Tracking With Deep Reinforcement Learning

Object tracking is an essential and challenging sub-domain in the field of computer vision owing to its wide range of applications and complexities of real-life situations. It has been studied extensively over the last decade, leading to the proposal of several tracking frameworks and approaches. Recently, the introduction of reinforcement learning and the Actor-Critic framework has effectively improved the tracking speed of deep learning trackers. However, most existing deep reinforcement learning trackers experience a slight performance degradation mainly owing to the drift issues. Drifts pose a threat to the tracking performance, which may lead to losing the tracked target. Herein, we propose a drift-proof tracker with deep reinforcement learning that aims to improve the tracking performance by counteracting drifts while maintaining its real-time advantage. We utilize a reward function with the Distance-IoU (DIoU) metric to guide the reinforcement learning to alleviate the drifts caused by the trained model. Furthermore, double negative samples (hard negative and drift samples) are constructed in tracking for network initialization, which is followed by calculating the loss by a small error-friendly loss function. Therefore, our tracker can better discriminate between the positive and negative samples and correct the predicted bounding boxes when the drift occurs. Meanwhile, a generative adversarial network is adopted for positive sample augmentation. Extensive experimental results on multiple popular benchmarks show that our algorithm effectively reduces the occurrences of drift and boosts the tracking performance, compared to those of other state-of-the-art trackers.

Deep Reinforcement Learning for Small Bowel Path Tracking using Different Types of Annotations.

Small bowel path tracking is a challenging problem considering its many folds and contact along its course. For the same reason, it is very costly to achieve the ground-truth (GT) path of the small bowel in 3D. In this work, we propose to train a deep reinforcement learning tracker using datasets with different types of annotations. Specifically, we utilize CT scans that have only GT small bowel segmentation as well as ones with the GT path. It is enabled by designing a unique environment that is compatible for both, including a reward definable even without the GT path. The performed experiments proved the validity of the proposed method. The proposed method holds a high degree of usability in this problem by being able to utilize the scans with weak annotations, and thus by possibly reducing the required annotation cost.

Rank-Based Verification for Long-Term Face Tracking in Crowded Scenes

Most current multi-object trackers focus on short-term tracking, and are based on deep and complex systems that often cannot operate in real-time, making them impractical for video-surveillance. In this paper we present a long-term, multi-face tracking architecture conceived for working in crowded contexts where faces are often the only visible part of a person. Our system benefits from advances in the fields of face detection and face recognition to achieve long-term tracking, and is particularly unconstrained to the motion and occlusions of people. It follows a tracking-by-detection approach, combining a fast short-term visual tracker with a novel online tracklet reconnection strategy grounded on rank-based face verification. The proposed rank-based constraint favours higher inter-class distance among tracklets, and reduces the propagation of errors due to wrong reconnections. Additionally, a correction module is included to correct past assignments with no extra computational cost. We present a series of experiments introducing novel specialized metrics for the evaluation of long-term tracking capabilities, and publicly release a video dataset with 10 manually annotated videos and a total length of 8' 54". Our findings validate the robustness of each of the proposed modules, and demonstrate that, in these challenging contexts, our approach yields up to 50% longer tracks than state-of-the-art deep learning trackers.

Accurate and robust tracking of rigid objects in real time

We present the shape model object tracker, which is accurate, robust, and real-time capable on a standard CPU. The tracker has a failure mode detection, is robust to nonlinear illumination changes, and can cope with occlusions. It uses subpixel-precise image edges to track roughly rigid objects with high accuracy and is virtually drift-free even for long sequences. Furthermore, it is inherently capable of object re-detection when tracking fails. To evaluate the accuracy, robustness, and efficiency of the tracker precisely, we present a challenging new tracking dataset with pixel-precise ground truth. The precise ground-truth labels are created automatically from the photo-realistic synthetic VIPER dataset. The tracker is thoroughly evaluated against the state of the art through a number of qualitative and quantitative experiments. It is able to perform on par with the current state-of-the-art deep-learning trackers, but is at least 45 times faster, even without using a GPU. The efficiency and low memory consumption of the tracker are validated in further experiments that are conducted on an embedded device.

Deep visual nerve tracking in ultrasound images

Ultrasound-guided regional anesthesia (UGRA) becomes a standard procedure in surgical operations and pain management, offers the advantages of nerve localization, and provides region of interest anatomical structure visualization. Nerve tracking presents a crucial step for practicing UGRA and it is useful and important to develop a tool to facilitate this step. However, nerve tracking is a very challenging task that anesthetists can encounter due to the noise, artifacts, and nerve structure variability. Deep-learning has shown outstanding performances in computer vision task including tracking. Many deep-learning trackers have been proposed, where their performance depends on the application. While no deep-learning study exists for tracking the nerves in ultrasound images, this paper explores thirteen most recent deep-learning trackers for nerve tracking and presents a comparative study for the best deep-learning trackers on different types of nerves in ultrasound images. We evaluate the performance of the trackers in terms of accuracy, consistency, time complexity, and handling different nerve situations, such as disappearance and losing shape information. Through the experimentation, certain conclusions were noted on deep learning trackers performance. Overall, deep-learning trackers provide good performance and show a comparative performance for tracking different kinds of nerves in ultrasound images.

Multi-object tracking with discriminant correlation filter based deep learning tracker

Multi-object tracking with discriminant correlation filter based deep learning tracker

Deep Learning Trackers Review and Challenge

Deep Learning Trackers Review and Challenge

Enhanced wavelet convolutional neural networks for visual tracking

Visual tracking based on deep learning technique is a very attractive research topic recently in the computer vision field. Deep convolutional neural networks (CNNs) are inherently limited to low spatial resolution, due to the max pooling process in the modules, and they are constrained by the high computation burden. We present a pretrained deep learning network architecture to the task of visual tracking, by introducing a wavelet representation in the network and a two-stage fine-tuning for learning appearance features, which improves the original deep learning tracker. Moreover, a loss layer based on Bayesian theorem is adopted to compute maximum classifier score, instead of the softmax loss layer, which can enhance the success rate. In addition, the idea of wavelet pooling helps perform feature dimension reduction. In addition, wavelet representation helps to reduce the computation time greatly. Compared with the original algorithm and other state-of-the-art methods, the proposed tracking method shows excellent performances on test baseline dataset. As our optimized spectrum CNN can extract a compact and efficient representation of objects, it can be further applied to multiple objects tracking.

Robust visual tracking via combinative deep learning

Object tracking is a hot topic in computer vision. In recent years, a large number of trackers has been proposed, in which the deep learning tracker has achieved excellent performance. The real-time capability of the deep learning tracker is not good enough due to the high-complexity of the network structures. This paper proposed an innovative tracking method to solve this problem. There are three important differences between this tracker and the other deep learning trackers. Firstly, the overcomplete basis in the deep learning tracker results in heavy computational cost. In order to reduce the complexity of the network, fewer units are used in the first hidden layer to replace the overcomplete basis. Secondly, a training method combining two observation models is used in the tracking process. The denoising automatic encoder is used in the first layer and the backpropagation is used in the other layers. This can avoid the diffusion of gradients which is caused by BP and adapt to the change of the targets easier. Thirdly, this tracker using adaptive particle filter to track targets. The number of particles is dynamic changes in tracking process. In this paper, we use different kinds of unlabelled datum to train network and initialize observation model. The observation model uses the samples collected in the tracking to adjust dynamically so as to adapt to the target appearance and complex environment. Compared with the existing methods, the results of experiments in different video sequences show that this tracker has a higher speed and the similar accuracy compared.

Visual Tracking with Online Incremental Deep Learning and Particle Filter

To solve the problem of tracking the trajectory of a moving object and learning a deep compact image representation in the complex environment, a novel robust incremental deep learning tracker is presented under the particle filter framework. The incremental deep classification neural network was composed of stacked denoising autoencoder, incremental feature learning and support vector machine to achieve the featureextracting and classification of particle set. Deep learning is successfully taken to express the image representations obtained effectively. Unsupervised feature learning is used to learn generic image features and transfer learning transforms knowledge from offline training to the online tracking process. The incremental feature learning was consisted of adding features and merging features to online learn compact feature set. Linear support vector machine increases the discretion for target with similar appearance and is further tuned to adapt to appearance changes of the moving object. Compared with the state-of-the-art trackers in the complex environment, the results of experiments on variant challenging image sequences show that incremental deep learning tracker solves the problem of existent trackers more efficiently, it has better robust and more accurate, especially for occlusions, background clutter, illumination changes and appearance changes.

Regional deep learning model for visual tracking

Deep learning has been successfully applied to visual tracking due to its powerful feature learning characteristic. However, existing deep learning trackers rely on single observation model and focus on the holistic representation of the tracking object. When occlusion occurs, the trackers suffer from the contaminated features obtained in occluded areas. In this paper, we propose a regional deep learning tracker that observes the target by multiple sub-regions and each region is observed by a deep learning model. In particular, we devise a stable factor, modeled as a hidden variable of the Factorial Hidden Markov Model, to characterize the stability of these sub-models. The stability indicator not only provides a confidence degree for the response score of each model during inference stage, but also determines the online training criteria for each deep learning model. This online training strategy enables the tracker to achieve more accurate local features compared with those fixed training trackers. In addition, to improve the computational efficiency, we exploit the structurized response property of the customized deep learning model to approximate the final tracking results by the weighted Gaussian Mixture Model under the particle filter framework. Qualitative and quantitative evaluations on the recent public benchmark dataset show that our approach outperforms most state-of-the-art trackers.