Visual Tracking Applications Research Articles

Retraction Note: Application of visual tracking based on infrared target imaging in aerobics action evaluation system

Retraction Note: Application of visual tracking based on infrared target imaging in aerobics action evaluation system

Green synthesis of a thermo/photochromic doped cellulose polymer: a biocompatible film for potential application in cold chain visual tracking

A novel photochromic compound has been embedded in a biocompatible cellulose film, creating a composite material that functions as a reusable colorimetric sensor with potential applications in cold chain monitoring.

Application of visual tracking based on infrared target imaging in aerobics action evaluation system

Application of visual tracking based on infrared target imaging in aerobics action evaluation system

Estimation of control area in badminton doubles with pose information from top and back view drone videos

The application of visual tracking to the performance analysis of sports players in dynamic competitions is vital for effective coaching. In doubles matches, coordinated positioning is crucial for maintaining control of the court and minimizing opponents’ scoring opportunities. The analysis of such teamwork plays a vital role in understanding the dynamics of the game. However, previous studies have primarily focused on analyzing and assessing singles players without considering occlusion in broadcast videos. These studies have relied on discrete representations, which involve the analysis and representation of specific actions (e.g., strokes) or events that occur during the game while overlooking the meaningful spatial distribution. In this work, we present the first annotated drone dataset from top and back views in badminton doubles and propose a framework to estimate the control area probability map, which can be used to evaluate teamwork performance. We present an efficient framework of deep neural networks that enables the calculation of full probability surfaces. This framework utilizes the embedding of a Gaussian mixture map of players’ positions and employs graph convolution on their poses. In the experiment, we verify our approach by comparing various baselines and discovering the correlations between the score and control area. Additionally, we propose a practical application for assessing optimal positioning to provide instructions during a game. Our approach offers both visual and quantitative evaluations of players’ movements, thereby providing valuable insights into doubles teamwork. The dataset and related project code is available at https://github.com/Ning-D/Drone_BD_ControlArea

Stability Analysis of 3D UAV Gimbal using Flexible Body

Unmanned aerial vehicle (UAV) application in visual object tracking requires a gimbal to stabilize the camera in following the object’s movement. External disturbance and gimbal stability are the issues in this application. This study aims to analyze disturbance effects and study the stress and modal analysis on UAV gimbal using flexible body concepts. The three-dimensional (3D) gimbal is modeled using the RecurDyn software consisting of 3 arms and a camera. Each of the arms is connected using a revolute joint and a rotational force to represent a motor. The considered disturbances are step, pulse, ramp, and sine wave input. The PID controller is used to stabilize the gimbal arm from the gravity of the camera and external disturbance. The result shows that the PID controller is robust to step, pulse, and ramp disturbance, but not to the sin wave disturbance. In addition, the second arm of the gimbal is the most stressed component and is prone to vibration.

Machine learning (deep learning) and visualization assisted ferrite content prediction in austenitic stainless steel

The ferrite content of 202 austenitic stainless-steel welds was predicted based on Machine Learning (ML), and the data set was analysed by manifold learning, feature engineering, and other methods. Multiple ML methods such as K-Nearest Neighbors (KNN), Support Vector Regression (SVR), Ensemble Learning (EL)、Multiple model Merging (MM), and Artificial Neural Network (ANN) performed algorithm model building and model tuning on the data set and evaluated the prediction performance of the model. The ML visualization and ML interpretation tools are used to demonstrate in detail the comparison of the prediction performance of ferrite number (FN) of various ML models and the effect of welding parameters on FN prediction. SVR, Gradient Boost, Xgboost, Catboost, Voting, Stacking, and MLP regression models demonstrate precision prediction and generalization. The coefficient of determination ( R 2 ) of the MLP model was 0.956 on the test set, which showed generalization and robustness. The visual data analysis and ML sample tracking methods are used to optimize the model. An ANN model with high prediction accuracy was proposed, and the R 2 of the test set was 0.972. This paper reveals the comprehensive application of ML-assisted material design and data sample visual tracking in the field of materials. • PyTorch deep learning framework to improve prediction accuracy. • Welding parameters are used to analyse welding parameters’ effect on ferrite content in welds. • 13 kinds of algorithms are selected for modelling and training. • Visualization tools explain the model prediction and feature importance. • The determination coefficient of the neural network model is 0.972.

SiamDA: distribution-aware Siamese network for visual tracking

Siamese-based trackers have achieved remarkable performance and widespread applications in visual tracking. However, most of the existing Siamese trackers are usually restricted by the difficulty of handling the misalignment between target localization and state estimation. To address the misalignment issue, we propose an efficient Siamese distribution-aware (DA) anchor-free (SiamDA) network for object tracking. First, the bounding box of regression targets is modeled by a flexible general distribution, avoiding the ambiguity of traditional rectangle representation in some complex scenes, such as occlusion and background clutter. On this basis, to eliminate the misalignment issue, a DA module is designed to connect the target localization and state estimation. In addition, the Pseudo-Intersection over Union sample assignment rule provides our anchor-free tracker with high-quality samples effectively during the training process, making it possible to further improve the tracking performance. Extensive experiments on official visual tracking benchmarks demonstrate the effectiveness and efficiency of SiamDA.

Visual object tracking using Fourier domain phase information

In this article, phase of the Fourier transform (FT), which has observed to be a crucial component in image representation, is utilized for visual target tracking. The main aim of the proposed scheme is to reduce the computational complexity of cross-correlation-based matching frameworks. Normalized cross-correlation (NCC) function-based object tracker is converted to a phase minimization problem under the following assumption: In visual object tracking applications, if the frame rate is high, the moving object can be considered to have translational shifts in image domain in a small time window. Since the proposed tracking framework works in the Fourier domain, the translational shifts in the image space are converted to phase variations in the Fourier domain due to the “translational invariance” property of the FT. The proposed algorithm estimates the spatial target position based on the phase information of the target region. The proposed framework uses the \(\ell _1\)-norm and provides a computationally efficient solution for the tracking problem. Experimental studies indicate that the proposed phase-based technique obtain comparable results with baseline tracking algorithms which are computationally more complex.

Identification of Technology Diffusion by Citation and Main Paths Analysis: The Possibility of Measuring Open Innovation

This study collected literature on augmented reality (AR) from academic and patent databases to plot the historic development trajectory of AR and forecast its future research and development trends. A total of 3193 and 13,629 papers were collected from academic and patent databases, respectively. First, a network was established using references from the academic literature; main path analysis was conducted on this reference network to plot the overall development trajectory. Subsequent cluster and word cloud analyses revealed the following five major groups of AR research topics: AR surgical navigation applications, AR education applications, AR applications in manufacturing, AR applications in architecture, and AR applications in visual tracking. Subsequently, the relationships between the overall development trajectory and the five AR research topics were compared. Next, the title and abstract of AR-related academic and patent papers were subjected to text mining to identify keywords with a high frequency of occurrence. The results can provide a reference for industry, government, and academia when planning future development strategies for the AR field. This research adopted an integrated analysis procedure to plot the trajectory of AR technology development and applications successfully and effectively, predict future patent research and development directions and produce technological forecasts.

물체 추적을 위한 딥 러닝 기반의 앙상블 모델 연구

컴퓨터 비전 분야에서 물체 추적은 비디오 스트림으로부터 입력되는 시각적 정보로부터 타겟물체의 위상 변화를 예측하는 분야이며, 보안 및 군사기술이 요구하는 응용분야에서 중요하게 적용될 수 있는 기술이다. 최근의 딥 러닝 기반의 물체 추적 기술들은 검출기 기반 접근법(Tracking-by-Detection) 및 템플릿 대응 기반 접근법(template matching) 등을 통해 그 성능을 크게 향상시켰지만 접근방식에 따라 그 장단점이 분명하였다. 본 논문에서는 위의 두 접근방법을 응용하는 앙상블 모델 연구를 통해 단점을 보완하는 알고리즘을 제안한다. 제안되는 앙상블 알고리즘은 최근의 저명한 추적 알고리즘을 위한 벤치마크, OTB100, UAV123, LaSOT에서 모두 높은 성능향상을 보고한다.

Fast Visual Tracking Using Spatial Temporal Background Context Learning

Visual Tracking by now has gained much provenience among researchers in recent years due to its vast variety of applications that occur in daily life. Various applications of visual tracking include counting of cars on a high way, analyzing the crowd intensity in a concert or a football ground or a surveillance camera tracking a single person to track its movements. Various techniques have been proposed and implemented in this research domain where researchers have analyzed various parameters. Still this area has a lot to offer. There are two common approaches that are currently deployed in visual tracking. One is discriminative tracking and the other one is generative tracking. Discriminative tracking requires a pre-trained model that requires the learning of the data and solves the object recognition as a binary classification problem. On the other hand, generative model in tracking makes use of the previous states so that next state can be predicted. In this paper, a novel tacking based on generative tracking method is proposed called as Illumination Inavariant Spatio Temporal Tracker (IISTC). The proposed technique takes into account of the nearby surrounding regions and performs context learning so that the state of the object under consideration and its surrounding regions can be estimated in the next frame. The learning model is deployed both in the spatial domain as well as the temporal domain. Spatial domain part of the tracker takes into consideration the nearby pixels in a frame while the temporal model takes account of the possible change of object location. The proposed tracker was tested on a set of 50 images against other state of the art four trackers. Experimental results reveal that our proposed tracker performs reasonably well as compared with other trackers. The proposed visual tracker is both efficiently with respect to computation power as well as accuracy. The proposed tracker takes only 4 fast Fourier transform computations thus making it reasonably faster. The proposed trackers perform exceptionally well when there is a sudden change in back ground illumination.

A novel real-time object tracking based on kernelized correlation filter with self-adaptive scale computation in combination with color attribution

Recently, object tracking model based on correlation filter has achieved great success. However, most of these methods relying on correlation filter train many classifiers individually, and fail to cope with significant appearance and scale change of object and complex challenging environment. How to accurately and robustly compute object scale relative to template in visual object tracking application is a challenge topic at home and abroad. Many existing tracking algorithms are not very good at big scale variation in complex video. In order to solve the problem, a novel and effective scale computation algorithm based on tracking in detection is proposed in paper. The tracking object is divided into four parts, and compute their scale factors, then get the maximum response location by adopting kernelized correlation filter based on color attribution. Specially, aimed at the difficulty of object location, weighted coefficients are used to remove the abnormal matching points and the desired training output from common classifier is transformed into other forms. Simulation experiment result shows our proposed algorithm outperform others algorithms in color video with big scale variation.

Mining Spatial-Temporal Similarity for Visual Tracking.

Correlation filter (CF) is a critical technique to improve accuracy and speed in the field of visual object tracking. Despite being studied extensively, most existing CF methods suffer from failing to make the most of the inherent spatial-temporal prior of videos. To address this limitation, as consecutive frames are eminently resemble in most videos, we investigate a novel scheme to predict targets' future state by exploiting previous observations. Specifically, in this paper, we propose a prediction based CF tracking framework by learning the spatial-temporal similarity of consecutive frames for sample managing, template regularization, and training response pre-weighting. We model the learning problem theoretically as a novel objective and provide effective optimization algorithms to solve the learning task. In addition, we implement two CF trackers with different features. Extensive experiments are conducted on three popular benchmarks to validate our scheme. The encouraging results demonstrate that the proposed scheme can significantly boost the accuracy of CF tracking, and the two trackers achieve competitive performances against state-of-the-art trackers. We finally present a comprehensive analysis on the efficacy of our proposed method and the efficiency of our trackers to facilitate real-world visual tracking applications.

Fast and sub-pixel precision target tracking algorithm for intelligent dual-resolution camera

The intelligent dual-resolution camera can provide large imaging field and high-resolution for the parts in which we are interested simultaneously. It has important applications in visual tracking. Though many trackers show great robustness on recent benchmarks, few of them make high precision and run in real-time, which is harmful to practical applications. In this paper, we propose a fast and sub-pixel precision tracker. It uses the time-shift property of Fourier transform to convert the displacement of target in space domain into the period of the response in Fourier domain. Then an improved Hough transform is introduced to measure this period, so that the displacement with sub-pixel precision can be calculated. Furthermore, a method obtaining benchmarks utilizing the intelligent dual-resolution camera is proposed to verify the high-precision of our tracker. Many experiments have been done to compare the proposed tracker with some state-of-the-art sub-pixel precision trackers, and the results have shown that the proposed tracker outperform the best tracker by 15.4% in average median distance precision, while feasible for real-time tracking with the speed faster than 80 fps. What’s more, the proposed tracker have been evaluated on the dual-resolution camera, the results show that it can make the observation of targets in the high-resolution image more complete.

Scalable implementation of particle filter-based visual object tracking on network-on-chip (NoC)

Particle filter algorithms have been successfully used in various visual object tracking applications. They handle non-linear model and non-Gaussian noise, but are computationally demanding. In this paper, we propose a scalable implementation of particle filter algorithm for visual object tracking, using scalable interconnect such as network-on-chip on an FPGA platform. Here, several processing elements execute parallelly to handle large number of particles. We propose two designs and implementations, with one optimized for speed and other optimized for area. These implementations can easily support different image sizes, object sizes, and number of particles, without modifying the complete architecture. Multi-target tracking is also demonstrated for four objects. We validated the particle filter-based visual tracking with video feed from a Petalinux-based system. With image size of $$320\times 240$$ , frame rates of 348 fps and 310 fps were achieved for single-object tracking of size $$17\times 17$$ and $$33\times 33$$ pixels, respectively, with a reasonable low-power consumption of 1.7 mW/fps on Zynq XC7Z020 (Zedboard) with an operating frequency of 69 MHz. This makes our implementation a good candidate for low-power, visual object tracking using FPGA, especially in low-power, smart camera applications.

Visual object tracking based on sequential learning of SVM parameter

In this paper, a training algorithm is proposed to online (sequentially) learn the SVM based parameter. The proposed online learning method is used to construct discriminative classifier for visual object tracking application, where new examples are available in each successive frame of a video. The iterative method is based on maximizing the magnitude of sum of projection values of a positive example and a negative example which are closest to the hyperplane formed by parameter to be updated. In the proposed training framework, even if there is non-accurate labeling of the training examples (which are received online), it is possible to learn the parameter that ensures maximum value from the classification function. The learned parameter along with some examples which are nearest to hyperplane are used for the construction of object likelihood model. Using likelihood model, tracking instance most similar to the target is selected, where the target candidates are generated using particle filter framework. An object representation is also learned based on sparse DCT coefficients. This representation contains the basic structure of the target appearance. The proposed object tracking method performs better than state-of-the-art trackers in a number of challenging video sequences. When high dimension feature vectors are used, instead of simple raw pixels based feature vectors, to represent the training examples, the performance of the object tracking is better in a number of video sequences even without integrating sparse DCT coefficient based object representation as an additional model.

Robust spatio-temporal context for infrared target tracking

Target tracking is one of the most important and active research areas in the field of computer vision. In this paper, we address the problem of tracking an object that is completely occluded in a video. The proposed robust spatio-temporal context (RSTC) method, inspired by the spatio-temporal context method, uses spatio-temporal context information to establish an early warning mechanism. The core technology of the early warning mechanism is to monitor the fluctuation of the relative change rate between two adjacent frames. When a target is completely occluded, the algorithm estimates the target’s location using accurate motion information saved during the early warning. Finally, the algorithm captures the target after the target reappears. Experimental results show that the proposed algorithm is very robust and efficient when used in visual tracking applications.

视觉跟踪算法综述

视觉跟踪是计算机视觉领域多年来的研究热点问题之一，随着技术的不断发展，视觉跟踪应用范围逐渐扩大。本文首先阐述了跟踪过程中的难点和视觉跟踪的应用，以及跟踪方法的两大分类。以时间顺序对跟踪算法的发展历史及现状进行介绍，并比较分析其优缺点，最后探讨了视觉跟踪算法未来的发展趋势。 Visual tracking is one of the hot topics in the field of computer vision. Over the years, the application of visual tracking has been gradually expanded with the continuous development of technology. First, the difficulties of tracking process and the application of visual tracking are introduced in this paper. Then, tracking methods are classified into two major classifications. The history of development and current research status of the tracking algorithm are introduced in time sequence, and the advantages and disadvantages are analyzed. Finally, the future trend of visual tracking is presented.

Deep convolutional neural networks for thermal infrared object tracking

Unlike the visual object tracking, thermal infrared object tracking can track a target object in total darkness. Therefore, it has broad applications, such as in rescue and video surveillance at night. However, there are few studies in this field mainly because thermal infrared images have several unwanted attributes, which make it difficult to obtain the discriminative features of the target. Considering the powerful representational ability of convolutional neural networks and their successful application in visual tracking, we transfer the pre-trained convolutional neural networks based on visible images to thermal infrared tracking. We observe that the features from the fully-connected layer are not suitable for thermal infrared tracking due to the lack of spatial information of the target, while the features from the convolution layers are. Besides, the features from a single convolution layer are not robust to various challenges. Based on this observation, we propose a correlation filter based ensemble tracker with multi-layer convolutional features for thermal infrared tracking (MCFTS). Firstly, we use pre-trained convolutional neural networks to extract the features of the multiple convolution layers of the thermal infrared target. Then, a correlation filter is used to construct multiple weak trackers with the corresponding convolution layer features. These weak trackers give the response maps of the target’s location. Finally, we propose an ensemble method that coalesces these response maps to get a stronger one. Furthermore, a simple but effective scale estimation strategy is exploited to boost the tracking accuracy. To evaluate the performance of the proposed tracker, we carry out experiments on two thermal infrared tracking benchmarks: VOT-TIR 2015 and VOT-TIR 2016. The experimental results demonstrate that our tracker is effective and achieves promising performance.

A novel foreground region analysis using NCP-DBP texture pattern for robust visual tracking

Robust visual tracking is the important stage in the computer vision applications such as robotics, man-free control systems, and the visual surveillance. Accurate motion states estimation and the target representation in visual tracking system are based on the appearances of the target. The factor affects the learning of target representation is the accumulated error due to the pose, illumination changes, and the uneven background. The presence of dynamic background and the shadowing effects causes the visual drift and destructive information. Besides, the misclassification of target region induces the false detection of moving objects. The K-means and Fuzzy-C-means clustering algorithms are available to segment the foreground/background and suppress the shadow region on the basis of the non-changing background of the surveillance area. This paper proposes the novel background normalization technique with textural pattern analysis to suppress the shadow region. The Neighborhood Chain Prediction (NCP) algorithm is used to cluster the uneven background and the Differential Boundary Pattern (DBP) extracts the texture of the video frame to suppress the shadow pixels present in the frame. The lower intensity estimation and the prediction of the area around the lower intensity in proposed work enhance the pixels for shadow removal. The shadow-free frame split up into several grids and the histograms of features are extracted from the grid formatted frame. Finally, the Machine Level Classification (MLC) finds the matching grid corresponds to the tracking region and provides the binary labeling to separate the background and foreground. The proposed DBP-based visual tracking system is high robustness over the sudden illumination changes and the dynamic background due to the texture pattern analysis. The comparison of proposed NCP-DBP combination with the existing segmentation techniques regarding the accuracy, precision, recall, F-measure, success and error rate assured the effectiveness in visual tracking applications.