Optical Flow Tracking Method Research Articles

Video velocity measurement: a two-stage flow velocity prediction method based on deep learning

Abstract Due to the uncertainty in output caused by environmental changes, significant discrepancies are expected between the surface flow velocities predicted using deep learning methods and the instantaneous flow velocities. In this paper, a two-stage deep learning flow velocity measurement algorithm is proposed. During the external calibration process, the upper and lower frames of the recorded water flow video are cyclically traversed to acquire predicted flow velocity values using the deep learning velocity measurement algorithm. Meanwhile, the pixel displacement is obtained using the sparse optical flow tracking method and then post-processed to derive the velocity calibration value and pixel calibration value. During the detection process, the deep learning-predicted flow velocity is internally calibrated using the velocity calibration value and the pixel calibration value to adapt to changes in water flows. Compared with the pre-improved algorithm, the method achieves the minimum root mean square error in five different flow velocity videos and maintains high accuracy when the flow velocity changes rapidly. The obtained results are very promising and can help improve the reliability of video flow rate assessment algorithms.

A local optical flow eye-tracking method for depression detection

Aiming at the problem that depression detection is easily affected by subjective factors such as patient willingness, self-awareness, and doctor’s level, as well as inconvenient detection and high cost, a universal detection method with low cost and convenient deployment is proposed. A local optical flow tracking method is adopted, which is realized by eye region detection and optical flow estimation. Then, three experiments are designed to verify the accuracy of the method and the effectiveness of eye movement tracking. The formula for calculating the depression state is proposed, and the eye movement of one normal participant’s elliptical trajectory of the left and right pupils in the horizontal and vertical directions is 44.4805, 42.7013, 29.3571, and 31.1364 mm, respectively. To verify the effectiveness of the proposed method, three example points are selected. Among them, the fixation duration at P1 is 1.87 s, the eye movement direction at P2 and P3 is 1.168°, counterclockwise around the X axis, and the number of fixation times is 46. Finally, 18 normal participants and four depression-positive participants are tested by the exploratory eye movement paradigm. The results show that the accuracy rate of identifying depression patients using the NEF metric method can reach ∼81.8%, which can help quantify the detection of depression and provide the possibility for professionals to remotely diagnose individuals with depression.

A Real-Time Stereo Visual-Inertial SLAM System Based on Point-and-Line Features

Simultaneous localization and mapping (SLAM) is widely used in various fields, such as unmanned driving, robotics, and VR. The SLAM system with multiple landmarks is also a research hotspot currently. In this study, a real-time stereo visual-inertial SLAM system based on point-and-line features is proposed; the system is studied based on VINS-fusion. It improves the front-end process of VINS fusion. First, we proposed an IMU-assisted hierarchical grid optical flow tracking method that can more accurately and quickly tracks points between frames. Second, we add line features on the basis of existing point features. To match line features in real time, we only select the best line features to participate in optimization. We combine line segments by geometric relation to reduce the line segment splitting and redundancy brought by LSD algorithm, and further predict line features by IMU-assisted optical flow tracking to achieve higher precision matching. In the back-end optimization process, we used a redundant structure to avoid the failure of stereo constraints in a highly dynamic environment. The proposed method outperformed the state-of-the-art methods (VINS-fusion and PL-VINS) on the EuRoC MAV dataset. The system also achieved good performance in a real environment.

EPLF-VINS: Real-Time Monocular Visual-Inertial SLAM With Efficient Point-Line Flow Features

This letter introduces an efficient visual-inertial simultaneous localization and mapping (SLAM) method using point and line features. Currently, point-based SLAM methods do not perform well in scenarios such as weak textures and motion blur. Many researchers have noticed the excellent properties of line features in space and have attempted to develop line-based SLAM systems. However, the vast computational effort of the line extraction and description matching process makes it challenging to guarantee the real-time performance of the whole SLAM system, and the incorrect line detection and matching limit the performance improvement of the SLAM system. In this letter, we improve the traditional line detection model by means of short-line fusion, line feature uniform distribution, and adaptive threshold extraction to obtain high-quality line features for constructing SLAM constraints. Based on the gray level invariance assumption and colinear constraint, we propose a line optical flow tracking method, which significantly improves the speed of line feature matching. In addition, a measurement model that is independent of line endpoints is presented for estimating line residuals. The experimental results show that our algorithm improves the efficiency of line feature detection and matching and localization accuracy.

Automatic Myotendinous Junction Identification In Ultrasound Images Based on Junction-based Template Measurements.

Tracking the myotendinous junction (MTJ) motion in consecutive ultrasound images is essential to assess muscle and tendon interaction and understand the mechanics' muscle-tendon unit and its pathological conditions during motion. However, the inherent speckle noises and ambiguous boundaries deter the reliable identification of MTJ, thus restricting their usage in human motion analysis. This study advances a fully automatic displacement measurement method for MTJ using prior shape knowledge on the Y-shape MTJ, precluding the influence of irregular and complicated hyperechoic structures in muscular ultrasound images. Our proposed method first adopts the junction candidate points using a combined measure of Hessian matrix and phase congruency, followed by a hierarchical clustering technique to refine the candidates approximating the position of the MTJ. Then, based on the prior knowledge of Y-shape MTJ, we finally identify the best matching junction points according to intensity distributions and directions of their branches using multiscale Gaussian templates and a Kalman filter. We evaluated our proposed method using the ultrasound scans of the gastrocnemius from 8 young, healthy volunteers. Our results present more consistent with the manual method in the MTJ tracking method than existing optical flow tracking methods, suggesting its potential in facilitating muscle and tendon function examinations with in vivo ultrasound imaging.

Dense point cloud map construction based on stereo VINS for mobile vehicles

Mobile vehicles require accurate localization and dense mapping for motion planning. In this paper, we propose a dense map construction algorithm based on a light-and-fast stereo visual-inertial navigation system (VINS). A tightly coupled nonlinear optimization method is used to calculate the position of adjacent keyframes. An optical flow tracking method fused with IMU information and ring matching constraints is used to improve the matching accuracy and speed of the feature points. In addition, we obtain the pose and depth values using the semi-global block matching (SGBM) method, which are used as the initial values of the depth filter to update the depth image and improve the convergence speed. Then, we further use the Truncated Signed Distance Function (TSDF) method to construct the dense map. We compare our algorithm with state-of-the-art algorithms on the EuRoc dataset and then compare the estimated depth image using the proposed algorithm and the point cloud construction with the probabilistic monocular dense reconstruction (REMODE). The experiments show that the proposed algorithm can obtain more accurate localization than VINS and OKVIS, as well as a faster tracking speed, a better depth map, a lower convergence time for the estimated image and a lower number of updated frames than REMODE.

A Robust Optical Flow Tracking Method Based On Prediction Model for Visual-Inertial Odometry

Indirect method is widely used in the field of visual SLAM at present, and it can be divided into feature matching method and optical flow tracking method according to whether matching descriptors are needed in the tracking process. Compared to feature matching method, optical flow does not need to spend too much time on calculating and matching feature points, but it is easy to be affected by illumination changes and fast motion. In recent years, the visual-inertial fusion method improves the localization accuracy and robustness on the condition that the illumination is poor or image blur caused by fast motion. A robust optical flow tracking method based on prediction model for visual-inertial odometry is proposed in this letter. A new optical flow and brightness correction method based on a linear illumination model is used to deal with the large changes of illumination. We also proposed an initial optical flow prediction method with the pre-integration of IMU and reprojection model between image frames to make the optical flow more robust. The results of optical flow are also used to judge the image quality of specific areas to make sure the extracted feature points are more stable and suitable for tracking. Finally, the method is proved that can outperform other state-of-the-art VIO methods in experiments on the public benchmark even without loop closure.

Unlabeled facial expression capture method in virtual reality system based on big data

In view of the problems of high error rate and low efficiency in the traditional method of facial expression capture without markers, a method of facial expression capture without markers based on large amount of data is proposed. Haar feature is used to determine the initial position of human face, and active shape model is used to extract unmarked facial feature points. The extracted feature points and the generated triangle mesh are tracked by the optical flow tracking method. The displacement of the face feature points is used to promote the overall change of the mesh and complete the unmarked facial expression capture. The experimental results show that the error rate of this method is in the range of 1.2%-1.7%, the error rate is small, and it needs 20 s-34 s to capture facial expression, which is more practical and efficient.

Automatic Myotendinous Junction Tracking in Ultrasound Images with Phase-Based Segmentation.

Displacement of the myotendinous junction (MTJ) obtained by ultrasound imaging is crucial to quantify the interactive length changes of muscles and tendons for understanding the mechanics and pathological conditions of the muscle-tendon unit during motion. However, the lack of a reliable automatic measurement method restricts its application in human motion analysis. This paper presents an automated measurement of MTJ displacement using prior knowledge on tendinous tissues and MTJ, precluding the influence of nontendinous components on the estimation of MTJ displacement. It is based on the perception of tendinous features from musculoskeletal ultrasound images using Radon transform and thresholding methods, with information about the symmetric measures obtained from phase congruency. The displacement of MTJ is achieved by tracking manually marked points on tendinous tissues with the Lucas-Kanade optical flow algorithm applied over the segmented MTJ region. The performance of this method was evaluated on ultrasound images of the gastrocnemius obtained from 10 healthy subjects (26.0 ± 2.9 years of age). Waveform similarity between the manual and automatic measurements was assessed by calculating the overall similarity with the coefficient of multiple correlation (CMC). In vivo experiments demonstrated that MTJ tracking with the proposed method (CMC = 0.97 ± 0.02) was more consistent with the manual measurements than existing optical flow tracking methods (CMC = 0.79 ± 0.11). This study demonstrated that the proposed method was robust to the interference of nontendinous components, resulting in a more reliable measurement of MTJ displacement, which may facilitate further research and applications related to the architectural change of muscles and tendons.

Optical flow tracking method for vibration identification of out-of-plane vision

Vibration measurement based on computer vision has been extensively studied and considered as a wide-range, non-contact measurement method. In this paper, the principle of vibration measurement using out-of-plane vision has been investigated under conventional imaging condition. A measurement model for out-of-plane vision has also been demonstrated. Combined the out-of-plane vision measurement model with the optical flow motion estimation principle, a novel model of optical flow tracking method for vibration detection based on out-of-plane vision has been proposed. It enables the identification of vibration parameters without image feature extraction. Visual vibration detection experiment has been conducted with a cantilever beam and a motor cover. Experimental results have been rigorously compared with finite element simulation to verify the efficacy of the proposed method. It shows that this method can effectively identify vibration parameters of the structure without image feature extraction.

A multi-target trapping and tracking algorithm for Bactrocera Dorsalis based on cost model

Researching a robust, high precision, stable performance and applicable target tracking method is of important theoretical significance and practical value. Using computer vision technology to detect and track moving object in video image is a kind of advanced target counting method and it is a research hotspot. That is a complex estimation problem and still faces enormous challenges. Bactrocera dorsalis, which is widely distributed in the southern area of China, has become the most serious pest and need to be controlled frequently. In this study, a B. dorsalis tracking and counting system based on computer vision has been built to find out the activity patterns of B. dorsalis so as to provide a better prevention and control of the pest. For counting the number of B. dorsalis passed through the monitoring area in real time, this study has analyzed the advantages and disadvantages of the mainstream methods, such as inter-frame difference method, background-difference method, optical flow tracking method, corner detection method, Mean Shift tracking method and other classical target tracking algorithms. Based on these analyses, this study has proposed a cost model for tracking B. dorsalis, which can track all the trajectories of B. dorsalis in the monitoring area in real time. A multi-B. dorsalis tracking algorithm based on this cost model and a B. dorsalis counting algorithm have been realized. Finally, the result of the experiment shows that the algorithm can solve the overlap of B. dorsalis. Additionally, a field experiment of the tracking algorithm has been carried out for 5days in Guangzhou Yangtao Park. The number of B. dorsalis detected by computer system was 1570 and the manually counted number was 1591.The tracking accuracy is 98.7%.

Robust Autonomous Car-Like Robot Tracking Based on Tracking-Learning-Detection

This paper proposes a robust tracking algorithm for an autonomous car-like robot, and this algorithm is based on the Tracking-Learning-Detection (TLD). In this paper, the TLD method is extended to track the autonomous car-like robot for the first time. In order to improve accuracy and robustness of the proposed algorithm, a method of symmetry detection of autonomous car-like robot rear is integrated into the TLD. Moreover, the Median-Flow tracker in TLD is improved with a pyramid-based optical flow tracking method to capture fast moving objects. Extensive experiments and comparisons show the robustness of the proposed method.

Hybrid intensity- and phase-based optical flow tracking of tagged MRI.

Accurate tracking of the myocardium tissues in tagged Magnetic Resonance Images (MRI) is essential for evaluating the cardiac function. Current tracking methods utilize either the image intensity or the image phase as landmarks that can be tracked. In either case, the performance is vulnerable to the image quality and the fading of the tag lines. In this work, we propose a hybrid optical flow tracking method that combines both the intensity and the phase features of the image. The method is validated using numerical cardiac phantom as well as real MRI data experiments. Both experiments showed that the proposed method outperforms current intensity-based optical flow tracking and the phase-based HARP method with maximum error of 1 pixel at extreme conditions of tag fading.

Right ventricular strain imaging using 3D SPAMM combined with optical flow tracking

Summary A SPAMM sequence and optical flow tracking method that permits evaluation of strain in all 3 dimensions simultaneously in a single acquisition was successfully applied to the right ventricle. Background Presently there is no method of evaluating right ventricle function that simultaneously accounts for its complex geometry and motion. RV strain imaging is promising for evaluating regional dysfunction but to date it has not been possible to assess all 3 dimensions in a single acquisition. A novel variant of SPAMM tagging combined with optical flow tracking is available at our institution that can evaluate strain and torsion simultaneously in all 3 dimensions. We have applied this method to the right ventricle. Methods Data was analyzed, with attention to the right ventricle, from data acquired during a study of left ventricular remodeling post acute infarction. Imaging was obtained using respiratory triggering, with animals ventilated under general anesthesia, and triggering from invasive monitoring of left-sided pressures. 3-D tagging was performed at early (less than one week) and later (4-6 week) postoperative time points. Typical parameters for the 3-D tagging sequence at high resolution are: slice thickness 1mmx0.5mmx2mm interpolated to 0.5x0.5x2mm; 4mm tag spacing; coverage of about half of the heart, at mid-ventricle level; 6 nex; 30 minute acquisition time with respiratory gating. 5 or 6 phases were acquired during average systole. Typical parameters for the sheep 3-D tagging sequence at low resolution are: slice thickness 2mmx1mmx2mm interpolated to 1x1x2 ; whole heart coverage; 6mm tag spacing; 4 nex (4 signal acquisitions); 45 minute acquisition time with respiratory gating. Tag tracking was performed using an optical flow method and principal strain was calculated from the displacement using a custom program.