Optical Flow Method Research Articles

Hierarchical data visualization of experimental erythrocyte aggregation employing cross correlation and optical flow applications

Appraisal of microvascular erythrocyte velocity as well as aggregation are critical features of hemorheological assessment. Examination of erythrocyte velocity-aggregate characteristics is critical in assessing disorders associated with coagulopathy. Microvascular erythrocyte velocity can be assessed using various methodologic approaches; however, the shared assessment of erythrocyte velocity and aggregation has not been well described. The purpose of this study therefore is to examine three independent erythrocyte assessment strategies with and without experimentally induced aggregation in order to elucidate appropriate analytic strategy for combined velocity/aggregation assessment applicable to in-vivo capillaroscopy. We employed a hierarchical microfluidic model combined with Bland-Altman analysis to examine agreement between three methodologies to assess erythrocyte velocity appropriate for interpretation of cinematography of in-vivo microvascular hemorheology. We utilized optical and manual techniques as well as a technique which we term transversal temporal cross-correlation (TTC) to observe and measure both erythrocyte velocity and aggregation. In general, optical, manual and TTC agree in estimation of velocity at relatively low flow rate, however with an increase in infusion rate the optical flow method yielded the velocity estimates that were lower than the TTC and manual velocity estimates. We suggest that this difference was due to the fact that slower moving particles close to the channel wall were better illuminated than faster particles deeper in the channel which affected the optical flow analysis. Combined velocity/aggregation appraisal using TTC provides an efficient approach for estimating erythrocyte aggregation appropriate for in-vivo applications. We demonstrated that the optical flow and TTC analyses can be used to estimate erythrocyte velocity and aggregation both in ex-vivo microfluidics laboratory experiments as well as in-vivo recordings. The simplicity of TTC method may be advantageous for developing velocity estimate methods to be used in the clinic. The trade-off is that TTC estimation cannot capture features of the flow based on optical flow analysis of individually tracked particles.

Measurement of Bridge Vibration by UAVs Combined with CNN and KLT Optical-Flow Method

A measurement method of bridge vibration by unmanned aerial vehicles (UAVs) combined with convolutional neural networks (CNNs) and Kanade–Lucas–Tomasi (KLT) optical-flow method is proposed. In this method, the stationary reference points in the structural background are required, a UAV is used to shoot the structure video, and the KLT optical-flow method is used to track the target points on the structure and the background reference points in the video to obtain the coordinates of these points on each frame. Then, the characteristic relationship between the reference points and the target points can be learned by a CNN according to the coordinates of the reference points and the target points, so as to correct the displacement time–history curves of target points containing the false displacement caused by the UAV’s egomotion. Finally, operational modal analysis (OMA) is used to extract the natural frequency of the structure from the displacement signal. In addition, the reliability of UAV measurement combined with CNN is proved by comparing the measurement results of the fixed camera and those of UAV combined with CNN, and the reliability of the KLT optical-flow method is proved by comparing the tracking results of the digital image correlation (DIC) and KLT optical-flow method in the experiment of this paper.

The Co-alignment of Winged Hα Data Observed by the New Vacuum Solar Telescope* *Supported by the National Natural Science Foundation of China.

The New Vacuum Solar Telescope (NVST) has been releasing its novel winged Hα data (WHD) since 2021 April, namely the Hα imaging spectroscopic data. Compared with the prior released version, the new data are further co-aligned among the off-band images and packaged into a standard solar physics community format. In this study, we illustrate the alignment algorithm used by the novel WHD, which is mainly based on the optical flow method to obtain the translation offset between the winged images. To quantitatively evaluate the alignment results of two images with different similarities, we calculate the alignment accuracies between the images of different off-band and line center, respectively. The result shows that our alignment algorithm could reach up to the accuracy of about 0.″1 when the off-band of winged image is lower than 0.6 Å. In addition, we introduce the final product of the WHD in detail, which can provide convenience for the solar physicists to use high-resolution Hα imaging spectroscopic data of NVST.

Improved Multidomain Convolutional Neural Network for Intelligent Tracking of Marangoni Effect Process Sequence Images

Sequence image target tracking has two requirements of accuracy and speed. Aiming at the problem of slow speed of convolutional neural network (CNN) target tracking algorithm, an improved algorithm for sequence image target tracking is designed by using a fusion of Faster multidomain CNN (MDNet) and optical flow method. The target algorithm designed in this article is improved on the basis of MDNet. The optical flow method is used to obtain the primary selection box first, which can effectively deal with the challenges brought by scale change and in-plane rotation, and has good tracking speed and performance. The objective algorithm of this article is used to explore the Marangoni effect in the melting process of solid iron tailings in molten blast furnace slag. Because the main component of iron tailings is Silica, and the melting point of Silica is the highest, so taking Silica as the research object, an intelligent analysis of Marangoni effect in high-temperature melting process of Silica particles is realized. The results show that the target algorithm achieves the real-time standard (FPS > 24), the average distance precision (DP) is 97.3%, the average success rate (SR) is 98.02%, the average center location error (CLE) is 6.2684 <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">μ</i> m, and the average overlap precision (OP) coverage is 63.86%. The target algorithm has certain advantages in the following five evaluation indexes: DP, SR, CLE, OP, and frames per second.

Spatiotemporal optimization on cross correlation for particle image velocimetry

We introduce an optimization method for the cross-correlation operation in particle image velocimetry by locating the correlation peaks assisted with constraint conditions. In this study, an objective function was constructed to include the residual of the normalized cross-correlation term, a component in charge of spatial smoothness (inspired by the optical flow method as used in a previous study) and a component for temporal smoothness (inspired by the concept of trajectory selection in particle tracking velocimetry). Minimizing the objective function gives optimized velocity fields for a series of tracer images for spatiotemporal smoothness. The proposed method was examined in synthetic images of turbulent flow and Batchelor vortex and in a laboratory experiment of vortex rings. The effect of image background noises and the initial guess for the optimization process were examined and discussed.

MOVING OBJECTS DETECTION, CLASSIFICATION AND TRACKING OF VIDEO STREAMING BY IMPROVED FEATURE EXTRACTION APPROACH USING K-SVM

The computer vision plays a vital role in variety of applications such as traffic surveillance, robotics, human interaction devices, etc. The video surveillance system has designed to detect, track and classify the moving objects. The moving object detection, classification and tracking of video streaming has various challenges, which utilizes various novel approaches. The existing work uses spatiotemporal feature analysis using sample consistency algorithm for moving object detection and classification. It is not performed well with the complex scene on the video. The binary masking representation of moving object is the challenging task for the researchers. These video streams are partitioned based on the frames, shots, and scenes; here the proposed research work utilizes kernel-Support Vector Machine learning technique for moving object detection and tracking. In this approach, the MIO-TCD DATASET is used for moving object detection. Here the feature extraction is the major part of foreground and background analysis in the video streaming, which utilizes the vehicle features based video data. The SURF (Speeded-Up Robust Feature) feature is used to recognize/register the object and it also used for classification of moving objects. Here the optical flow method is to quantify the relative motion of object in the video streams. Based on the differences on the partitioned frames, the optical flow features hold the object for measuring the pixel of the moving objects. The feature extraction process is improved by combining feature class with intensity level of optical flow result, which makes the gradient analysis of first order derivative function. The proposed method achieves the result of recall, precision, and f-measures than the existing work. The proposed method is done with the help of MATLAB 2018a. Keywords: Computer Vision and Pattern Recognition; Kernel-SVM; SURF features; Optical Flow; Texture feature; Moving object detection, tracking and classification;

Approach of recognition and precision location for the beacon in satellite optical communications

Satellite optical communications have high communication speed and good confidentiality, which can establish high-speed and large-capacity communication links between satellites and the ground or other satellites. The requirements for the PAT (pointing, acquisition and tracking) system of satellite optical communication terminals are very high. In satellite optical communications, the biggest influence on the beacon’s recognition and tracking is the background light on CCD (Charge Coupled Device). Therefore, denoising work is required during the recognition or tracking process. This paper uses corner detection method to extract the beacon frame by frame. Mark the spot, and use the optical flow method to calculate the optical flow displacement based on the two consecutive frames. In addition, calculation accuracy of the beacon’s centroid is compared with that of the tradition method. The feasibility is verified by simulation experiment, the tracking process is completed, and the noise can be precisely eliminated.

Construction and Evaluation of Sports Rehabilitation Training Model under Intelligent Health Monitoring

Sports rehabilitation training is a comprehensive discipline that solves and helps motor function recovery through physical exercise, also known as rehabilitation sports training. Sports rehabilitation training can help athletes restore health, treat injuries, prevent recurrence, improve physical fitness, and meet the needs of daily training. Starting from the concept of sports rehabilitation training, this paper analyzes the application of sports rehabilitation training in physical training and summarizes and analyzes the role of sports rehabilitation training. It provides reference for the application of physical rehabilitation training in sports training. Intelligent health monitoring technology has been widely used and developed, and many classic algorithms have emerged to complete the extraction of moving targets. This paper is aimed at studying the application of intelligent health monitoring to the construction of sports rehabilitation training models. It proposes the overall design of the intelligent health monitoring terminal and the common algorithms for moving target monitoring, such as optical flow method, frame difference method, and background difference method. It takes the rehabilitation training of an athlete in X city as an example to establish a model study. The final result of the experiment through sports rehabilitation training under intelligent health monitoring is that 92.6% of the athletes recovered from sports injuries through rehabilitation physical training.

Robust depth estimation on real-world light field images using Gaussian belief propagation

Depth estimation on light field images has been well researched on synthetic datasets. However, existing methods often fail on real-world light field images because of the real-world noises. Due to the lack of real-world light field image datasets for depth estimation, the learning-based methods, e.g., convolutional neural networks, are not capable of performing well on real-world light field images with only synthetic datasets available. In this paper, we adopt the optical flow estimation method to estimate depth from light field images since (i) the existing optical flow methods have shown robustness with significant real-world noises; (ii) the adopted optical flow estimation method does not require any training data, thus not limited to the synthetic datasets. The depth can be solved via the approach of optical flow estimation because (i) a sub-aperture image array can be converted from a light field image; (ii) the optical flow between every two adjacent sub-aperture images of the image array is exactly the disparity between the two images. Furthermore, since a sub-aperture array contains numerous adjacent image pairs, numerous optical flow maps can be generated from the sub-aperture array. We show the depth can be well refined from these numerous optical flow maps through modeling these maps in a single graphical model (a pairwise Gaussian MRF to be precise) and inferencing this graphical model by Gaussian belief propagation (GaBP). Experiments show that i) the proposed method achieves better depth estimation than the state-of-the-art methods on real-world light field images; ii) the proposed method can estimate the depth of objects at a far distance (3.5–9.5 m) much more accurately than the state-of-the-art convolutional neural networks.

Moving Object Tracking Based on Sparse Optical Flow with Moving Window and Target Estimator.

Moving object detection and tracking are technologies applied to wide research fields including traffic monitoring and recognition of workers in surrounding heavy equipment environments. However, the conventional moving object detection methods have faced many problems such as much computing time, image noises, and disappearance of targets due to obstacles. In this paper, we introduce a new moving object detection and tracking algorithm based on the sparse optical flow for reducing computing time, removing noises and estimating the target efficiently. The developed algorithm maintains a variety of corner features with refreshed corner features, and the moving window detector is proposed to determine the feature points for tracking, based on the location history of the points. The performance of detecting moving objects is greatly improved through the moving window detector and the continuous target estimation. The memory-based estimator provides the capability to recall the location of corner features for a period of time, and it has an effect of tracking targets obscured by obstacles. The suggested approach was applied to real environments including various illumination (indoor and outdoor) conditions, a number of moving objects and obstacles, and the performance was evaluated on an embedded board (Raspberry pi4). The experimental results show that the proposed method maintains a high FPS (frame per seconds) and improves the accuracy performance, compared with the conventional optical flow methods and vision approaches such as Haar-like and Hog methods.

Bayesian inference for the dynamic properties of long-span bridges under vortex-induced vibration with Scanlan’s model and dense optical flow scheme

The dynamic properties of long-span bridges with flexible and light-weight features are of critical importance in safety assessment and response prediction. The primary concern of this study was to infer the dynamic characteristics of long-span bridges subjected to vortex-induced vibration (VIV) by reasonably accommodating the fluid–structure interactions during VIV and the various uncertainties arising in the dynamic characterization of the structures due to measurement noise and modeling errors. By taking the advantage of advanced image processing techniques, the Farnebäck dense optical flow method was used to process the vibration video to extract the displacement time history. In contrast to conventional operational modal analysis (OMA) approaches, which are usually based on the white noise assumption, a new frequency-domain Bayesian method is proposed to make a statistical inference for the dynamic characteristics of the bridge based on the power spectral density of the VIV measurements and the Scanlan’s empirical VIV model for bridge decks. A fast computation scheme is also proposed to achieve the posterior uncertainties of the modal frequency of the bridge, the structural damping of the bridge, and the total damping of the VIV system. The efficiency and the accuracy of the proposed algorithms have been verified with the VIV motion of a super long-span suspension bridge. Comparison with those identified using different OMA approaches in the time domain and the frequency domain is also be presented.

Initial evaluation of 2D and 3D simulated high-speed 1000 fps vascular contrast-flow image sequences using computational fluid dynamics (CFD).

Digital subtraction angiography (DSA) remains the clinical standard for detailed visualization of the neurovasculature due to its high-spatial resolution; however, detailed blood-flow quantification is impaired by its low-temporal resolution. Advances in photon-counting detector technology have led us to develop High-Speed Angiography (HSA), where x-ray images are acquired at 1000 fps for more accurate visualization and quantification of blood flow. We have implemented a physics-based optical flow method to extract such information from HSA, but validation of the angiography-derived velocity distributions is not straightforward. Computational fluid dynamics (CFD) is widely regarded as the benchmark for hemodynamic analysis, as it provides a multitude of quantitative flow parameters throughout the volume of interest. However, there are several limitations with this method related to over-simplification of boundary conditions and suboptimal meshing (spatial resolution), that make CFD simulation results an inexact criterion for validation. To overcome this issue for HSA validation, CFD was used to generate both simulated high-speed angiograms and the corresponding ground-truth 3D flow fields to better understand the relationship between the 3D volumetric-flow distribution and the 2D projected-flow distribution as is obtained with angiography, and the subsequent 2D approximation of flow velocity. Several geometries were investigated, ranging from simple pipe models to complex patient-specific aneurysms. Simulated datasets were analyzed with the optical flow algorithm, and the effects of flow divergence, quantum mottle, and intensity gradient on the calculation were evaluated. From these simulations, we can evaluate whether flow fields reconstructed from HSA are representative of significant flow patterns in the 3D vasculature.

Study on flow distribution of irrigation canal system based on image velocimetry

• The YOLOV5 algorithm combined with the monocular ranging principle is used for fast flow measurement of UAV images for UAV hovering type measurement of local river surface flow fields. • Based on the smooth circular arc channel distribution model, the velocity distribution law of Yongji section, Yonggang section and Yonglan section is studied and summarized. • Based on The Rule of Tyson polygon, combined with the representative point of average velocity, the measurement of flow through water section is calculated, and the contactless measurement is realized. With the development of computer technology, it has become possible to measure surface flow velocity using video technology. Ripples are produced when flowing, and flow velocity can be determined based on the ripples and floating objects in water. A UAV velocity measurement system based on optical flow method and YOLOV5 algorithm in deep learning has been established, to realize the monitoring of surface flow field of large rivers. The optical flow method and the deep learning algorithm are used to track the target, convert the pixel distance to actual distance by monocular ranging, and calculate the actual flow velocity. The method was successfully applied to the surface flow field measurement of the Yongji canal in the river-loop irrigation area of Inner Mongolia, and the results showed that the orthophoto graphic images obtained by the method were of high quality, the flow field calculation results were reasonable, and the calculation results agreed well with the measurements.

Effectiveness Of Moving Objects Detecting And Tracking In Airspace By Images In NearInfrared

Objects in nearinfrared (NIR) images can often have different linear scales and shapes than the same objects in optical images for visible spectrum (RedGreenBlue, RGB). Therefore they can require different computer vision methods for detection, tracking, and classification. This paper devoted to the methods by which the problems of moving objects detecting and tracking in NIR images are solved. The main characteristics of moving objects in image sequences are highlighted. Advantages and disadvantages of different methods for detecting and tracking objects in NIR images of airspace are considered and two of the most promising methods classes are selected. Studies have been carried out on the effectiveness of LucasKanade method, which is one of the methods of local optical flow, and the ORB method of scaleinvariant transformation of features when detecting and tracking moving objects in NIR images. In numerical experiments, more than 5000 NIR images containing moving objects of three types were used as well as three combinations of considered methods. It is shown which combination is the most accurate in the tasks of moving objects detection and tracking and can be used for airspace automatic operational control and management based on computer vision systems. Probably, other combinations of methods from the two considered classes also can help to increase accuracy of moving objects detection and tracking in airspace by NIR images.

Comparison of Two Methods for Calculating Magnetic Helicity in the Solar Corona

Duo to the large magnetic Reynolds number, the magnetic helicity originating from the solar interior can be carried away through the photosphere into the corona. However, the relationship between the accumulated magnetic helicity flux through the photosphere and the magnetic helicity in the corona is still unclear. By selecting 36 newly emerging active regions in the 23rd solar cycle, we apply optical flow methods to derive the accumulated magnetic helicity through the photosphere (H m p ) by using the sequential longitudinal magnetograms, we use nonlinear force-free field extrapolation to obtain the 3D coronal magnetic field, and we adopt finite volume methods to calculate the instantaneous relative magnetic helicity in the corona (H m c ) by using vector magnetograms. It is found that the local correlation tracking (LCT)–based H m p is larger than H m c in 1″, and that the Differential Affine Velocity Estimator–based H m p is more consistent with H m c than the LCT-based H m p . H m p is more consistent with H m c in evaluation from 2″ than from 1″. Moreover, systematically shows consistency with the Hemispheric Helicity Rule (over 55%), no matter which resolution and method are used. These estimations suggest that the consistency of H m c and H m p is partly dependent on the resolution of the magnetograms and the calculation methods.

DeepFake Video Detection through Facial Sparse Optical Flow based Light CNN

DeepFake detection has become an attractive research topic with tremendous growth of interests recently. However, existing DeepFake detection studies spare no effort to improve accuracy or Area Under Curve metric, regardless of computing costs. In this work, the tradeoff between result accuracy and computing resources is taken into consideration. A facial sparse optical flow method is proposed to extract spatio-temporal features representing facial expression incoherence, which helps to distinguish fake videos and real videos. The features fed into a light CNN model are highly compact and low-dimensional. The proposed method has an amazing small amount of parameters with high training speed and low usage of GPU memory. The low resource requirement makes it possible to port to embedded development platform.

Accurate prediction of the particle image velocimetry flow field and rotor thrust using deep learning

With particle image velocimetry (PIV), cross-correlation and optical flow methods have been mainly adopted to obtain the velocity field from particle images. In this study, a novel artificial intelligence (AI) architecture is proposed to predict an accurate flow field and drone rotor thrust from high-resolution particle images. As the ground truth, the flow fields past a high-speed drone rotor obtained from a fast Fourier transform-based cross-correlation algorithm were used along with the thrusts measured by a load cell. Two deep-learning models were developed, and for instantaneous flow-field prediction, a generative adversarial network (GAN) was employed for the first time. It is a spectral-norm-based residual conditional GAN translator that provides a stable adversarial training and high-quality flow generation. Its prediction accuracy is 97.21 % (coefficient of determination, or R2). Subsequently, a deep convolutional neural network was trained to predict the instantaneous rotor thrust from the flow field, and the model is the first AI architecture to predict the thrust. Based on an input of the generated flow field, the network had an R2 accuracy of 94.57 %. To understand the prediction pathways, the internal part of the model was investigated using a class activation map. The results showed that the model recognized the area receiving kinetic energy from the rotor and successfully made a prediction. The proposed architecture is accurate and nearly 600 times faster than the cross-correlation PIV method for real-world complex turbulent flows. In this study, the rotor thrust was calculated directly from the flow field using deep learning for the first time.

A Study on the Design and Implementation of an Improved AdaBoost Optimization Mathematical Algorithm Based on Recognition of Packaging Bottles

In this paper, a special design system is developed based on the design of the packaging bottle to achieve the effective acquisition of the image of the cross-section of the packaging bottle to be measured under the condition of limited space size, avoiding the distortion of the object to be measured. At the same time, the image of the area where the target packaging bottle is located is segmented, and the curve features are quickly determined and effectively matched with the template library to realize the recognition of the shape features of the bottle. In this paper, the design of the packaging bottle is first designed by mechanism design and 3D modeling, followed by rapid prototyping methods such as 3D printing, and the prototype is made for functional verification. Finally, the transmission speed and stability of the design system for packaging bottle recognition are improved through structural analysis and optimization methods. To realize the intelligent control of the packaging bottle transmission and identification system, the hardware control circuit is designed and the relevant intelligent control program is prepared based on the embedded system so that the packaging bottles in the transmission process can be quickly and accurately positioned and identified. An improved AdaBoost algorithm is proposed for packaging bottle detection. In the process of algorithm learning, the Haar features are too large and time-consuming, and the training sample is cropped to remove the sample edge pixels, which effectively reduces the number of features, thus reducing the computation. The proposed optical flow method is used to obtain the motion region in the video image as the region of interest, and the canny operator is used in the region of interest for edge detection, and the region of interest is filtered by the edge energy to exclude the noninterest region. Finally, the AdaBoost algorithm is used to detect the region of interest, which reduces the detection area and decreases the detection time. The improved AdaBoost algorithm has a high accuracy improvement over the traditional AdaBoost algorithm for the recognition of various packaging bottles with relatively suitable training set samples, and the system recognition time has reached the requirements of industrial recognition.

Nowcasting Meso-γ-Scale Convective Storms Using Convolutional LSTM Models and High-Resolution Radar Observations

As a deep learning approach to improving precipitation nowcasting, this study proposes convolutional Long Short-Term Memory (LSTM) models which specifically target meso-γ-scale, localized convective storms although there are numerous types of thunderstorms across scales. A Convolutional LSTM Model (CLM) and an Encoder-Decoder Model (EDM) were built by employing LSTM networks that perform better with sequential data. For training, radar reflectivity (<em>Z</em>) datasets of the Multi-Parameter Phased Array Weather Radar (MP-PAWR) for a convective storm event that occurred over the Tokyo metropolitan area were used as input to the models. An Advection Forecast Model (AFM) that utilizes an optical flow method to generate <em>u</em> and <em>v</em> motion vectors and the Lagrangian advection scheme was also developed to compare with nowcasts from the LSTM models. Model performances were assessed using statistics and skill score measures such as Critical Success Index (CSI) and Probability of Detection (POD) with lead time up to 10 min. It was found that for a total rain area with <em>Z</em> &gt; 10 dB<em>Z</em>, the CLM had the best skill, showing higher CSI scores and correlation coefficients than other models for all lead times. For a convective rain area with <em>Z</em> &gt; 35 dB<em>Z</em>, the CLM showed equal to or slightly higher CSI scores than the AFM until the lead time of 7.5 min but underpredicted the strength of convective cells with a lower CSI at 10 min. Both the CLM and EDM showed more negative <em>Z</em> biases at longer lead times, resulting in lower CSI and POD scores than the AFM in this convective rain category. The higher skill of the AFM at longer lead times is most likely because the convective cells were advected without changing their shapes and intensities largely in this short period.

Two-Dimensional Flow Field Measurement Method for Sediment-Laden Flow Based on Optical Flow Algorithm

This paper proposes a novel particle image velocimetry (PIV) technique to generate an instantaneous two-dimensional velocity field for sediment-laden fluid based on the optical flow algorithm of ultrasound imaging. In this paper, an ultrasonic PIV (UIV) system is constructed by integrating a medical ultrasound instrument and an ultrasonic particle image velocimetry algorithm. The medical ultrasound instrument with a phased sensor array is used to acquire acoustic echo signals of particles in water and generate two-dimensional underwater ultrasound images. Based on the optical flow field, the instantaneous velocity of the particles in water corresponding to the pixels in the ultrasonic particle images is derived from the grayscale change between adjacent images under the L-K local constraint based on the optical flow field, and finally, the two-dimensional flow field is obtained. Through multiple sets of experiments, the proposed algorithm is verified. The experimental results are compared with those of the conventional cross-correlation algorithms. The results show that the L-K optical flow method can not only obtain the underwater velocity field accurately, but also has the advantages of good smoothness and extensive suitability, especially for the flow field measurement in sediment-laden fluid than conventional algorithms.