Optical Flow Equation Research Articles

Adaptive Fractional-Order Multi-Scale Optimization TV-L1 Optical Flow Algorithm

We propose an adaptive fractional multi-scale optimization optical flow algorithm, which for the first time improves the over-smoothing of optical flow estimation under the total variation model from the perspective of global feature and local texture balance, and solves the problem that the convergence of fractional optical flow algorithms depends on the order parameter. Specifically, a fractional-order discrete L1-regularization Total Variational Optical Flow model is constructed. On this basis, the Ant Lion algorithm is innovatively used to realize the iterative calculation of the optical flow equation, and the fractional order is dynamically adjusted to obtain an adaptive optimization algorithm with strong search accuracy and high efficiency. In this paper, the flexibility of optical flow estimation in weak gradient texture scenes is increased, and the optical flow extraction rate of target features at multiple scales is greatly improved. We show excellent recognition performance and stability under the MPI_Sintel and Middlebury benchmarks.

Optical flow-based vertical angular rate fault detection on UAVs

This paper discusses optical flow-based vertical angular rate fault detection based-on real fight IMU data and camera image sequences. The fight test platform utilizes a gimbal stabilized downward looking camera that is why only the angular rate around the inertial vertical axis can be estimated. Improvements relative to the previous work of the authors are the application of forward-backward optical flow to filter outliers, the direct detection of angular rate faults without the need for Euler angles or GPS velocities and tuning and verification on real images and fight data. After presenting the optical flow equations and the method for image-based vertical angular rate estimation the fight test scenarios and video processing steps are introduced. Four different Artificial errors are added to the IMU measurements and the fault detection applies residual thresholding and up-down counters. Tuning is based on false alarm and missed detection rates and the application of receiver operating characteristics. Finally, the estimation results and possible further improvements are presented.

Image-based UAV position and velocity estimation using a monocular camera

Autonomous landing of aerial vehicles is challenging, especially in emergency flight scenarios in which precise information about the vehicle and the environment is required for near-to-ground maneuvers. In this paper, the optic-flow concept based on feature detection is applied to estimate the vertical distance and the velocity vector of a multirotor UAV (MUAV) for landing. The UAV kinematics, the optical flow equations, and the detected feature states, provided by a low-cost monocular camera, are combined to develop a novel appropriate model for estimation. The proposed algorithm applies the variation of detected features, the angular velocities, as well as the Euler angles, measured by the Inertial Measurement Unit (IMU), to estimate the vertical distance of the UAV to the ground, the MUAV velocity vector, and also to predict the future features position. Extended Kalman filter (EKF) is applied as the estimation method on the coupled optic-flow and kinematic equations. The accuracy of state estimation is enhanced by the idea of multiple-feature tracking. The 6-DOF simulations, laboratory experiments, and comparison of results demonstrate the capability of height and velocity estimation of a MUAV in the landing phase of flight by just applying the low-cost camera information. Monte Carlo simulations have been performed to study the effect of IMU acceleration, and angular velocity measurement noises as well as the number of the detected features on the success probability of the estimation process. The results reveal that increasing the number of detected features, i.e tracking multiple features, increases the estimation accuracy, however, it mainly improves the success probability, which is a more important factor in practical scenarios.

Attention-Based Bi-Prediction Network for Versatile Video Coding (VVC) over 5G Network

As the demands of various network-dependent services such as Internet of things (IoT) applications, autonomous driving, and augmented and virtual reality (AR/VR) increase, the fifthgeneration (5G) network is expected to become a key communication technology. The latest video coding standard, versatile video coding (VVC), can contribute to providing high-quality services by achieving superior compression performance. In video coding, inter bi-prediction serves to improve the coding efficiency significantly by producing a precise fused prediction block. Although block-wise methods, such as bi-prediction with CU-level weight (BCW), are applied in VVC, it is still difficult for the linear fusion-based strategy to represent diverse pixel variations inside a block. In addition, a pixel-wise method called bi-directional optical flow (BDOF) has been proposed to refine bi-prediction block. However, the non-linear optical flow equation in BDOF mode is applied under assumptions, so this method is still unable to accurately compensate various kinds of bi-prediction blocks. In this paper, we propose an attention-based bi-prediction network (ABPN) to substitute for the whole existing bi-prediction methods. The proposed ABPN is designed to learn efficient representations of the fused features by utilizing an attention mechanism. Furthermore, the knowledge distillation (KD)- based approach is employed to compress the size of the proposed network while keeping comparable output as the large model. The proposed ABPN is integrated into the VTM-11.0 NNVC-1.0 standard reference software. When compared with VTM anchor, it is verified that the BD-rate reduction of the lightweighted ABPN can be up to 5.89% and 4.91% on Y component under random access (RA) and low delay B (LDB), respectively.

Motion acoustic flow field: Underwater small target motion characterization for active sonar echograph of harbor environment.

Spatial-temporal variations of active sonar echo intensity can provide effective motion information for characterizing intruding small targets and play a key role in follow-up tracking, behavioral analysis, and recognition, etc. Inspired by the idea of optical flow, which can be used to calculate subtle spatial-temporal variations of each pixel in image sequences, a different motion acoustic flow field (MAFF) is proposed for estimating the motion of underwater small targets in successive active sonar echographs from harbor environments. This is because directly applying current calculation framework for optical flow presents two challenges in this case. The first challenge is that the echo intensity fluctuation breaks its potential assumption of brightness consistency in the pre-processing stage. The second challenge is that the small size of the blob targets could be smoothed out as outliers by its median filter-based post-processing. Hence starting from the classical optical flow equation, MAFF introduces a novel spatial-temporal connected component pre-processing and a novel blob shape segmentation refinement post-processing. Experiments on a set of real-world harbor datasets demonstrate the efficacy of our MAFF calculation framework.

Optical modeling for recursional Hasimoto map with normalized time fractional applications

In this paper, we construct the optical flow equations obtained with the help of a timelike curve in De Sitter 2D space S12 and with the aid of the Hasimoto map defined by this curve. First, we introduce the Hasimoto map, which we call the De-Hasimoto map, for a timelike curve in S12. Then, with the help of this map and the equation representing the vortex filament motion, we obtain some flow equations with time and arc length parameters. Finally, we compute exact solutions of time fraction of an obtained optical motion equation, and we present some physical reviews on it.

Skin friction and surface optical flow in viscous flows

The relationship between skin friction and the surface optical flow (SOF) in viscous flows is discussed based on the evolution equations of surface temperature, scalar, and enstrophy, where the SOF is defined as the convection velocity of these quantities. It is found that the SOF is proportional to skin friction, which can be determined by solving the optical flow equation re-cast from these evolution equations. This optical flow method can be applied to surface temperature and mass transfer visualizations to extract skin friction fields in experiments. To examine this method, it is first applied to complex surface enstrophy structures obtained in direct numerical simulation data of a turbulent channel flow. Furthermore, it is applied to surface temperature structures obtained in time-resolved temperature sensitive paint measurements in a flow over a National Advisory Committee for Aeronautics (NACA) 0015 airfoil model and an impinging jet.

A comparison of satellite cloud motion vectors techniques to forecast intra-day hourly solar global horizontal irradiation

Solar forecasting provides valuable information for grid management. Satellite-based forecasting tools account for the short-term intra-day time horizons, typically outperforming numerical weather predictions up to 4–5 h ahead. The method consist of three separated stages, namely, cloud motion estimation, motion extrapolation and satellite-to-irradiation conversion. In this work we compare different satellite-based proposals for hourly irradiation forecast up to 5 h ahead using a 2-years data set. The widely-used Lorenz’s block matching technique and four optical flow (OF) algorithms are assessed, both at image and irradiation levels. All the methods are locally optimized to obtain their peak performance. It is found that the OF algorithm which combines an L1 data penalty term on the optical flow equation with total variation regularization (TVL1) outperforms the rest. Different image extrapolation approaches and spatial smoothing are also tested. It is found that changing the extrapolation technique does not have much impact in the overall performance and that important gains can be obtained by optimally smoothing the predicted images previous to solar irradiation conversion. By doing this, all methods outperform the exigent convex persistence benchmark, achieving positive forecasting skills. The tests are performed using GOES-East satellite images of south-east South America, and the methods’ optimal parameters are given.

Real-Time Analysis of Athletes’ Physical Condition in Training Based on Video Monitoring Technology of Optical Flow Equation

This article is dedicated to the research of video motion segmentation algorithms based on optical flow equations. First, some mainstream segmentation algorithms are studied, and on this basis, a segmentation algorithm for spectral clustering analysis of athletes’ physical condition in training is proposed. After that, through the analysis of the existing methods, compared with some algorithms that only process a single frame in the video, this article analyzes the continuous multiple frames in the video and extracts the continuous multiple frames of the sampling points through the Lucas-Kanade optical flow method. We densely sampled feature points contain as much motion information as possible in the video and then express this motion information through trajectory description and finally achieve segmentation of moving targets through clustering of motion trajectories. At the same time, the basic concepts of image segmentation and video motion target segmentation are described, and the division standards of different video motion segmentation algorithms and their respective advantages and disadvantages are analyzed. The experiment determines the initial template by comparing the gray-scale variance of the image, uses the characteristic optical flow to estimate the search area of the initial template in the next frame, reduces the matching time, judges the template similarity according to the Hausdorff distance, and uses the adaptive weighted template update method for the templates with large deviations. The simulation results show that the algorithm can achieve long-term stable tracking of moving targets in the mine, and it can also achieve continuous tracking of partially occluded moving targets.

Magnetic charged particles of optical spherical antiferromagnetic model with fractional system

Abstract In this article, we first consider approach of optical spherical magnetic antiferromagnetic model for spherical magnetic flows of ϒ \Upsilon -magnetic particle with spherical de-Sitter frame in the de-Sitter space S 1 2 {{\mathbb{S}}}_{1}^{2} . Hence, we establish new relationship between magnetic total phases and spherical timelike flows in de-Sitter space S 1 2 {{\mathbb{S}}}_{1}^{2} . In other words, the applied geometric characterization for the optical magnetic spherical antiferromagnetic spin is performed. Moreover, this approach is very useful to analyze some geometrical and physical classifications belonging to ϒ \Upsilon -particle. Besides, solutions of fractional optical systems are recognized for submitted geometrical designs. Geometrical presentations for fractional solutions are obtained to interpret the model. These obtained results represent that operation is a compatible and significant application to restore optical solutions of some fractional systems. Components of models are described by physical assertions with solutions. Additionally, we get solutions of optical fractional flow equations with designs of our results in de-Sitter space S 1 2 {{\mathbb{S}}}_{1}^{2} .

Optical flow method for neutron radiography flow diagnostics

The principle of an optical flow method is formulated for neutron radiography (NR) flow diagnostics to determine high-resolution velocity fields of two-phase flows. This method is based on solving an optical flow equation derived from the continuous equation for NR image analysis as an inverse problem using a variational method. The mathematical definition and physical meaning of the optical flow in NR images of a two-phase flow are clarified. As an example, the optical flow method is used to extract complex velocity fields from a time sequence of dynamic NR images acquired in an air–water two-phase flow in a flat bubbler.

Differential Scene Flow from Light Field Gradients

This paper presents novel techniques for recovering 3D dense scene flow, based on differential analysis of 4D light fields. The key enabling result is a per-ray linear equation, called the ray flow equation, that relates 3D scene flow to 4D light field gradients. The ray flow equation is invariant to 3D scene structure and applicable to a general class of scenes, but is under-constrained (3 unknowns per equation). Thus, additional constraints must be imposed to recover motion. We develop two families of scene flow algorithms by leveraging the structural similarity between ray flow and optical flow equations: local 'Lucas-Kanade' ray flow and global 'Horn-Schunck' ray flow, inspired by corresponding optical flow methods. We also develop a combined local-global method by utilizing the correspondence structure in the light fields. We demonstrate high precision 3D scene flow recovery for a wide range of scenarios, including rotation and non-rigid motion. We analyze the theoretical and practical performance limits of the proposed techniques via the light field structure tensor, a 3x3 matrix that encodes the local structure of light fields. We envision that the proposed analysis and algorithms will lead to design of future light-field cameras that are optimized for motion sensing, in addition to depth sensing.

Research on Moving Target Tracking Based on FDRIG Optical Flow

Aiming at the problem of moving target recognition, a moving target tracking model based on FDRIG optical flow is proposed. First, the optical flow equation was analyzed from the theory of optical flow. Then, with the energy functional minimization, the FDRIG optical flow technique was proposed. Taking a road section of a university campus as an experimental section, 30 vehicle motion sequence images were considered as objects to form a vehicle motion sequence image with a complex background. The proposed FDRIG optical flow was used to calculate the vehicle motion optical flow field by the Halcon software. Comparable with the classic Horn and Schunck (HS) and Lucas and Kande (LK) optical flow algorithm, the monitoring results proved that the FDRIG optical flow was highly precise and fast when tracking a moving target. The Ettlinger Tor traffic scene was then taken as the second experimental object; FDRIG optical flow was used to analyze vehicle motion. The superior performance of the FDRIG optical flow was further verified. The whole research work shows that FDRIG optical flow has good performance and speed in tracking moving targets and can be used to monitor complex target motion information in real-time.

A field-segmentation-based variational optical flow method for PIV measurements of nonuniform flows

Air flow fields have nonuniform velocity distributions because of the compressibility of air. Current optical flow-based particle image velocimetry (PIV) methods cannot thoroughly solve these distributions because of the global optical flow formulation. In this study, we propose a novel field-segmentation-based variational optical flow (FS-VOF) method for preserving the spatially discontinuous characteristics of nonuniform flow fields. The proposed method is designed based on the level-set segmentation method, in which the particle image is segmented according to the velocity distribution of the fluid flow. Subsequently, a segmented smoothness constraint term built from the assumption that velocity varies continuously in each segmented particle image region and the discontinuous structures of the velocity field are preserved at the boundaries of the segmented regions. In addition, the data term of the proposed method is based on the segmented regions and derived from the physics-based optical flow equation. The proposed method is evaluated over synthetic and experimental images, and the comparison with advanced variational optical flow algorithms demonstrates the effectiveness of our method in preserving the spatially discontinuous characteristics of nonuniform flow fields.

Robust optical flow estimation based on wavelet

Concentrating on the issue that the systematic error caused by variation of illumination conditions and sensor noise leads to poor robustness and low accuracy of optical flow calculation, based on the wavelet multi-resolution theory, a robust optical flow estimation method is developed in this paper. With the multi-resolution characteristics of wavelet, the systematic error is incorporated into the calculation of optical flow to improve the robustness and estimation accuracy. In what follows, the total least squares method can be exploited to solve the obtained overdetermined wavelet optical flow equations such that the optical flow vector can be achieved. As compared to the traditional Lucas–Kanade approach, Horn–Schunck method, and the optical flow estimation algorithm in omnidirectional images using wavelet approach, simulation results show that the proposed algorithm can significantly improve the accuracy of optical flow estimation and the robustness of the optical flow field.

Fast Von Mises strain imaging on ultrasound carotid vessel wall by flow driven diffusion method.

The elasticity of the vessel wall is important for the clinical identification of rupture-risks. The Von Mises strain can be a potential index for the indication of carotid vessel pathologies. In this paper, a fast clinically applicable real-time algorithm from time-sequence of B-mode carotid images is developed. Due to the compression induced by the normal cardiac pulsation, tissue motion occurs radially and non-rigidly. To obtain an accurate motion field, we developed a variational functional integrating the optical flow equation and an anisotropic regularizer, and designed a diffusion tensor to encourage coherence diffusion. The motion field is smoothed along the desired motion flow orientation. A fast, additive operator splitting scheme, which is ten times faster than the conventional discrete method, is used for the numerical implementation. To demonstrate the efficiency of the proposed approach, finite element models are set up for normal and pathological carotid vessel walls. The results indicate that the proposed diffusion approach obtains an accurate smooth and continuous motion field and greatly improves the follow up strain estimation using a fast differential strain filter. Furthermore, our approach using the Von Mises strain imaging on clinical ultrasound images of the carotid artery is validated. Participants above 65-years in age suffering from different stages of atherosclerosis in their carotid artery are selected. The results are evaluated by an experienced physician. The evaluation results demonstrate that the Von Mises strain has a good correspondence to the presence of certain suspicious areas in the B-mode images. The proposed method is therefore clinically applicable for the real-time Von Mises strain imaging of carotid vessel walls, and can be of great value as a complementary method to B-mode image for the clinical identification of the risk of plaque vulnerability.

OpenOpticalFlow: An Open Source Program for Extraction of Velocity Fields from Flow Visualization Images

This paper describes OpenOpticalFlow, an open source optical flow program in Matlab for extraction of high-resolution velocity fields from various flow visualization images. This program is a useful tool for researchers to use the optical flow method in various flow measurements. The principles of the optical flow method are concisely described, including the physics-based optical flow equation, the variational solution, and errors. The central part of this paper is the descriptions of the main program, relevant subroutines and selection of the relevant parameters in optical flow computation. Examples are given to demonstrate the applications of the optical flow method.

Vision-based target motion analysis and collision avoidance of unmanned surface vehicles

This article proposes a vision-based target motion analysis and collision avoidance method for unmanned surface vehicles. To estimate the relative position of the target, the calibrated camera model and camera height constraint are used. In addition, the optical flow equation is adopted to estimate the relative velocity of the target. The estimated relative position and velocity are then integrated using the Kalman filter in order to reduce the effect of measurement noise. Once the kinematic information of the target is determined by the vision-based target motion analysis, the collision risk is calculated by a fuzzy estimator. Based on the collision risk, vision-based collision avoidance control is performed. To validate the effectiveness of the suggested method in various encounter situations, a collision avoidance simulation is conducted.

Video Extrapolation Method Based on Time-Varying Energy Optimization and CIP.

Video extrapolation/prediction methods are often used to synthesize new videos from images. For fluid-like images and dynamic textures as well as moving rigid objects, most state-of-the-art video extrapolation methods use non-physics-based models that learn orthogonal bases from a number of images but at high computation cost. Unfortunately, data truncation can cause image degradation, i.e., blur, artifact, and insufficient motion changes. To extrapolate videos that more strictly follow physical rules, this paper proposes a physics-based method that needs only a few images and is truncation-free. We utilize physics-based equations with image intensity and velocity: optical flow, Navier-Stokes, continuity, and advection equations. These allow us to use partial difference equations to deal with the local image feature changes. Image degradation during extrapolation is minimized by updating model parameters, where a novel time-varying energy balancer model that uses energy based image features, i.e., texture, velocity, and edge. Moreover, the advection equation is discretized by high-order constrained interpolation profile for lower quantization error than can be achieved by the previous finite difference method in long-term videos. Experiments show that the proposed energy based video extrapolation method outperforms the state-of-the-art video extrapolation methods in terms of image quality and computation cost.

Contour Tracking with a Spatio-Temporal Intensity Moment.

Standard edge detection operators such as the Laplacian of Gaussian and the gradient of Gaussian can be used to track contours in image sequences. When using edge operators, a contour, which is determined on a frame of the sequence, is simply used as a starting contour to locate the nearest contour on the subsequent frame. However, the strategy used to look for the nearest edge points may not work when tracking contours of non isolated gray level discontinuities. In these cases, strategies derived from the optical flow equation, which look for similar gray level distributions, appear to be more appropriate since these can work with a lower frame rate than that needed for strategies based on pure edge detection operators. However, an optical flow strategy tends to propagate the localization errors through the sequence and an additional edge detection procedure is essential to compensate for such a drawback. In this paper a spatio-temporal intensity moment is proposed which integrates the two basic functions of edge detection and tracking.