Optical Flow Method Research Articles

Computer-Vision-Aided Deflection Influences Line Identification of Concrete Bridge Enhanced by Edge Detection and Time-Domain Forward Inference

To enhance the accuracy and efficiency of the deflection response measurement of concrete bridges with a non-contact scheme and address the ill-conditioned nature of the inverse problem in influence line (IL) identification, this study introduces a computer-vision-aided deflection IL identification method that integrates edge detection and time-domain forward inference (TDFI). The methodology proposed in this research leverages computer vision technology with edge detection to surpass traditional contact-based measurement methods, greatly enhancing the operational efficiency and applicability of IL identification and, in particular, addressing the challenge of accurately measuring small deflections in concrete bridges. To mitigate the limitations of the Lucas–Kanade (LK) optical flow method, such as unclear feature points within the camera’s field of view and occasional point loss in certain video frames, an edge detection technique is employed to identify maximum values in the first-order derivatives of the image, creating virtual tracking points at the bridge edges through image processing. By precisely defining the bridge boundaries, only the essential structural attributes are preserved to enhance the reliability of minimal deflection deformations under vehicular loads. To tackle the ill-posed nature of the inverse problem, a TDFI model is introduced to identify IL, recursively capturing the static bridge response generated by the bridge under the influence of successive axles of a multi-axle vehicle. The IL is then computed by dividing the response by the weight of the preceding axle. Furthermore, an axle weight ratio reduction coefficient is proposed to mitigate noise amplification issues, ensuring that the weight of the preceding axle surpasses that of any other axle. To validate the accuracy and robustness of the proposed method, it is applied to numerical examples of a simply supported concrete beam, indoor experiments on a similar beam, and field tests on a three-span continuous concrete beam bridge.

Motion process and failure mode of Pusa landslide, China based on dense optical flow method

ABSTRACT Unmanned aerial vehicle (UAV) is widely applied in geohazard monitoring with its high resolution and feasibility features, where UAV videos can record the whole landslide failure process. Previous studies can only provide the qualitative depiction of the landslide failure process, rather than the quantitative motion process, which makes it difficult to analyse the landslide dynamic characteristics and failure mode. In this letter, we take UAV video of Pusa landslide, China obtained on 28 August 2017 to retrieve the landslide motion process by using the dense optical flow method. Firstly, the landslide boundary can be clearly depicted by the cumulative landslide motion time series maps. Secondly, the maximum cumulative motion distance of the Pusa landslide was located in the middle of the slope body, with the maximum distance as 79 m, while the maximum instantaneous motion rate was around 42 m/s in the middle and lower part of the slope body. Finally, the failure mode of Pusa landslide can be characterized as ‘fissure zone in the middle, fissure extension to the top, rock bulges outward in the middle and landslide failure to high-speed runout debris flow’.

MorphFlow: Estimating Motion in In-Situ Tests of Concrete

Abstract Background In situ Computed Tomography is a valuable tool to investigate failure mechanics of materials in 3D. For brittle materials with sudden fracture like concrete however, state-of-the-art methods such as Digital Volume Correlation fail to produce displacement fields that display the discontinuous behavior of load induced cracking correctly. Objective The main objective is to develop an algorithm that calculates displacement fields for large-scale in situ experiments on concrete. Methods The algorithm presented is based on a 3D Optical Flow method solved by a primal-dual procedure and equipped with a coarse-to-fine scheme based on morphological wavelets. The algorithm is publicly available. Our evaluation focuses on the beneficial use of morphological wavelets over classical ones, and on the ability to produce reliable results with limited data. Applying the primal-dual scheme to in situ tests and using morphological wavelets are novel contributions. Results The results show that our algorithm cannot only cope with large volume images, but also produces discontinuous displacement fields that yield high strain in fractured regions. It does not only perform better than state-of-the-art methods, but also achieves sufficient results on reduced data. The morphological wavelets play a key role in this finding - they even allow to deduce cracks of widths less than a voxel. Conclusion Displacement calculation for in situ tests of brittle materials requires voxel-accurate displacement fields that allow for discontinuities. The presented algorithm fulfills these requirements and therefore is a powerful tool for future understanding of failure mechanics in concrete.

Observation-based quantification of aerosol transport using optical flow: A satellite perspective to characterize interregional transport of atmospheric pollution

Interregional transport plays a significant role in haze formation with varying and disputable contribution extent. Current research on quantitatively analyzing interregional atmospheric pollution transport has mainly relied on meteorological and chemical models. However, these models are typically affected by uncertainties due to the assumptions and simplifications inherent in the numerical simulations and source emission estimations. In this study, a comprehensive optical flow framework is developed to offer a new perspective on quantitative characterization of interregional transport of atmospheric pollution based on synergistic observations from geostationary and sun-synchronous satellites. In this framework, the high-frequency continuous aerosol observing images are regarded as video in computer vision, and an aerosol dynamic optical flow algorithm is proposed by incorporating aerosol-specific assumptions and constraints, overcoming the limitation that traditional optical flow methods are typically confined to rigid bodies. Results demonstrate that the developed optical flow framework could distinguish the aerosol transport process from other dynamic processes of aerosol development and accurately capture the fast-changing details of transport processes. Moreover, the satellite-based optical flow framework achieves aerosol transport results comparable to those of widely accepted model-based methods, demonstrating the physical interpretation of pixel-based optical flow results and highlighting its effectiveness in quantitative characterization of the atmospheric pollution transport process via the Aerosol Transport Index (ATI). Furthermore, a case analysis of long-term assessments of interregional transport of atmospheric pollution indicates that Beijing acts as a “sink” of atmospheric pollution, and a downward trend could be found from the annually averaged transported aerosol net loadings due to the emission reduction policy. Compared with model-based methods, the satellite-based optical flow framework is directly grounded in observations and does not rely on emission inventories that take years to update. Therefore, it not only helps improve understanding the patterns of atmospheric pollution interregional transport, but also provides a more efficient and economical way to assess the effectiveness of regional joint control policy.

Quantifying Uncertainty in Atmospheric Winds Retrieved from Optical Flow: Dependence on Weather Regime

Abstract This study explores the performance of a dense optical flow method in comparison to pattern-matching techniques for retrieving atmospheric motion vectors (AMVs) from water vapor images. Using high-resolution simulated datasets that represent various weather phenomena, we assess the performance of these methods across different weather regimes, time intervals, and pressure levels and quantify the uncertainties associated with retrieved winds. The optical flow algorithm consistently outperforms the feature matching approach. Notably, it produces wind speeds and AMVs that closely resemble the wind fields from the simulations, and unlike the feature matching algorithm, the optical flow algorithm exhibits consistent performance across different time intervals. In contrast, the feature matching approach yields vector fields that exhibit oversmoothing in certain areas and erratic behavior in others, while also producing less detailed, regionally constant speed maps. Furthermore, unlike feature matching, the optical flow method effectively calculates AMV near regions with missing data, generating a dense AMV field for every pixel in a pair of images. This superior performance and flexibility significantly influence the planning for future satellite missions aimed at retrieving atmospheric winds. As such, our work plays a critical role in determining the mission architecture and projected instrument performance for future atmospheric wind retrieval satellite missions. The study underscores the potential of the optical flow algorithm as a robust and efficient approach for atmospheric motion retrieval, thus contributing to advances in climate research and weather prediction. Significance Statement This research investigates the efficacy of two methods, optical flow and feature matching, for detecting atmospheric winds, referred to as atmospheric motion vectors, from satellite images of water vapor. Employing detailed simulated datasets that replicate real-world weather patterns, we found that optical flow consistently outperforms feature matching in various aspects. Notably, the optical flow method is not only more precise but also maintains its accuracy across different scenarios. These insights are critical for the design of future satellite missions focused on advancing our understanding of the atmosphere and enhancing weather predictions. This study contributes to advancements in climate research and supports improved weather forecasting, benefiting both scientific and societal needs.

DZ-SLAM: A SAM-based SLAM algorithm oriented to dynamic environments

Precise localization is a fundamental prerequisite for the effective operation of Simultaneous Localization and Mapping (SLAM) systems. Traditional visual SLAM is based on static environments and therefore performs poorly in dynamic environments. While numerous visual SLAM methods have been proposed to address dynamic environments, these approaches are typically based on certain prior knowledge. This paper introduces DZ-SLAM, a dynamic SLAM algorithm that does not require any prior knowledge, based on ORB-SLAM3, to handle unknown dynamic elements in the scene. This work first introduces the FastSAM to enable comprehensive image segmentation. It then proposes an adaptive threshold-based dense optical flow approach to identify dynamic elements within the environment. Finally, combining FastSAM with optical flow method and embedding it into the SLAM framework to eliminate dynamic objects and improve positioning accuracy in dynamic environments. The experiment shows that compared with the original ORB-SLAM3 algorithm, the algorithm proposed in this paper reduces the absolute trajectory error by up to 96%; Compared to the most advanced algorithms currently available, the absolute trajectory error of our algorithm can be reduced by up to 46%. In summary, the proposed dynamic object segmentation method without prior knowledge can significantly reduce the positioning error of SLAM algorithm in various dynamic environments.

Novel intelligent defects detection of boiler water walls in thermal power plants based on OFM_SSD

The boiler is a critical component of conventional thermal power systems, where surface flaws in boiler water walls can significantly compromise safety and availability, potentially leading to substantial loss of life and property. Traditional detection methods, whether manual or based on machine learning, often prove costly, inefficient and time-consuming, failing to meet the stringent requirements for water wall inspection. Therefore, a novel surface defect detection model integrating an improved single shot multibox detector (SSD) with the optical flow method (OFM) (OFM_SSD) is proposed. The OFM enhances data sample diversity by augmenting the dataset derived from thermal power plants, while the incorporation of deconvolution techniques improves the model receptive field, thereby enhancing its ability to detect and classify small defects. Comprehensive experiments demonstrate that the OFM_SSD outperforms several existing models including the SSD model based on traditional expanded datasets (T_SSD), you only look once (YOLO), ordinary SSD, Regions with the CNN(R_CNN), and Deconvolution-only SSD (DSSD) in terms of accuracy in defect localization and classification. This advancement of the OFM_SSD not only reduces operational costs but also enhances detection capabilities, ultimately contributing to safer and more efficient operations within thermal power plants.

TEDR: A spatiotemporal attention radar extrapolation network constrained by optical flow and distribution correction

In recent years, deep learning has been widely applied to meteorological radar extrapolation due to the shortcomings of traditional optical flow methods in predicting the genesis and dissipation of radar echoes. However, it still faces challenges in addressing issues of clarity and overall intensity attenuation caused by uncertainty. This study implemented a dual-path spatiotemporal attention network that integrates optical flow techniques by employing intra-frame static attention and inter-frame dynamic attention, which could simulate motion fields and the overall intensity distribution of radar echoes separately. Our approach effectively resolve the issues of systematic intensity attenuation and clarity degradation introduced by deep learning methods. Through the comparisons of key metrics such as MSE, SSIM, CSI20, CSI30, and CSI40, the results demonstrated significant improvements over traditional approaches, particularly in CSI30 and CSI40, where the metrics improved by more than 35 %.

Assessment of Carotid Artery Vibrations by Using Optical Flow Methods on Ultrasound Images and Optical Flow

Purpose: This study aims to extract carotid wall vibrations non-invasively and evaluate changes in the carotid artery caused by age, BMI, and sex. Such evaluation can increase the possibility of detecting wall stiffness and atherosclerosis in the early stages and prevent heart attack and death. Materials and Methods: To extract small vibrations of the carotid wall, the image-tracking method, and optical flow with four different methods were used. The study involved twenty participants, comprising six females and fourteen males, with a mean age of 36.25 years, mean weight of 75.2 Kg, and mean BMI of 25. The posterior wall motion and vibration were extracted using ultrasound RF signals. Results: 4 optical flow methods, Gunnar-Farneback, Horn- Schunk, Lucas-Kanade, and Lucas-Kanade derivative were used for all samples, and covariance, correlation, P-value, and R-squared were estimated. Results showed the differences in parameters such as age and BMI with carotid wall vibration. These values for age are (Cov = -0.1891, Corr = -0.6770, P-value < 0.001, r2 = -0.4590) and for BMI are (Cov = -0.0564, Corr = -0.6510, P-value < 0.001, r2 = 0.4238), respectively. For gender as a new parameter, a comparison between men’s and women’s vibrations was estimated. The range of measured vibrations by optical flow methods is about 0.0002 µm to 0.1µm, and the mean standard deviation is 0.04967 µm. Conclusion: The results presented that gender affects the vibration of the vessel wall, which in men is more than in women. In addition, increasing age and BMI may increase the stiffness of the carotid wall and reduce vibrations that were evaluated previously. Using the Gunner-Farneback method as image tracking for small vibrations is the best way with the highest accuracy.

Framework for UAV-based river flow velocity determination employing optical recognition

The determination of river velocity is important for hydromorphological analyses and river monitoring systems. Indirect measurements of river velocity using videos recorded by unmanned aerial vehicles (UAV) allow fast and cost-effective processing of information about the river stretch. This paper presents a method for computing flow velocity of the river surface using deep supervised model RAFT to determine the optical flow in combination with image pre-processing by convolutional operations. Moreover, the windiness coefficients and variance score were proposed to evaluate reliability of the collected data and the obtained results of optical flow detection. Various image pre-processing techniques were applied, namely the selection of the analysed area and the number of convolutional operations to select the one with the lowest variance score. This score represents the consistency of the river flow velocity during the video and can be used to filter out unreliable results. The numerical experiments were performed using the videos and directly measured velocity values of 4 shallow rivers in Lithuania collected during the field surveys. The optical velocity estimation method showed good correspondence to the directly measured values for the velocity range from 0 m/s to 0.8 m/s in the points with low variance score up to 0.192 that represents the first quartile of the variance. The optical flow method tends to underestimate the velocity up to 0.5 m/s for the quartiles with the higher variance scores. It was shown that in most cases the lowest variance score value was obtained using pre-processing techniques without convolutional operations. However, the need to analyse various pre-processing techniques arises from the different origin of the objects moving on the river surface.

Hybrid Visual Odometry Algorithm Using a Downward-Facing Monocular Camera

The increasing interest in developing robots capable of navigating autonomously has led to the necessity of developing robust methods that enable these robots to operate in challenging and dynamic environments. Visual odometry (VO) has emerged in this context as a key technique, offering the possibility of estimating the position of a robot using sequences of onboard cameras. In this paper, a VO algorithm is proposed that achieves sub-pixel precision by combining optical flow and direct methods. This approach uses only a downward-facing, monocular camera, eliminating the need for additional sensors. The experimental results demonstrate the robustness of the developed method across various surfaces, achieving minimal drift errors in calculation.

Mix-VIO: A Visual Inertial Odometry Based on a Hybrid Tracking Strategy.

In this paper, we proposed Mix-VIO, a monocular and binocular visual-inertial odometry, to address the issue where conventional visual front-end tracking often fails under dynamic lighting and image blur conditions. Mix-VIO adopts a hybrid tracking approach, combining traditional handcrafted tracking techniques with Deep Neural Network (DNN)-based feature extraction and matching pipelines. The system employs deep learning methods for rapid feature point detection, while integrating traditional optical flow methods and deep learning-based sparse feature matching methods to enhance front-end tracking performance under rapid camera motion and environmental illumination changes. In the back-end, we utilize sliding window and bundle adjustment (BA) techniques for local map optimization and pose estimation. We conduct extensive experimental validations of the hybrid feature extraction and matching methods, demonstrating the system's capability to maintain optimal tracking results under illumination changes and image blur.

Study on the tribocharging properties of MgCO3 particles based on LFN-en-A model

Abstract As an efficient and environment-friendly method, electrostatic separation has gradually replaced flotation methods in the separation of magnesite in recent years. In the process of triboelectrostatic separation, the mineral particles are tribocharged driven by the air flow, then the trajectory is shifted under the action of the electric field, so as to realize the separation. The useful mineral in magnesite is MgCO3, but the theoretical research related to the charge characteristics of MgCO3 is not sufficient. Particle image velocimetry (PIV), as an indirect measurement technique, is able to obtain the velocity field of the fluids from images. However, the particles moving in the air have the issues such as excessive speed and small particle size, which make the traditional PIV has low accuracy in estimating the motion of particles. In this paper, a high-speed camera is used to capture the motion trajectory of tribocharged MgCO3 particles in a parallel electric field. A new optical flow method LFN-en-A network based on LiteFlowNet-en network is proposed to compute the particle motion trajectory by combining the deep learning method with the traditional PIV, which realizes the displacement estimation of particles moving in the air. It ultimately realizes the calculation of the charge-to-mass ratio on single particles. Analyzing the accuracy of the LFN-en-A network’s estimation in the experiments, the estimation of LiteFlowNet-en was compared. Changing the shooting frame rate analyzes the optimal one required by the LFN-en-A network. Combining the estimation results of LFN-en-A to calculate the particle charge-to-mass ratio (Q/m), the Q/m of MgCO3 particle was analyzed by changing the experimental conditions in the process of particles’ tribocharging, which provided a new method for particle-to-charge ratio measurement.

Enhanced river suspended sediment concentration identification via multimodal video image, optical flow, and water temperature data fusion

Monitoring suspended sediment concentration (SSC) in rivers is pivotal for water quality management and sustainable river ecosystem development. However, achieving continuous and precise SSC monitoring is fraught with challenges, including low automation, lengthy measurement processes, and high cost. This study proposes an innovative approach for SSC identification in rivers using multimodal data fusion. We developed a robust model by harnessing colour features from video images, motion characteristics from the Lucas–Kanade (LK) optical flow method, and temperature data. By integrating ResNet with a mixed density network (MDN), our method fused the image and optical flow fields, and temperature data to enhance accuracy and reliability. Validated at a hydropower station in the Xinjiang Uygur Autonomous Region, China, the results demonstrated that while the image field alone offers a baseline level of SSC identification, it experiences local errors under specific conditions. The incorporation of optical flow and water temperature information enhanced model robustness, particularly when coupling the image and optical flow fields, yielding a Nash–Sutcliffe efficiency (NSE) of 0.91. Further enhancement was observed with the combined use of all three data types, attaining an NSE of 0.93. This integrated approach offers a more accurate SSC identification solution, enabling non-contact, low-cost measurements, facilitating remote online monitoring, and supporting water resource management and river water–sediment element monitoring.

P-125 Relationship between ooplasmic fluidity and oocyte degeneration examined by image analysis after Piezo-intracytoplasmic sperm injection (ICSI)

Abstract Study question Can image analysis of ooplasmic fluidity in Piezo-ICSI predict unintentional membrane rupture (UMR) and subsequent oocyte degeneration? Summary answer Image analysis of ooplasmic fluidity using “optical flow” can be used to predict UMR and subsequent oocyte degeneration in Piezo-ICSI. What is known already UMR during Piezo-ICSI sometimes leads to oocyte degeneration, but not all oocytes degenerate. Leakage of ooplasm is often observed in degenerating oocytes after Piezo-ICSI. We speculated that the occurrence of ooplasmic leakage could relate to the ooplasmic fluidity. Optical flow is an image processing algorithm that analyses the movement of an object in a motion picture. In this study, the motion vectors for all pixels in the image were calculated using the dense optical flow method. Study design, size, duration We developed an image analysis system that detects the movement of the ooplasm on Piezo-ICSI videos using the OpenCV library. Videos of 96 oocytes recorded during Piezo ICSI (mean maternal age: 39.2±5.4 years) retrieved in our institute between January 2020 and September 2023 were retrospectively analyzed using the system. To minimize the potential influence of technical factors, Piezo-ICSI was performed by two embryologists. Participants/materials, setting, methods The oocytes were characterized by survival or degeneration and labeled with and without UMR. To evaluate the ooplasm fluidity, we analyzed the vertical motion vectors of all pixels between the start to puncture and the rupture of the oolemma (1st period) and between sperm injection and withdrawal of the pipette (2nd period). The velocity between groups was compared at the 95th, 99th, and 99.5th percentiles, followed by statistical analysis by the Mann-Whitney U test. Main results and the role of chance The 95th, 99th, and 99.5th percentiles of the survival group (n = 75) were significantly higher than those of the degeneration group (n = 21) in the 1st period (P < 0.01). Additionally, the 99th, and 99.5th percentiles of the survival group were significantly higher than those of the degenerative group in the 2nd period (P < 0.05). The 95th, 99th, and 99.5th percentiles without UMR (n = 52) were significantly higher than those with UMR (n = 44) in the 1st period (P < 0.001). Similarly, the 95th, 99th, and 99.5th percentiles without UMR were significantly higher than those with UMR in the 2nd period (P < 0.001). ROC curves were generated at the 95th, 99th, and 99.5th percentiles in the 1st period. The resultant areas under the curve (AUC) were 0.73, 0.75, and 0.73, respectively. Using a threshold of 0.75 for the 99th percentile, oocyte degeneration was predicted with a sensitivity of 85% and specificity of 57%. In the 2nd period, ROC curves at the same percentiles resulted in the AUCs of 0.63, 0.69, and 0.67, respectively. Using a threshold of 0.69 for the 99th percentile, oocyte degeneration was predictable with sensitivity as 57% and specificity as 86%. Limitations, reasons for caution This study did not control for variations among the embryologists performing Piezo-ICSI. The culture outcomes and clinical outcomes had not been evaluated. We only showed the relationship between ooplasmic fluidity and oocyte degeneration because it was a retrospective study. A prospective study using image analysis is needed. Wider implications of the findings Before the investigation, we expected that more ooplasmic fluidity (less viscosity) could increase UMR and oocyte degeneration. This study, however, indicated that more ooplasmic fluidity (less viscosity) decreased UMR and oocyte degeneration. Elucidation of the mechanism under this phenomenon may lead to reduced UMR and oocyte degeneration. Trial registration number Not Applicable

Semantic Visual SLAM Algorithm Based on Improved DeepLabV3+ Model and LK Optical Flow

Aiming at the problem that dynamic targets in indoor environments lead to low accuracy and large errors in the localization and position estimation of visual SLAM systems and the inability to build maps containing semantic information, a semantic visual SLAM algorithm based on the semantic segmentation network DeepLabV3+ and LK optical flow is proposed based on the ORB-SLAM2 system. First, the dynamic target feature points are detected and rejected based on the lightweight semantic segmentation network DeepLabV3+ and LK optical flow method. Second, the static environment occluded by the dynamic target is repaired using the time-weighted multi-frame fusion background repair technique. Lastly, the filtered static feature points are used for feature matching and position calculation. Meanwhile, the semantic labeling information of static objects obtained based on the lightweight semantic segmentation network DeepLabV3+ is fused with the static environment information after background repair to generate dense point cloud maps containing semantic information, and the semantic dense point cloud maps are transformed into semantic octree maps using the octree spatial segmentation data structure. The localization accuracy of the visual SLAM system and the construction of the semantic maps are verified using the widely used TUM RGB-D dataset and real scene data, respectively. The experimental results show that the proposed semantic visual SLAM algorithm can effectively reduce the influence of dynamic targets on the system, and compared with other advanced algorithms, such as DynaSLAM, it has the highest performance in indoor dynamic environments in terms of localization accuracy and time consumption. In addition, semantic maps can be constructed so that the robot can better understand and adapt to the indoor dynamic environment.

Experimental study on wavefront flow characteristics of dam-break wave at initial stage on wet bed

The flow characteristics of dam-break wave in the initial stage of downstream wet bed are studied experimentally by digital image measurement technologies. First, the fine wavefront structure and its velocity were captured by the optical flow method, and an image measurement technology of water level based on edge detection was proposed. Then, the comparison and verification were carried out using the numerical simulation. The mean error is −7.369%, −1.243%, and 1.317% under depth ratio (σ) is 0.2, 0.25, and 0.33, and the error of most cases is within ±15% except σ = 0.2. The results show that large eddy simulation combined with volume of fluid method could accurately predict the distribution of dam-break water level, but it tends to overestimate the propagation velocity of the wavefront by about 10.3%. In addition, Stoker's quasi-steady paradigm has been proven to accurately predict the mean and steady-state flow characteristics of dam-break wave. Furthermore, the wavefront structure of the initial stage was subdivided into three sub-stages, namely, the jump stage, the transition stage, and the stable stage. Following that, the flow characteristics of each stage under the condition of the σ = 0.25 were studied in detail. The results show that the morphology of the wavefront structure is driven by the transformation of its internal energy in the initial stage. In summary, the work reveals the flow characteristics and quantitative flow results of the initial stage of dam-break wave under the wet river bed, thus improving the accuracy of dam-break accident prediction.

Dense Continuous-Time Optical Flow From Event Cameras.

We present a method for estimating dense continuous-time optical flow from event data. Traditional dense optical flow methods compute the pixel displacement between two images. Due to missing information, these approaches cannot recover the pixel trajectories in the blind time between two images. In this work, we show that it is possible to compute per-pixel, continuous-time optical flow using events from an event camera. Events provide temporally fine-grained information about movement in pixel space due to their asynchronous nature and microsecond response time. We leverage these benefits to predict pixel trajectories densely in continuous time via parameterized Bézier curves. To achieve this, we build a neural network with strong inductive biases for this task: First, we build multiple sequential correlation volumes in time using event data. Second, we use Bézier curves to index these correlation volumes at multiple timestamps along the trajectory. Third, we use the retrieved correlation to update the Bézier curve representations iteratively. Our method can optionally include image pairs to boost performance further. To the best of our knowledge, our model is the first method that can regress dense pixel trajectories from event data. To train and evaluate our model, we introduce a synthetic dataset (MultiFlow) that features moving objects and ground truth trajectories for every pixel. Our quantitative experiments not only suggest that our method successfully predicts pixel trajectories in continuous time but also that it is competitive in the traditional two-view pixel displacement metric on MultiFlow and DSEC-Flow. Open source code and datasets are released to the public.

Cross-correlation-based convolutional neural network with velocity regularization for high-resolution velocimetry of particle images

Particle image velocimetry (PIV) stands as a pivotal experimental technique in fluid dynamics, enabling the visualization and analysis of fluid flows. Traditional methods for extracting velocity fields from particle images often rely on window-cross correlation PIV or, more recently, optical flow techniques rooted in intensity conservation principles. However, the former approach suffers from low resolution, whereas the latter is hampered by computational inefficiency and a high susceptibility to noise. Recent studies have demonstrated the effectiveness of convolutional neural networks (CNNs) in processing particle images to obtain high-resolution and high-accuracy velocity fields, though traditional CNN architectures are still not satisfying in accuracy. The present study introduces an enhanced network, En-FlowNetC, based on the cross correlation-based CNN FlowNetC, specifically designed to process PIV particle images and achieve high-accuracy, high-resolution velocity fields. It incorporates a velocity regularization and is trained and validated on canonical datasets. The results indicate that En-FlowNetC surpasses traditional CNN networks in accuracy and markedly outperforms the classic Horn–Schunck optical flow method in both complex and simple flow scenarios. Furthermore, this study confirms the beneficial impact of velocity regularization, when judiciously applied, on network accuracy. The proposed modifications compared to the original FlowNetC are also examined in the ablation experiments. Overall, En-FlowNetC provides an effective deep-learning solution for PIV analysis, paving the way for future endeavors aimed at achieving highly accurate and resolved velocimetry.

Structural displacement measurement using deep optical flow and uncertainty analysis

Displacement serves as a crucial indicator in the field of structure health monitoring for assessing the condition of civil engineering structures. Computer vision is commonly employed for this purpose due to its cost-effectiveness and convenience. Traditional non-contact computer vision methods, such as optical flow, have been applied to obtain the displacement of bridges. However, real-time monitoring remains challenging due to the computational complexity involved. Therefore, optical flow based on deep learning (referred to as deep optical flow) has gained significant attention. Its main advantage lies in its ability to achieve real-time displacement monitoring, which is of great significance. Furthermore, the accuracy of deep optical flow is comparable to that of traditional optical flow methods for displacement measurement. In this paper, the deep optical flow, RAFT-GOCor, is proposed and applied to extract the displacement of structures, then Bayesian RAFT-GOCor based on Monte Carlo dropout is also presented and applied to analyze the uncertainty of the displacement. The algorithms are verified by laboratory simulated experiments and field bridge loading test. The results indicate that both deep optical flow and uncertainty analysis are feasible methods for measuring structural displacement.