Optical Flow Technology Research Articles

Motion Decoupling Network for Intra-Operative Motion Estimation Under Occlusion.

In recent intelligent-robot-assisted surgery studies, an urgent issue is how to detect the motion of instruments and soft tissue accurately from intra-operative images. Although optical flow technology from computer vision is a powerful solution to the motion-tracking problem, it has difficulty obtaining the pixel-wise optical flow ground truth of real surgery videos for supervised learning. Thus, unsupervised learning methods are critical. However, current unsupervised methods face the challenge of heavy occlusion in the surgical scene. This paper proposes a novel unsupervised learning framework to estimate the motion from surgical images under occlusion. The framework consists of a Motion Decoupling Network to estimate the tissue and the instrument motion with different constraints. Notably, the network integrates a segmentation subnet that estimates the segmentation map of instruments in an unsupervised manner to obtain the occlusion region and improve the dual motion estimation. Additionally, a hybrid self-supervised strategy with occlusion completion is introduced to recover realistic vision clues. Extensive experiments on two surgical datasets show that the proposed method achieves accurate motion estimation for intra-operative scenes and outperforms other unsupervised methods, with a margin of 15% in accuracy. The average estimation error for tissue is less than 2.2 pixels on average for both surgical datasets.

Automated measurement of endometrial peristalsis in cine transvaginal ultrasound images.

Objectives: Endometrial peristalsis (EP) in non-pregnant uterine can be assessed by visual assessment of transvaginal ultrasound (TVUS). However, visual assessment is subjective, and the outcome depends on the sonographers and video analysts. This study aimed to create a newly developed automatic analysis algorithm for measuring the EP compared to visual assessment. Methods: A retrospective analysis was performed using the datasets from in vitro fertilization and embryo transfer (IVF-ET), who underwent the evaluation of EP by TVUS within 5 days prior to transplantation. 158 cine TVUS images were used to develop the automated analysis algorithm, and 37 cine TVUS images were evaluated by both visual and automated analysis algorithms. The algorithm was developed by applying the optical flow technology and enabled objective analysis of the number, direction, and intensity of EP. Results: The number of peristaltic waves counted by visual assessment was 4.2 ± 2.3 (mean ± standard deviation) and 4.1 ± 2.1 for doctors one and two, respectively. The number of waves counted with the algorithm was 3.6 ± 2.1 at first evaluation and 3.7 ± 2.0 at repeated evaluation. A significant difference was found between the algorithm count and visual assessment (p = 0.001, 0.002, 0.003, 0.008). The ICC values for algorithm versus manuals ranged from 0.84 to 0.96 and 0.87 to 0.96. The numbers of the cervix-to-fundus (CF), fundus-to-cervix (FC), and both cervix-to-fundal and fundus-to-cervix (CF + FC) directions of EP counted by the algorithm were 50, 52, and 32, respectively. The numbers counted by visual assessment were 43, 45, and 46, respectively. The number of EP was the same in 87% of the two algorithm counts. The number was lower between the algorithm and visual analysis (79% with complete agreement). The EP intensity assessed by the algorithm was 2.6 ± 1.1, and the peristalsis velocity was 0.147 (0.07) mm/s. Conclusion: The fully automated analysis algorithm can be used to quantify uterine peristalsis comparable to visual assessment.

Learning-based risk assessment and motion estimation by vision for unmanned aerial vehicle landing in an unvisited area

We proposed a vision-based methodology as an aid for an unmanned aerial vehicle (UAV) landing on a previously unsurveyed area. When the UAV was commanded to perform a landing mission in an unknown airfield, the learning procedure was activated to extract the surface features for learning the obstacle appearance. After the learning process, while hovering the UAV above the potential landing spot, the vision system would be able to predict the roughness value for confidence in a safe landing. Finally, using hybrid optical flow technology for motion estimation, we successfully carried out the UAV landing without a predefined target. Our work combines a well-equipped flight control system with the proposed vision system to yield more practical versatility for UAV applications.

A two-dimension velocity field measurement method for the thermal smoke basing on the optical flow technology

The confine and heat are remarkable features in building fire research, as well as is a barrier for flow velocity profiles measurement, which the general method is unsuitable for this experimental environment. Therefore, this paper develops a measurement method for the acquisition of the thermal smoke flow velocity profiles in the fire building experiment. Firstly, we utilize the smoke itself particle distribution to replace the manual track particles in which reduce the complexity of the measurement method for fire experiments system. Secondly, we optimize the non-uniform distribution of smoke soot particle and utility the smoke stratification characteristics basing on the optical flow technology. The four optimized methods were compared to acquire reasonable smoke velocity distributions. Finally, we take advantage of filter post-process method to smooth the smoke velocity profiles in the main transportation direction. Moreover, a series of experiments and simulations show the feasibility of this image-based method in which the pre-process, optical calculation and smooth methods were established to denote reasonably the smoke flow velocity gradient and inclined upward trend nearby the corridor open. Meanwhile, it is not only in acquiring two-dimension smoke field velocity, while also can fulfill the other particle flow movement.

Inter-vendor reproducibility and accuracy of segmental left ventricular strain measurements using CMR feature tracking.

Our aim was to evaluate the inter-vendor reproducibility of cardiovascular MR feature tracking (CMR-FT) for the measurement of segmental strain (SS) of the left ventricle (LV) as well as to test the accuracy of CMR-FT to detect regional myocardial pathology. We selected 45 patients: 15 with normal CMR findings, 15 with dilated cardiomyopathy, and 15 with acute myocardial infarction. Segmental longitudinal, circumferential, and radial strains were assessed with 4 different software. The inter-vendor difference as well as intra- and inter-observer variability was investigated. Furthermore, the accuracy of CMR-FT for the detection of structural (infarcted segments) as well as functional pathology (septal vs. lateral wall strain in left bundle branch block) was tested. Between vendors, there were significant differences in values for all strains (p < 0.001). The software using a non-rigid algorithm for image registration and segmentation demonstrated the best intra- as well as inter-observer variability with interclass correlation coefficient (ICC) > 0.962 and coefficient of variation (CV) < 24%. For infarct location, the same software yielded the highest area under the curve values for radial and circumferential SS (0.872 and 0.859, respectively). One of the other three software using optical flow technology performed best for longitudinal SS (0.799) and showed the largest differences in SS between septum and lateral wall in the dilated cardiomyopathy group. SS values obtained by CMR-FT are not interchangeable between vendors, and intra- and inter-observer reproducibility shows substantial variability among vendors. Overall, the different packages score relatively well to depict focal structural or functional LV pathology. • Segmental myocardial strain values obtained by CMR feature tracking are not interchangeable between different vendors. • Intra- and inter-observer reproducibility shows substantial variability among vendors. • Segmental myocardial strains measured by CMR feature tracking score relatively well to depict focal structural or functional LV pathology.

Motion Estimation by Hybrid Optical Flow Technology for UAV Landing in an Unvisited Area.

The capability of landing on previously unvisited areas is a fundamental challenge for an unmanned aerial vehicle (UAV). In this paper, we developed a vision-based motion estimation as an aid to improve landing performance. As an alternative to the common scenarios accompanying by external infrastructures or well-defined marker, the proposed hybrid framework can successfully land on a new area without any prior information about guiding marks. The implementation was based on the optical flow technique associated with a multi-scale strategy to overcome the decreasing field-of-view during the UAV descending. Compared with a commercial Global Positioning System (GPS) through a sequence of flight trials, the vision-aided scheme can effectively minimize the possible sensing error, thus, leading to a more accurate result. Moreover, this work has potential to integrate the fast-growing image learning process and yields more practical versatility for UAV applications in the future.

A Study on the Estimation of Smartphone Movement Distance using Optical Flow Technology on a Limited Screen

A Study on the Estimation of Smartphone Movement Distance using Optical Flow Technology on a Limited Screen

Method of Image Quality Improvement for Atmospheric Turbulence Degradation Sequence Based on Graph Laplacian Filter and Nonrigid Registration

It is challenging to restore a clear image from an atmospheric degraded sequence. The main reason for the image degradation is geometric distortion and blurring caused by turbulence. In this paper, we present a method to eliminate geometric distortion and blur and to recover a single high-quality image from the degraded sequence images. First, we use optical flow technology to register the sequence images, thereby suppressing the geometric deformation of each frame. Next, sequence images are summed by a temporal filter to obtain a single blurred image. Then, the graph Laplacian matrix is used as the cost function to construct the regularization term. The final clear image and point spread function are obtained by iteratively solving the problem. Experiments show that the method can effectively eliminate the distortion and blur, restore the image details, and significantly improve the image quality.

Weekly Glacier Flow Estimation from Dense Satellite Time Series Using Adapted Optical Flow Technology

Contemporary optical remote sensing satellites or constellations of satellites can acquire imagery at sub-weekly or even daily timescales. Thus, these systems facilitate the potential for within-season velocity estimation of glacier surfaces. State-of-the-art techniques for displacement estimation are based on matching image pairs and are thus constrained by the need of significant displacement and/or preservation of the surface over time. Consequently, such approaches cannot benefit entirely from the increasing satellite revisit times. Here, we explore an approach that is fundamentally different from image correlation or similar techniques and exploits the concept of optical flow. Our goal is to assess if this concept could overcome above current limitations of image matching and thus give new insights in glacier flow dynamics. We implement two different methods of optical flow, and test these on the SPOT5 Take5 dataset over Kronebreen, Svalbard and over Kaskawulsh Glacier, Yukon. For Kaskawulsh Glacier we are able to extract seasonal velocity variation, that temporally coincide with events of increased air temperatures. Furthermore, even for the cloudy dataset of Kronebreen, we were able to extract spatio-temporal trajectories which correlate well with measured GPS flow paths. Because the underlying concept is simple and computationally efficient due to data-reduction, our methodology can easily be used for exploratory regional studies of several glaciers or estimation of small and slow flowing glaciers.

Moving Target Tracking Algorithm Based on Improved Optical Flow Technology

A method of moving target tracking based on optical flow technology is proposed in this paper. Extract the fea- ture points of the moving target by the SIFT algorithm, and replace the interesting points in traditional optical flow meth- od with the extracted feature points, then track the moving target through these points. The proposed moving target track- ing method has better tracking precision for the SIFT algorithm is invariant to rotation and scale change of moving target .At the same time, the optical flow estimation is obtained using the pyramid LK optical flow algorithm so that the compu- tation of the tracking process is reduced, the accuracy is improved and the movement target tracking has a real-time per- formance .In this paper, three groups of experiments was conducted, a set of contrast experiments under the proposed method and the traditional optical flow method, a set of validation experiments of single moving target with some shelter and another set of validation experiments of multiple moving target with irregular shelters .The experimental results verify that the the proposed method has a good accuracy, robustness and real-time performance.

Real-time moving object detection for video monitoring systems

Moving object detection is one of the challenging problems in video monitoring systems, especially when the illumination changes and shadow exists. A method for real-time moving object detection is described. A new background model is proposed to handle the illumination varition problem. With optical flow technology and background subtraction, a moving object is extracted quickly and accurately. An effective shadow elimination algorithm based on color features is used to refine the moving objects. Experimental results demonstrate that the proposed method can update the background exactly and quickly along with the varition of illumination, and the shadow can be eliminated effectively. The proposed algorithm is a real-time one which the foundation for further object recognition and understanding of video monitoring systems.