Lucas-Kanade Optical Flow Research Articles

Volumetric 3D stitching of optical coherence tomography volumes

Abstract Optical coherence tomography (OCT) is a noninvasive medical imaging modality, which provides highresolution transectional images of biological tissue. However, its potential is limited due to a relatively small field of view. To overcome this drawback, we describe a scheme for fully automated stitching of multiple 3D OCT volumes for panoramic imaging. The voxel displacements between two adjacent images are calculated by extending the Lucas-Kanade optical flow a lgorithm to dense volumetric images. A RANSAC robust estimator is used to obtain rigid transformations out of the resulting flow v ectors. T he i mages a re t ransformed into the same coordinate frame and overlapping areas are blended. The accuracy of the proposed stitching scheme is evaluated on two datasets of 7 and 4 OCT volumes, respectively. By placing the specimens on a high-accuracy motorized translational stage, ground truth transformations are available. This results in a mean translational error between two adjacent volumes of 16.6 ± 0.8 μm (2.8 ± 0.13 voxels). To the author’s knowledge, this is the first reported stitching of multiple 3D OCT volumes by using dense voxel information in the registration process. The achieved results are sufficient for providing high accuracy OCT panoramic images. Combined with a recently available high-speed 4D OCT, our method enables interactive stitching of hand-guided acquisitions.

Subpixel-Based Precipitation Nowcasting with the Pyramid Lucas–Kanade Optical Flow Technique

Short-term high-resolution quantitative precipitation forecasting (QPF) is very important for flash-flood warning, navigation safety, and other hydrological applications. This paper proposes a subpixel-based QPF algorithm using a pyramid Lucas–Kanade optical flow technique (SPLK) for short-time rainfall forecast. The SPLK tracks the storm on the subpixel level by using the optical flow technique and then extrapolates the precipitation using a linear method through redistribution and interpolation. The SPLK compares with object-based and pixel-based nowcasting algorithms using eight thunderstorm events to assess its performance. The results suggest that the SPLK can perform better nowcasting of precipitation than the object-based and pixel-based algorithms with higher adequacy in tracking and predicting severe storms in 0–2 h lead-time forecasting.

Research on Non - contact Measurement of Open Channel Flow Based on Machine Vision

The two-frame images are captured continuously by vertically shooting the camera, and the strong angle points are used to track the velocity information. This design takes the ARM board of Banana PI (BPI-M2 Ultra) as the core, combines the processing of OpenCv visual function library, extracts and tracks the feature points by using the strong angle extraction algorithm, and uses the pyramid Lucas-Kanade optical flow method can track the feature points. The tracking of the characteristic point information is tracked to remove the distorted displacement information and calculate the velocity information using the relationship of the frame rate displacement.

Technique for two-dimensional displacement field determination using a reliability-guided spatial-gradient-based digital image correlation algorithm.

This paper proposed a novel in-plane displacement field measurement algorithm using an optical flow strategy. We built a linear illumination model between images before and after deformation to guarantee intensity invariability. We used image upsampling and a reliability-guided strategy to find the matching points accurate to 0.5pixels in the reference and deformed images. The criterion to determine the reliability is zero-mean normalized cross-correlation coefficient. Afterward, we used the brightness constancy constraint combined with Lucas-Kanade optical flow constraint in a specific image region to obtain an overdetermined linear equation. We applied the noniterative least-squares algorithm to solve the equations and to achieve the displacement offsets. This research utilized multithread calculation to handle the complete cracking applications. We estimated the computing efficiency and calculation precision of the proposed method through a series of experimental speckle patterns. All results demonstrated the correctness, effectiveness, and robustness of the proposed method.

Local Motion Intensity Clustering (LMIC) Model for Segmentation of Right Ventricle in Cardiac MRI Images.

Analysis of the morphology and function of the right ventricle (RV) can be used for the prediction and diagnosis of cardiovascular disease. Accurate description of the structure and function of heart can be provided by analyzing cardiac magnetic resonance imaging (MRI) images. Noise interference and intensity inhomogeneity of MRI images can be addressed by using a local intensity clustering (LIC) model. However, the segmentation of the RV in MRI images still remains a challenge mainly due to its ill-defined borders. To address such a challenge, an algorithm for segmenting the RV based on a local motion intensity clustering (LMIC) model is proposed in this paper. The LMIC model combines the LIC model with the motion intensity information, due to cardiac motion and blood flow. The motion intensity is calculated by using the Lucas Kanade optical flow method and utilized in the LMIC model as an energy parameter. Because the motion intensity of the RV region is stronger than other areas, the RV can be accurately segmented by this approach. Experimental results demonstrate that the LMIC model is able to address the challenge of the ill-defined RV borders in cardiac MRI images and improved RV segmentation accuracy over existing methods.

Implementation and evaluation of optical flow methods for two-dimensional deformation measurement in comparison to digital image correlation

In experimental solid mechanics optical methods allow for non-contact derivation of full-field information. In this paper, optical flow based deformation measurement routines are implemented and compared to various commonly used digital image correlation procedures. The global Horn–Schunck and the local Lucas–Kanade optical flow method are utilized and adapted for large deformation measurement. Systematic tests with synthetic images are carried out for a detailed comparison of optical flow and correlation methods for varying subset sizes, noise levels, shape functions and gray level interpolations. The synthetic test data consists of rigid translation, different sinusoidal displacements and planar rotation. Within the study, the displacement calculation and the subsequent strain calculation are discussed and analyzed separately. Furthermore, the additive effect of measurement errors, when calculating with more than two successive images is investigated. Real experimental data is processed by all implemented methods and finally compared for validation and verification purposes.

Automatic Myotendinous Junction Tracking in Ultrasound Images with Phase-Based Segmentation.

Displacement of the myotendinous junction (MTJ) obtained by ultrasound imaging is crucial to quantify the interactive length changes of muscles and tendons for understanding the mechanics and pathological conditions of the muscle-tendon unit during motion. However, the lack of a reliable automatic measurement method restricts its application in human motion analysis. This paper presents an automated measurement of MTJ displacement using prior knowledge on tendinous tissues and MTJ, precluding the influence of nontendinous components on the estimation of MTJ displacement. It is based on the perception of tendinous features from musculoskeletal ultrasound images using Radon transform and thresholding methods, with information about the symmetric measures obtained from phase congruency. The displacement of MTJ is achieved by tracking manually marked points on tendinous tissues with the Lucas-Kanade optical flow algorithm applied over the segmented MTJ region. The performance of this method was evaluated on ultrasound images of the gastrocnemius obtained from 10 healthy subjects (26.0 ± 2.9 years of age). Waveform similarity between the manual and automatic measurements was assessed by calculating the overall similarity with the coefficient of multiple correlation (CMC). In vivo experiments demonstrated that MTJ tracking with the proposed method (CMC = 0.97 ± 0.02) was more consistent with the manual measurements than existing optical flow tracking methods (CMC = 0.79 ± 0.11). This study demonstrated that the proposed method was robust to the interference of nontendinous components, resulting in a more reliable measurement of MTJ displacement, which may facilitate further research and applications related to the architectural change of muscles and tendons.

An Efficient Scan-to-Map Matching Approach Based on Multi-channel Lidar

Accurately localizing the vehicle against a pre-built high precision map is an essential step for the Autonomous Land Vehicle (ALV). This paper proposes an efficient scan-to-map matching approach based on multi-channel lidar. We firstly advocate the usage of both the lidar reflectance map and the height map, as these two maps contain complementary information. Then, borrowing ideas from the Lucas-Kanade optical flow approach, we formulate the scan-to-map matching problem in a similar form, and propose an efficient gradient descent approach to solve it. Finally, the proposed approach is integrated into a filtering framework for real-time online localization. Experiments on real-world dataset have demonstrated the validity of our approach.

Enhancing pedestrian detection using optical flow for surveillance

Optical flow can be used to segment a moving object from its backgrounds and track it. In this paper, an Enhanced Lucas-Kanade optical flow technique was used to improve human detection in terms of speed and accuracy. We combined object segmentation output with a human detector using an optical flow algorithm. The proposed technique used the optical flow to find the area of interest to complete object segmentation and use those results as an input for the human detector. This technique has been developed to be used in surveillance systems. Our experiments indicated that the proposed method was 37% faster and 118% more accurate than the standard Felzenszwalb (PFF) detector.

Enhancing pedestrian detection using optical flow for surveillance

Optical flow can be used to segment a moving object from its backgrounds and track it. In this paper, an Enhanced Lucas-Kanade optical flow technique was used to improve human detection in terms of speed and accuracy. We combined object segmentation output with a human detector using an optical flow algorithm. The proposed technique used the optical flow to find the area of interest to complete object segmentation and use those results as an input for the human detector. This technique has been developed to be used in surveillance systems. Our experiments indicated that the proposed method was 37% faster and 118% more accurate than the standard Felzenszwalb PFF detector.

Onboard Robust Visual Tracking for UAVs Using a Reliable Global-Local Object Model.

In this paper, we present a novel onboard robust visual algorithm for long-term arbitrary 2D and 3D object tracking using a reliable global-local object model for unmanned aerial vehicle (UAV) applications, e.g., autonomous tracking and chasing a moving target. The first main approach in this novel algorithm is the use of a global matching and local tracking approach. In other words, the algorithm initially finds feature correspondences in a way that an improved binary descriptor is developed for global feature matching and an iterative Lucas–Kanade optical flow algorithm is employed for local feature tracking. The second main module is the use of an efficient local geometric filter (LGF), which handles outlier feature correspondences based on a new forward-backward pairwise dissimilarity measure, thereby maintaining pairwise geometric consistency. In the proposed LGF module, a hierarchical agglomerative clustering, i.e., bottom-up aggregation, is applied using an effective single-link method. The third proposed module is a heuristic local outlier factor (to the best of our knowledge, it is utilized for the first time to deal with outlier features in a visual tracking application), which further maximizes the representation of the target object in which we formulate outlier feature detection as a binary classification problem with the output features of the LGF module. Extensive UAV flight experiments show that the proposed visual tracker achieves real-time frame rates of more than thirty-five frames per second on an i7 processor with 640 × 512 image resolution and outperforms the most popular state-of-the-art trackers favorably in terms of robustness, efficiency and accuracy.

Feature Tracking and Synchronous Scene Generation with a Single Camera

This paper shows a method of tracking feature points to update camera pose and generating a synchronous map for AR (Augmented Reality) system. Firstly we select the ORB (Oriented FAST and Rotated BRIEF) [1] detection algorithm to detect the feature points which have depth information to be markers, and we use the LK (Lucas-Kanade) optical flow [2] algorithm to track four of them. Then we compute the rotation and translation of the moving camera by relationship matrix between 2D image coordinate and 3D world coordinate, and then we update the camera pose. Last we generate the map, and we draw some AR objects on it. If the feature points are missing, we can compute the same world coordinate as the one before missing to recover tracking by using new corresponding 2D/3D feature points and camera poses at that time. There are three novelties of this study: an improved ORB detection, which can obtain depth information, a rapid update of camera pose, and tracking recovery. Referring to the PTAM (Parallel Tracking and Mapping) [3], we also divide the process into two parallel sub-processes: Detecting and Tracking (including recovery when necessary) the feature points and updating the camera pose is one thread. Generating the map and drawing some objects is another thread. This parallel method can save time for the AR system and make the process work in real-time.

A Novel Hardware Architecture of the Lucas–Kanade Optical Flow for Reduced Frame Memory Access

The Lucas–Kanade (LK) algorithm is a cost-efficient gradient-based algorithm for real-time optical flow generation. An excessive external memory access limits the LK algorithm from being broadly used in practical high-frame-rate applications. To overcome this limitation, this paper proposes a novel hardware architecture that stores the input image after the Gaussian filtering operation instead of the original input image itself. The Gaussian-filtered image is downsampled in both the horizontal and vertical directions, thus reducing the external memory access to one quarter of the original data. The downsampling operation does not cause a significant degradation of accuracy because the Gaussian filter is a low-pass filter that reduces the aliasing effect of downsampling. The downsampled pixels are selected in an interleaved manner across multiple frames to reduce the degradation of accuracy. Experimental results show that the proposed algorithm reduces the frame memory access by 61%–75% compared with the previous research.

Human Detection Framework for Automated Surveillance Systems

Vision-based systems for surveillance applications have been used widely and gained more research attention. Detecting people in an image stream is challenging because of their intra-class variability, the diversity of the backgrounds, and the conditions under which the images were acquired. Existing human detection solutions suffer in their effectiveness and efficiency. In particular, the accuracy of the existing detectors is characterized by their high false positive and negative. In addition, existing detectors are slow for online surveillance systems which lead to large delay that is not suitable for surveillance systems for real-time monitoring. In this paper, a holistic framework is proposed for enhancing the performance of human detection in surveillance system. In general, the framework includes the following stages: environment modeling, motion object detection, and human object recognition. In environment modeling, modal algorithm has been suggested for background initialization and extraction. Then for effectively classifying the motion object, edge detecting and B-spline algorithm have been used for shadow detection and removal. Then, enhanced Lucas–Kanade optical flow has been used to get the area of interest for object segmentation. Finally, to enhance the segmentation, some morphological processes were performed. In the motion object recognition stage, segmentation for each blob is performed and processed to the human detector which is a complete learning-based system for detecting and localizing objects/humans in images using mixtures of deformable part models (PFF detector). Results show enhancement in each phase of the proposed framework. These enhancements are shown in the overall performance of human detection in surveillance system.

Consistent depth maps estimation from binocular stereo video sequence

In the paper, an approach is proposed for the problem of consistency in depth maps estimation from binocular stereo video sequence. The consistent method includes temporal consistency and spatial consistency to eliminate the flickering artifacts and smooth inaccuracy in depth recovery. So the improved global stereo matching based on graph cut and energy optimization is implemented. In temporal domain, the penalty function with coherence factor is introduced for temporal consistency, and the factor is determined by Lucas-Kanade optical flow weighted histogram similarity constraint (LKWHSC). In spatial domain, the joint bilateral truncated absolute difference (JBTAD) is proposed for segmentation smoothing. The method can smooth naturally and uniformly in low-gradient region and avoid over-smoothing as well as keep edge sharpness in high-gradient discontinuities to realize spatial consistency. The experimental results show that the algorithm can obtain better spatial and temporal consistent depth maps compared with the existing algorithms.

PULP: A Ultra-Low Power Parallel Accelerator for Energy-Efficient and Flexible Embedded Vision

Novel pervasive devices such as smart surveillance cameras and autonomous micro-UAVs could greatly benefit from the availability of a computing device supporting embedded computer vision at a very low power budget. To this end, we propose PULP (Parallel processing Ultra-Low Power platform), an architecture built on clusters of tightly-coupled OpenRISC ISA cores, with advanced techniques for fast performance and energy scalability that exploit the capabilities of the STMicroelectronics UTBB FD-SOI 28nm technology. We show that PULP performance can be scaled over a 1x-354x range, with a peak theoretical energy efficiency of 211 GOPS/W. We present performance results for several demanding kernels from the image processing and vision domain, with post-layout power modeling: a motion detection application that can run at an efficiency up to 192 GOPS/W (90 % of the theoretical peak); a ConvNet-based detector for smart surveillance that can be switched between 0.7 and 27fps operating modes, scaling energy consumption per frame between 1.2 and 12mJ on a 320 ×240 image; and FAST + Lucas-Kanade optical flow on a 128 ×128 image at the ultra-low energy budget of 14 μJ per frame at 60fps.

Effects of the Forecasting Methods, Precipitation Character, and Satellite Resolution on the Predictability of Short-Term Quantitative Precipitation Nowcasting (QPN) from a Geostationary Satellite.

The prediction of the short-term quantitative precipitation nowcasting (QPN) from consecutive gestational satellite images has important implications for hydro-meteorological modeling and forecasting. However, the systematic analysis of the predictability of QPN is limited. The objective of this study is to evaluate effects of the forecasting model, precipitation character, and satellite resolution on the predictability of QPN usingimages of a Chinese geostationary meteorological satellite Fengyun-2F (FY-2F) which covered all intensive observation since its launch despite of only a total of approximately 10 days. In the first step, three methods were compared to evaluate the performance of the QPN methods: a pixel-based QPN using the maximum correlation method (PMC); the Horn-Schunck optical-flow scheme (PHS); and the Pyramid Lucas-Kanade Optical Flow method (PPLK), which is newly proposed here. Subsequently, the effect of the precipitation systems was indicated by 2338 imageries of 8 precipitation periods. Then, the resolution dependence was demonstrated by analyzing the QPN with six spatial resolutions (0.1atial, 0.3a, 0.4atial rand 0.6). The results show that the PPLK improves the predictability of QPN with better performance than the other comparison methods. The predictability of the QPN is significantly determined by the precipitation system, and a coarse spatial resolution of the satellite reduces the predictability of QPN.

Accurate inverse-consistent symmetric optical flow for 4D CT lung registration

Deformable image registration remains a challenging research area due to difficulties associated with local intensity variation and large motion. In this paper, an Accurate Inverse-consistent Symmetric Optical Flow (AISOF) method is proposed to overcome these difficulties. The two main contributions of AISOF include the following: (1) a coarse-to-fine strategy for an inverse-consistent symmetric method and (2) a novel Hybrid Local Binary Pattern (HLBP) to the classical Lucas–Kanade optical flow method. The HLBP consists of a median binary pattern and a generalised centre-symmetric local binary pattern. The generalised centre-symmetric local binary pattern has two thresholds, and this pattern can capture more information than the classical centre-symmetric local binary pattern, which has one threshold. The proposed HLBP can cope well with high contrast intensity and local intensity variation. Because the inverse-consistent symmetric method can reduce inverse consistency errors in Markov random fields based registration methods, we adopted this method to improve the accuracy of registration. In addition, a coarse-to-fine strategy was adopted to handle large motion. The proposed AISOF method was evaluated for 10 publicly available 4D CT lung datasets from the DIR-Lab. The mean target registration error of the AISOF method is 1.16 mm, which is significantly superior to the error of the classical Lucas–Kanade optical flow method, 2.83 mm. Moreover, this error is also the smallest of all unmasked registration methods using these datasets.

Development of a Cost-Effective Tele-Robot System Delivering Speaker’s Affirmative and Negative Intentions

A telerobot offers a more engaging and enjoyable interaction with people at a distance by communicating via audio, video, expressive gestures, body pose and proxemics. To provide its potential benefits at a reasonable cost, this paper presents a telepresence robot system for video communication which can deliver speaker`s head motion through its display stanchion. Head gestures such as nodding and head-shaking can give crucial information during conversation. We also can assume a speaker`s eye-gaze, which is known as one of the key non-verbal signals for interaction, from his/her head pose. In order to develop an efficient head tracking method, a 3D cylinder-like head model is employed and the Harris corner detector is combined with the Lucas-Kanade optical flow that is known to be suitable for extracting 3D motion information of the model. Especially, a skin color-based face detection algorithm is proposed to achieve robust performance upon variant directions while maintaining reasonable computational cost. The performance of the proposed head tracking algorithm is verified through the experiments using BU`s standard data sets. A design of robot platform is also described as well as the design of supporting systems such as video transmission and robot control interfaces.

Horn–Schunck optical flow applied to deformation measurement of a birdlike airfoil

Current deformation measurement techniques suffer from limited spatial resolution. In this work, a highly accurate and high-resolution Horn–Schunck optical flow method is developed and then applied to measuring the static deformation of a birdlike flexible airfoil at a series of angles of attack at Reynolds number 100,000 in a low speed, low noise wind tunnel. To allow relatively large displacements, a nonlinear Horn–Schunck model and a coarse-to-fine warping process are adopted. To preserve optical flow discontinuities, a nonquadratic penalization function, a multi-cue driven bilateral filtering and a principle component analysis of local image patterns are used. First, the accuracy and convergence of this Horn–Schunck technique are verified on a benchmark. Then, the maximum displacement that can be reliably calculated by this technique is studied on synthetic images. Both studies are compared with the performance of a Lucas–Kanade optical flow method. Finally, the Horn–Schunck technique is used to estimate the 3-D deformation of the birdlike airfoil through a stereoscopic camera setup. The results are compared with those computed by Lucas–Kanade optical flow, image correlation and numerical simulation.