Optical Flow Techniques Research Articles

An Enhanced Optical Flow Technique for Radar Nowcasting of Precipitation and Winds

AbstractThe atmospheric state evolution is an inherently highly complex three-dimensional problem that numerical weather prediction (NWP) models attempt to solve. Although NWP models are being successfully employed for medium- and long-range forecast, their short-duration forecast (or nowcast) capabilities are still limited because of model initialization challenges. On the lower end of the complexity scale, nowcasting by extrapolation of two-dimensional weather radar images has long been the most effective tool for nowcasting precipitation. Attempts are being made to take advantage of both approaches by blending extrapolation and numerical model forecasts. In this work a different approach is presented, relying on the additional Doppler radar wind information and a simplified modeling of basic physical processes. Instead of mixing the outputs of different forecasts as in blended approaches, the idea behind this study is to combine extrapolation and precipitation modeling in a new technique with a higher level of complexity with respect to conventional nowcasting methods, although still much simpler than NWP models. As a preliminary step, the Variational Doppler Radar Analysis System (VDRAS) is used to provide an initial analysis exploiting all the available dual-polarization and Doppler radar observations. The rainwater and wind fields are then advected using an optical flow technique that is subject to simplified physical interactions. As a result precipitation and wind nowcasting are obtained and are successively validated up to a 1-h lead time, showing potential improvement upon standard extrapolation.

Application of local optical flow methods to high-velocity free-surface flows: Validation and application to stepped chutes

The entrainment of air on a high-velocity spillway leads to a rapid bulking in flow depth and developments of complex flow patterns downstream of the inception point of aeration. This study examines the feasibility of two local optical flow techniques – the Lucas-Kanade method and the Farneback method – applied to high-velocity air-water skimming flows above a stepped chute. Such methods are not widely known to the multiphase flow community. Experimental studies were undertaken in a relatively large-size stepped spillway model. Validation test cases were performed using a synchronised ultra-high-speed camera and phase-detection probe setup. The optical flow technique detected changes in brightness due to reflectance difference associated with passages of air-water interfaces. The standard deviation of luminance correlates with void fraction and bubble count rate, and may be used asa predictor for uncertainties in optical flow estimation. The streamwise optical flow properties were in close agreement with those determined by the phase-detection probe next to the sidewall, with increasing differences for void fraction greater than 50% (C>0.5). In the sidewall region, however the bubble count rate and interfacial velocity distributions were underestimated compared to the channel centreline's interfacial properties. The tests demonstrated that the optical flow methods can provide useful qualitative and quantitative information on complex air-water flow patterns.

Optically Remote Noncontact Heart Rates Sensing Technique

Heart rate monitoring via optically remote noncontact technique was reported in this research. A green laser (5 mW, 532±10 nm) was projected onto the left carotid artery. The reflected laser light on the screen carried the deviation of the interference patterns. The interference patterns were recorded by the digital camera. The recorded videos of the interference patterns were frame by frame analysed by 2 standard digital image processing (DIP) techniques, block matching (BM) and optical flow (OF) techniques. The region of interest (ROI) pixels within the interference patterns were analysed for periodically changes of the interference patterns due to the heart pumping action. Both results of BM and OF techniques were compared with the reference medical heart rate monitoring device by which a contact measurement using pulse transit technique. The results obtained from BM technique was 74.67 bpm (beats per minute) and OF technique was 75.95 bpm. Those results when compared with the reference value of 75.43±1 bpm, the errors were found to be 1.01% and 0.69%, respectively.

Correlating and Modeling of Extracted Features from PVT Images of Composites using Optical Flow Technique and Weight Elimination Algorithm Optimization [OFT-WEA

A new approach to the use and implementation of Optical Flow technique is presented. The technique extracts features from presented images as a function of reference image and produces percentage of matching between the reference and tested images. The new approach in using Optical Flow lies in replacing the motion part of the algorithm with differential time related changes in an infrared thermal image sequence with frames of images taken as a result of applying the Pulse Video Thermography (PVT) technique. The sequence of images or frames is obtained for the tested structures of composites before and after impact damage. The resulted data of the tested images is used to establish mathematical model that can be used to predict impact energy from collected features or predict expected features from knowing impact damage level. To optimize the mathematical model, a new way of using Neural Networks is employed, which aims at obtaining a best fit for the used variables in the mathematical model, hence resulting in a better testing interpretation and more accurate prediction and classification of image features to improve future composite structures designs. The Neural Network Weigh Elimination Algorithm (WEA) is used and proved effective in predicting areas of damage.

A transfer learning framework for traffic video using neuro-fuzzy approach

One of the main challenges in the Traffic Anomaly Detection (TAD) system is the ability to deal with unknown target scenes. As a result, the TAD system performs less in detecting anomalies. This paper introduces a novelty in the form of Adaptive Neuro-Fuzzy Inference System-Lossy-Count-based Topic Extraction (ANFIS-LCTE) for classification of anomalies in source and target traffic scenes. The process of transforming the input variables, learning the semantic rules in source scene and transferring the model to target scene achieves the transfer learning property. The proposed ANFIS-LCTE transfer learning model consists of four steps. (1) Low level visual items are extracted only for motion regions using optical flow technique. (2) Temporal transactions are created using aggregation of visual items for each set of frames. (3) An LCTE is applied for each set of temporal transaction to extract latent sequential topics. (4) ANFIS training is done with the back-propagation gradient descent method. The proposed ANFIS model framework is tested on standard dataset and performance is evaluated in terms of training performance and classification accuracies. Experimental results confirm that the proposed ANFIS-LCTE approach performs well in both source and target datasets.

Four dimensional material movies: High speed phase-contrast tomography by backprojection along dynamically curved paths

We present an approach towards four dimensional (4d) movies of materials, showing dynamic processes within the entire 3d structure. The method is based on tomographic reconstruction on dynamically curved paths using a motion model estimated by optical flow techniques, considerably reducing the typical motion artefacts of dynamic tomography. At the same time we exploit x-ray phase contrast based on free propagation to enhance the signal from micron scale structure recorded with illumination times down to a millisecond (ms). The concept is demonstrated by observing the burning process of a match stick in 4d, using high speed synchrotron phase contrast x-ray tomography recordings. The resulting movies reveal the structural changes of the wood cells during the combustion.

Evidence for a Magnetic Reconnection Origin of Plasma Outflows along Post-CME Rays

Abstract Bright rays are often observed after coronal mass ejections (CMEs) erupt. These rays are dynamical structures along which plasmas move outward. We investigated the outflows along the post-CME rays observed by the COR2 on board STEREO Behind on 2013 September 21 and 22. We tracked two CMEs, two ray tips, and seven blobs using the NAVE optical flow technique. As a result, we found that the departure times of blobs and ray tips from the optimally chosen starting height of 0.5 R ☉ coincided with the occurrence times of the corresponding recurrent small flares within 10 minutes. These small flares took place many hours after the major flares. This result supports a magnetic reconnection origin of the outward flows along the post-CME ray and the importance of magnetic islands for understanding the process of magnetic reconnection. The total energy of magnetic reconnection maintaining the outflows for 40 hr is estimated at 1.4 × 10 30 erg. Further investigations of plasma outflows along post-CME rays will shed much light on the physical properties of magnetic reconnection occurring in the solar corona.

Full-field OCT technique for high speed event-based optical flow and particle tracking.

This article introduces a method to extract the speed and density of microparticles in real time at several kHz using an asynchronous event-based camera mounted on a full-field optical coherence tomography (FF-OCT) setup. These cameras detect significant amplitude changes, allowing scene-driven acquisitions. They are composed of an array of autonomously operating pixels. Events are triggered when an illuminance change at the pixel level is significant at 1μs time precision. The event-driven FF-OCT algorithm relies on a time-based optical flow computation to operate directly on incoming events and updates the estimation of velocity, direction and density while reducing both computation and data load. We show that for fast moving microparticles in a range of 0.4 - 6.5mm/s, the method performs faster and more efficiently than existing techniques in real time. The target application of this work is to evaluate erythrocyte dynamics at the microvascular level in vivo with a high temporal resolution.

Optical-flow analysis toolbox for characterization of spatiotemporal dynamics in mesoscale optical imaging of brain activity

Wide-field optical imaging techniques constitute powerful tools to investigate mesoscale neuronal activity. The sampled data constitutes a sequence of image frames in which one can investigate the flow of brain activity starting and terminating at source and sink locations respectively. Approaches to the analyses of information flow include qualitative assessment to identify sources and sinks of activity as well as their trajectories, and quantitative measurements based on computing the temporal variation of the intensity of pixels. Furthermore, in a few studies estimates of wave motion have been reported using optical-flow techniques from computer vision. However, a comprehensive toolbox for the quantitative analyses of mesoscale brain activity data is still lacking. We present a graphical-user-interface toolbox based in Matlab® for investigating the spatiotemporal dynamics of mesoscale brain activity using optical-flow analyses. The toolbox includes the implementation of three optical-flow methods namely Horn-Schunck, Combined Local-Global, and Temporospatial algorithms for estimating velocity vector fields of flow of mesoscale brain activity. From the velocity vector fields we determined the locations of sources and sinks as well as the trajectories and temporal velocities of flow of activity. Using simulated data as well as experimentally derived sensory-evoked voltage and calcium imaging data from mice, we compared the efficacy of the three optical-flow methods for determining spatiotemporal dynamics. Our results indicate that the combined local-global method we employed, yields the best results for estimating wave motion. The automated approach permits rapid and effective quantification of mesoscale brain dynamics and may facilitate the study of brain function in response to new experiences or pathology.

Operational Application of Optical Flow Techniques to Radar-Based Rainfall Nowcasting

Hong Kong Observatory has been operating an in-house developed rainfall nowcasting system called “Short-range Warning of Intense Rainstorms in Localized Systems (SWIRLS)” to support rainstorm warning and rainfall nowcasting services. A crucial step in rainfall nowcasting is the tracking of radar echoes to generate motion fields for extrapolation of rainfall areas in the following few hours. SWIRLS adopted a correlation-based method in its first operational version in 1999, which was subsequently replaced by optical flow algorithm in 2010 and further enhanced in 2013. The latest optical flow algorithm employs a transformation function to enhance a selected range of reflectivity for feature tracking. It also adopts variational optical flow computation that takes advantage of the Horn–Schunck approach and the Lucas–Kanade method. This paper details the three radar echo tracking algorithms, examines their performances in several significant rainstorm cases and summaries verification results of multi-year performances. The limitations of the current approach are discussed. Developments underway along with future research areas are also presented.

Congestion analysis of pilgrims in Hajj and Umrah congregation using block matching and optical flow

A novel method has been proposed to classify the motion of pilgrims with respect to congestion level in the holy mosque of Makkah. Millions of Muslims visit this mosque during Hajj and Umrah every year. A large number of security personnel is required to maintain the smooth flow of pilgrims in order to avert any catastrophe. Therefore, it is inevitable to design a computer aided system to reduce human effort. The proposed system pre-processes input images to segregate the moving shadows and pilgrims in order to nullify the false motion due to moving shadows. A hybrid method consisting of block matching and optical flow techniques has been used for the computation of motion vectors. Decision tree classifier is used on the number of motion vectors having non-zero magnitude. Experiments show that the proposed system has promising results yielding an accuracy of 90.58% for the congestion classification of pilgrims.

Enhancing pedestrian detection using optical flow for surveillance

Optical flow can be used to segment a moving object from its backgrounds and track it. In this paper, an Enhanced Lucas-Kanade optical flow technique was used to improve human detection in terms of speed and accuracy. We combined object segmentation output with a human detector using an optical flow algorithm. The proposed technique used the optical flow to find the area of interest to complete object segmentation and use those results as an input for the human detector. This technique has been developed to be used in surveillance systems. Our experiments indicated that the proposed method was 37% faster and 118% more accurate than the standard Felzenszwalb (PFF) detector.

Enhancing pedestrian detection using optical flow for surveillance

Optical flow can be used to segment a moving object from its backgrounds and track it. In this paper, an Enhanced Lucas-Kanade optical flow technique was used to improve human detection in terms of speed and accuracy. We combined object segmentation output with a human detector using an optical flow algorithm. The proposed technique used the optical flow to find the area of interest to complete object segmentation and use those results as an input for the human detector. This technique has been developed to be used in surveillance systems. Our experiments indicated that the proposed method was 37% faster and 118% more accurate than the standard Felzenszwalb PFF detector.

A variational approach to estimate incompressible fluid flows

A variational approach is used to recover fluid motion governed by Stokes and Navier–Stokes equations. Unlike previous approaches where optical flow method is used to track rigid body motion, this new framework aims at investigating incompressible flows using optical flow techniques. We formulate a minimization problem and determine conditions under which unique solution exists. Numerical results using finite element method not only support theoretical results but also show that Stokes flow forced by a potential are recovered almost exactly.

A fused deep learning architecture for viewpoint classification of echocardiography

This study extends the state of the art of deep learning convolutional neural network (CNN) to the classification of video images of echocardiography, aiming at assisting clinicians in diagnosis of heart diseases. Specifically, the architecture of neural networks is established by embracing hand-crafted features within a data-driven learning framework, incorporating both spatial and temporal information sustained by the video images of the moving heart and giving rise to two strands of two-dimensional convolutional neural network (CNN). In particular, the acceleration measurement along the time direction at each point is calculated using dense optical flow technique to represent temporal motion information. Subsequently, the fusion of both networks is conducted via linear integrations of the vectors of class scores obtained from each of the two networks. As a result, this architecture maintains the best classification results for eight viewpoint categories of echo videos with 92.1% accuracy rate whereas 89.5% is achieved using only single spatial CNN network. When concerning only three primary locations, 98% of accuracy rate is realised. In addition, comparisons with a number of well-known hand-engineered approaches are also performed, including 2D KAZE, 2D KAZE with Optical Flow, 3D KAZA, Optical Flow, 2D SIFT and 3D SIFT, which delivers accuracy rate of 89.4%, 84.3%, 87.9%, 79.4%, 83.8% and 73.8% respectively.

Camera-based photoplethysmography in critical care patients.

Camera-based photoplethysmography (cbPPG) is an optical measurement technique that reveals pulsatile blood flow in cutaneous microcirculation from a distance. cbPPG has been shown to reflect pivotal haemodynamic events like cardiac ejection in healthy subjects. In addition, it provides valuable insight into intrinsic microcirculatory regulation as it yields dynamic, two-dimensional perfusion maps. In this study, we evaluate the feasibility of a clinical cbPPG application in critical care patients. A mobile camera set-up to record faces of patients at the bed site was constructed. Videos were made during the immediate recovery after cardiac surgery under standard critical care conditions and were processed offline. Major motion artefacts were detected using an optical flow technique and suitable facial regions were manually annotated. cbPPG signals were highpass filtered and Fourier spectra out of consecutive 10s signal segments calculated for heart rate detection. Signal-to-noise ratios (SNR) of the Fourier spectra were derived as a quality measure. Reference data of vital parameters were synchronously acquired from the bed site monitoring system. Seventy patient videos of an average time of 28.6±2.8 min were analysed. Heart rate (HR) was detected within a±5 bpm range compared to reference in 83% of total recording time. Low SNR and HR detection failure were mostly, but not exclusively, attributed to non-physiological events like patient motion, interventions or sudden changes of illumination. SNR was reduced by low arterial blood pressure, whereas no impact of other perioperative or disease-related parameters was identified. Cardiac ejection is detectable by cbPPG under pathophysiologic conditions of cardiovascular disease and perioperative medicine. cbPPG measurements can be seamlessly integrated into the clinical work flow of critical care patients.

Optical flow and motion detection for navigation and control of biological and technological systems

We review neural and computational measurement of optical flow. Insects are sensitive to visual motion which has been established by elecrophysiological and behavioural studies over more than half a century. We discuss the elementary motion detector theory of how optical flow is detected in the nervous system and the limitations of this theory in predicting some insect visual behaviour that is apparently driven by a measurement of optical flow. The use of optical flow techniques in aerial robotics applications that mimic the challenges faced by flying insects are compared. The limitations of optical flow measurements in computing distance to targets or speed of travel is observed and the merits of proposed solutions discussed. Finally we examine a range of optical devices that were designed to overcome the limitations of monocular optical flow and consider the new limitations that they create which have prevented their wide acceptance in robotics.

A novel algorithm for the objective detection of tropical cyclone centres using infrared satellite images

ABSTRACTTropical cyclones (TCs) are one of nature’s most destructive phenomena, and a key element in forecasting TC tracks is the ability to accurately detect TC centres. In this paper, a novel algorithm has been proposed to objectively detect TC centres using infrared satellite images. Pyramid representation and optical flow technique are utilized to construct the cloud motion wind (CMW) field of each cyclone, and thereafter the centre is determined by analysing the constructed CMW field. Ten TCs formed in the Northwestern Pacific Ocean in 2014 have been tested to evaluate the performance of the present method, and TC Halong and Rammasun were analysed in detail as instances. Experimental results comparing with forecast track derived from Unisys Weather show that the proposed method provides an accurate detection of TC centres. The present algorithm has a potential to be employed to assist forecasters to detect TC centres.

Riverine skin temperature response to subsurface processes in low wind speeds

AbstractBoth surface and subsurface processes modulate the surface thermal skin and as such the skin temperature may serve as an indicator for coastal, estuarine, and alluvial processes. Infrared (IR) imagery offers the unique tool to survey such systems, allowing not only to assess temperature variability of the thermal boundary layer, but also to derive surface flow fields through digital particle image velocimetry, optical flow techniques, or spectral methods. In this study, IR time‐series imagery taken from a boat moored in the Hudson River estuary is used to determine surface flow, turbulent kinetic energy dissipation rate, and characteristic temperature and velocity length scales. These are linked to subsurface measurements provided by in situ instruments. Under the low wind conditions and weak stratification, surface currents and dissipation rate are found to reflect subsurface mean flow (r2 = 0.89) and turbulence (r2 = 0.75). For relatively low dissipation rates, better correlations are obtained by computing dissipation rates directly from wavenumber spectra rather than when having to assume the validity of the Taylor hypothesis. Furthermore, the subsurface dissipation rate scales with the surface length scales (L) and mean flow (U) using (r2 = 0.9). The surface length scale derived from the thermal fields is found to have a strong linear relationship (r2 = 0.88) to water depth (D) with (D/L) ∼ 13. Such a relation may prove useful for remote bathymetric surveys when no waves are present.

Multiview Plus Depth Video Coding With Temporal Prediction View Synthesis

Multiview video (MVV) plus depths formats use view synthesis to build intermediate views from existing adjacent views at the receiver side. Traditional view synthesis exploits the disparity information to interpolate an intermediate view by considered inter-view correlations. However, temporal correlation between different frames of the intermediate view can be used to improve the synthesis. We propose a new coding scheme for 3-D High Efficiency Video Coding (HEVC) that allows us to take full advantage of temporal correlations in the intermediate view and improve the existing synthesis from adjacent views. We use optical flow techniques to derive dense motion vector fields (MVF) from the adjacent views and then warp them at the level of the intermediate view. This allows us to construct multiple temporal predictions of the synthesized frame. A second contribution is an adaptive fusion method that judiciously selects between temporal and inter-view prediction to eliminate artifacts associated with each prediction type. The proposed system is compared against the state-of-the-art view synthesis reference software 1-D Fast technique used in 3-D HEVC standardization. Three intermediary views are synthesized. Gains of up to 1.21-dB Bjontegaard Delta peak SNR are shown when evaluated on several standard MVV test sequences.