Optical Flow Analysis Research Articles

Camera Motion Detection Through Frame Splitting and Combination of Region-Based Motion Signals

Most of the existing approaches for camera motion detection are based on optical flow analysis and the use of the affine motion model. However, these methods are computationally expensive due to the cost of optical flow estimation and may be inefficient in the presence of moving objects whose motion is independent of the camera motion. We present an effective approach to detect camera motions by considering four trapezoidal regions in each frame and computing the horizontal and vertical translations of those regions. Then, simple decision rules based on the translations of the regions are employed in order to decide for the existence and the type of camera motion in each frame. In this way, three signals are constructed (pan, tilt, zoom) which are subsequently filtered to improve the robustness of the method. Comparative experiments on a variety of videos indicate that our method efficiently detects any type of camera motion (pan, tilt, zoom), even in the case where moving objects exist in the video sequence.

Quantitatively characterizing drug-induced arrhythmic contractile motions of human stem cell-derived cardiomyocytes.

Quantification of abnormal contractile motions of cardiac tissue has been a noteworthy challenge and significant limitation in assessing and classifying the drug-induced arrhythmias (i.e., Torsades de pointes). To overcome these challenges, researchers have taken advantage of computational image processing tools to measure contractile motion from cardiomyocytes derived from human induced pluripotent stem cells (hiPSC-CMs). However, the amplitude and frequency analysis of contractile motion waveforms does not produce sufficient information to objectively classify the degree of variations between two or more sets of cardiac contractile motions. In this paper, we generated contractile motion data from beating hiPSC-CMs using motion tracking software based on optical flow analysis, and then implemented a computational algorithm, phase space reconstruction (PSR), to derive parameters (embedding, regularity, and fractal dimensions) to further characterize the dynamic nature of the cardiac contractile motions. Application of drugs known to cause cardiac arrhythmia induced significant changes to these resultant dimensional parameters calculated from PSR analysis. Integrating this new computational algorithm with the existing analytical toolbox of cardiac contractile motions will allow us to expand current assessments of cardiac tissue physiology into an automated, high-throughput, and quantifiable manner which will allow more objective assessments of drug-induced proarrhythmias.

12.2: A 3D Display Parallel System: Light Field Re‐rendering and Depth Sense Optimization

The main benefit of 3D display over 2D display is the obvious ability to create a more lifelike character with high depth sense. However, the limitation of human eye's visual mechanism, unartful 3D scene structure design, or bad viewing condition always emerges poor depth perception experience or even physiological discomfort during the watching time, which is often sub‐optimal for mass high‐quality 3D display productions. To solve this problem, we propose a novel 3D display parallel system for depth sense optimization and it empirically guides how the light field should be re‐rendered. Structurally, the parallel system consists of an artificial perception measurement system, a display evaluation model and a light field display rendering system, which includes the display calibration, scene capture, light field data processing and display. Particularly, the system can systematically analyze and model various factors affecting the depth sense which learned through the measurement system, like scene structure, objects’ speeds in 3D video and so on. And those sense factors can be personally modified or increased according to the viewer's demands or technical improvement. Moreover, the light field could be real‐time re‐rendered, based on some image processing technology, optical flow analysis and object segmentation (or tracking) (especially the one‐shot video segmentation). Theory and algorithms are developed and experimental validation results show a superior performance.

Silicone-Based Molding Technique for Optical Flow Analysis in Transparent Models of Fluidic Components

Optical flow analysis methods such as particle image velocimetry can only be performed in fluid systems or components with optical access. Many fluidic components, such as metallic tubes, do not typically feature optical accessibility. A new silicone-based molding technique is presented that makes it possible to replicate non-transparent fluidic components as models with optical accessibility that can be used to perform optical flow analysis. Furthermore, to avoid errors due to refraction, a test fluid is presented whose refractive index matches that of the silicone material of the replication. This new method allows flows to be analyzed in tubes or similar components with diameters in the range of only a few millimeters. An initial test was performed demonstrating the proof of concept and the velocity field of the flow inside a manifold was measured using a micro-particle image velocimetry setup. The study showed that both simple parts like tubes and complex parts such as manifolds can be replicated and investigated.

An Exploratory Framework for Cyclone Identification and Tracking.

Analyzing depressions plays an important role in meteorology, especially in the study of cyclones. In particular, the study of the temporal evolution of cyclones requires a robust depression tracking framework. To cope with this demand we propose a pipeline for the exploration of cyclones and their temporal evolution. This entails a generic framework for their identification and tracking. The fact that depressions and cyclones are not well-defined objects and their shape and size characteristics change over time makes this task especially challenging. Our method combines the robustness of topological approaches and the detailed tracking information from optical flow analysis. At first cyclones are identified within each time step based on well-established topological concepts. Then candidate tracks are computed from an optical flow field. These tracks are clustered within a moving time window to distill dominant coherent cyclone movements, which are then forwarded to a final tracking step. In contrast to previous methods our method requires only a few intuitive parameters. An integration into an exploratory framework helps in the study of cyclone movement by identifying smooth, representative tracks. Multiple case studies demonstrate the effectiveness of the method in tracking cyclones, both in the northern and southern hemisphere.

MUSCLEMOTION: A Versatile Open Software Tool to Quantify Cardiomyocyte and Cardiac Muscle Contraction In Vitro and In Vivo.

There are several methods to measure cardiomyocyte and muscle contraction, but these require customized hardware, expensive apparatus, and advanced informatics or can only be used in single experimental models. Consequently, data and techniques have been difficult to reproduce across models and laboratories, analysis is time consuming, and only specialist researchers can quantify data. Here, we describe and validate an automated, open-source software tool (MUSCLEMOTION) adaptable for use with standard laboratory and clinical imaging equipment that enables quantitative analysis of normal cardiac contraction, disease phenotypes, and pharmacological responses. MUSCLEMOTION allowed rapid and easy measurement of movement from high-speed movies in (1) 1-dimensional in vitro models, such as isolated adult and human pluripotent stem cell-derived cardiomyocytes; (2) 2-dimensional in vitro models, such as beating cardiomyocyte monolayers or small clusters of human pluripotent stem cell-derived cardiomyocytes; (3) 3-dimensional multicellular in vitro or in vivo contractile tissues, such as cardiac "organoids," engineered heart tissues, and zebrafish and human hearts. MUSCLEMOTION was effective under different recording conditions (bright-field microscopy with simultaneous patch-clamp recording, phase contrast microscopy, and traction force microscopy). Outcomes were virtually identical to the current gold standards for contraction measurement, such as optical flow, post deflection, edge-detection systems, or manual analyses. Finally, we used the algorithm to quantify contraction in in vitro and in vivo arrhythmia models and to measure pharmacological responses. Using a single open-source method for processing video recordings, we obtained reliable pharmacological data and measures of cardiac disease phenotype in experimental cell, animal, and human models.

Ground-Based Remote Sensing of Volcanic CO2 Fluxes at Solfatara (Italy)—Direct Versus Inverse Bayesian Retrieval

CO2 is the second most abundant volatile species of degassing magma. CO2 fluxes carry information of incredible value, such as periods of volcanic unrest. Ground-based laser remote sensing is a powerful technique to measure CO2 fluxes in a spatially integrated manner, quickly and from a safe distance, but it needs accurate knowledge of the plume speed. The latter is often difficult to estimate, particularly for complex topographies. So, a supplementary or even alternative way of retrieving fluxes would be beneficial. Here, we assess Bayesian inversion as a potential technique for the case of the volcanic crater of Solfatara (Italy), a complex terrain hosting two major CO2 degassing fumarolic vents close to a steep slope. Direct integration of remotely sensed CO2 concentrations of these vents using plume speed derived from optical flow analysis yielded a flux of 717 ± 121 t day−1, in agreement with independent measurements. The flux from Bayesian inversion based on a simple Gaussian plume model was in excellent agreement under certain conditions. In conclusion, Bayesian inversion is a promising retrieval tool for CO2 fluxes, especially in situations where plume speed estimation methods fail, e.g., optical flow for transparent plumes. The results have implications beyond volcanology, including ground-based remote sensing of greenhouse gases and verification of satellite soundings.

The Initial Development of Transient Volcanic Plumes as a Function of Source Conditions

AbstractTransient volcanic plumes, having similar eruption duration and rise timescales, characterize many unsteady Strombolian to Vulcanian eruptions. Despite being more common, such plumes are less studied than their steady state counterpart from stronger eruptions. Here we investigate the initial dynamics of transient volcanic plumes using high‐speed (visible light and thermal) and high‐resolution (visible light) videos from Strombolian to Vulcanian eruptions of Stromboli (Italy), Fuego (Guatemala), and Sakurajima (Japan) volcanoes. Physical parameterization of the plumes has been performed by defining their front velocity, velocity field, volume, and apparent surface temperature. We also characterized the ejection of the gas‐pyroclast mixture at the vent, in terms of number, location, duration, and frequency of individual ejection pulses and of time‐resolved mass eruption rate of the ejecta's ash fraction. Front velocity evolves along two distinct trends related to the initial gas‐thrust phase and later buoyant phase. Plumes' velocity field, obtained via optical flow analysis, highlights different features, including initial jets and the formation and/or merging of ring vortexes at different scales. Plume volume increases over time following a power law trend common to all volcanoes and affected by discharge history at the vent. Time‐resolved ash eruption rates range between 102 and 107 kg/s and may vary up to 2 orders of magnitude within the first seconds of eruption. Our results help detailing how the number, location, angle, duration, velocity, and time interval between ejection pulses at the vents crucially control the initial (first tens of second), and possibly later, evolution of transient volcanic plumes.

Motion process monitoring using optical flow–based principal component analysis-independent component analysis method

In this article, for the first time, the optical flow and principal component analysis followed independent component analysis are combined for monitoring the motion process of robotic-arm-based sy...

Dancing with the Stars: Using Image Analysis to Study the Choreography of the Endoplasmic Reticulum and Its Partners and of Movement Within Its Tubules.

In this chapter, approaches to the image analysis of the choreography of the plant endoplasmic reticulum (ER) labeled with fluorescent fusion proteins ("stars," if you wish) are presented. The approaches include the analyses of those parts of the ER that are attached through membrane contact sites to moving or nonmoving partners (other "stars"). Image analysis is also used to understand the nature of the tubular polygonal network, the hallmark of this organelle, and how the polygons change over time due to tubule sliding or motion. Furthermore, the remodeling polygons of the ER interact with regions of fundamentally different topology, the ER cisternae, and image analysis can be used to separate the tubules from the cisternae. ER cisternae, like polygons and tubules, can be motile or stationary. To study which parts are attached to nonmoving partners, such as domains of the ER that form membrane contact sites with the plasma membrane/cell wall, an image analysis approach called persistency mapping has been used. To study the domains of the ER that are moving rapidly and streaming through the cell, the image analysis of optic flow has been used. However, optic flow approaches confuse the movement of the ER itself with the movement of proteins within the ER. As an overall measure of ER dynamics, optic flow approaches are of value, but their limitation as to what exactly is "flowing" needs to be specified. Finally, there are important imaging approaches that directly address the movement of fluorescent proteins within the ER lumen or in the membrane of the ER. Of these, fluorescence recovery after photobleaching (FRAP), inverse FRAP (iFRAP), and single particle tracking approaches are described.

Biological motion cues aid identification of self-motion from optic flow but not heading detection.

When we move through the world, a pattern of expanding optic flow is generated on the retina. In completely rigid environments, this pattern signals one's direction of heading and is an important source of information for navigation. When we walk towards an oncoming person, the optic environment is not rigid, as the motion vectors generated by the other person represent a composite of that person's movement, his or her limb motion, and the observer's self-motion. Though this biological motion obfuscates the optic flow pattern, it also provides cues about the movement of other actors in the environment. It may be the case that the visual system takes advantage of these cues to simplify the decomposition of optic flow in the presence of other moving people. The current study sought to probe this possibility. In four experiments self-motion was simulated through an environment that was empty except for a single, walking point-light biological motion stimulus. We found that by using biological motion cues, observers were able to identify the presence of self-motion despite the lack of stable scene information. However, when estimating heading based on these stimuli, the pattern of observer heading estimates could be approximately reproduced by computing the vector sum of the walker's translation and the stimulated self-motion. This suggests that though biological motion can be used to disentangle self-motion in ambiguous situations, optic flow analysis does not use this information to derive heading estimates.

Optical flow and image segmentation analysis for noninvasive precise mapping of microwave thermal ablation in X-ray CT scans - ex vivo study

Purpose: To develop image processing algorithms for noninvasive mapping of microwave thermal ablation using X-ray CT.Methods: Ten specimens of bovine liver were subjected to microwave ablation (20–80 W, 8 min) while scanned by X-ray CT at 5 s intervals. Specimens were cut and manually traced by two observers. Two algorithms were developed and implemented to map the ablation zone. The first algorithm utilises images segmentation of Hounsfield units changes (ISHU). The second algorithm utilises radial optical flow (ROF). Algorithm sensitivity to spatiotemporal under-sampling was assessed by decreasing the acquisition rate and reducing the number of acquired projections used for image reconstruction in order to evaluate the feasibility of implementing radiation reduction techniques.Results: The average radial discrepancy between the ISHU and ROF contours and the manual tracing were 1.04±0.74 and 1.16±0.79mm, respectively. When diluting the input data, the ISHU algorithm retained its accuracy, ranging from 1.04 to 1.79mm. By contrast, the ROF algorithm performance became inconsistent at low acquisition rates. Both algorithms were not sensitive to projections reduction, (ISHU: 1.24±0.83mm, ROF: 1.53±1.15mm, for reduction by eight fold). Ablations near large blood vessels affected the ROF algorithm performance (1.83±1.30mm; p < 0.01), whereas ISHU performance remained the same.Conclusion: The two suggested noninvasive ablation mapping algorithms can provide highly accurate contouring of the ablation zone at low scan rates. The ISHU algorithm may be more suitable for clinical practice as it appears more robust when radiation dose reduction strategies are employed and when the ablation zone is near large blood vessels.

A new method for automating the investigation of stem cell populations based on the analysis of the integral optical flow of a video sequence

A new method is presented for automating the investigation of stem cell populations based on the automatic analysis of video sequences of images. For automatic image analysis, an integral optical flow apparatus is used. The proposed method classifies dynamic objects by constructing a pyramid of the integral optical flow. The main stages of the method are as follows: image capturing and processing, segmentation, measurement, and tissue description generation. Experimental tests confirm that the proposed method is capable of identifying the main stages of cellular development (mitosis, differentiation, and apoptosis).

Ultrasound field characterization using synthetic schlieren tomography.

Synthetic schlieren imaging, also known as background oriented schlieren imaging, is used to determine the acoustical field of a focused ultrasound transducer operating at 1.01 MHz frequency with peak pressure amplitude of 0.97 MPa. The measurement setup is composed of a commercial off-the-shelf digital single-lens reflex (DSLR) camera with an ordinary objective, a high power light-emitting diode driven in pulsating mode, water tank, ultrasound transducer, rotation stage, and driving electronics. Measurements are performed in tomographic fashion by rotating the ultrasound transducer within the water tank and photographing an imaged target behind the ultrasound field. The photographs are processed with a Horn-Schunck-type algorithm, commonly used in optical flow analysis, in order to determine the deflection of light rays as caused by ultrasound field induced acousto-optic effect. Inverse Radon transform is then used, with the deflection data, to obtain three-dimensional spatial distribution of the pressure field gradient, from which an approximation of the ultrasonic pressure field is computed. The pressure field obtained with synthetic schlieren tomography is then compared to hydrophone measurements mainly qualitatively.

Optical-flow analysis toolbox for characterization of spatiotemporal dynamics in mesoscale optical imaging of brain activity

Wide-field optical imaging techniques constitute powerful tools to investigate mesoscale neuronal activity. The sampled data constitutes a sequence of image frames in which one can investigate the flow of brain activity starting and terminating at source and sink locations respectively. Approaches to the analyses of information flow include qualitative assessment to identify sources and sinks of activity as well as their trajectories, and quantitative measurements based on computing the temporal variation of the intensity of pixels. Furthermore, in a few studies estimates of wave motion have been reported using optical-flow techniques from computer vision. However, a comprehensive toolbox for the quantitative analyses of mesoscale brain activity data is still lacking. We present a graphical-user-interface toolbox based in Matlab® for investigating the spatiotemporal dynamics of mesoscale brain activity using optical-flow analyses. The toolbox includes the implementation of three optical-flow methods namely Horn-Schunck, Combined Local-Global, and Temporospatial algorithms for estimating velocity vector fields of flow of mesoscale brain activity. From the velocity vector fields we determined the locations of sources and sinks as well as the trajectories and temporal velocities of flow of activity. Using simulated data as well as experimentally derived sensory-evoked voltage and calcium imaging data from mice, we compared the efficacy of the three optical-flow methods for determining spatiotemporal dynamics. Our results indicate that the combined local-global method we employed, yields the best results for estimating wave motion. The automated approach permits rapid and effective quantification of mesoscale brain dynamics and may facilitate the study of brain function in response to new experiences or pathology.

Optical Flow and Prediction Residual Based Hybrid Forensic System for Inter-Frame Tampering Detection

In the wake of widespread proliferation of inexpensive and easy-to-use digital content editing software, digital videos have lost the idealized reputation they once held as universal, objective and infallible evidence of occurrence of events. The pliability of digital content and its innate vulnerability to unobtrusive alterations causes us to become skeptical of its validity. However, in spite of the fact that digital videos may not always present a truthful picture of reality, their usefulness in today’s world is incontrovertible. Therefore, the need to verify the integrity and authenticity of the contents of a digital video becomes paramount, especially in critical scenarios such as defense planning and legal trials where reliance on untrustworthy evidence could have grievous ramifications. Inter-frame tampering, which involves insertion/removal/replication of sets of frames into/from/within a video sequence, is among the most un-convoluted and elusive video forgeries. In this paper, we propose a potent hybrid forensic system that detects inter-frame forgeries in compressed videos. The system encompasses two forensic techniques. The first is a novel optical flow analysis based frame-insertion and removal detection procedure, where we focus on the brightness gradient component of optical flow and detect irregularities caused therein by post-production frame-tampering. The second component is a prediction residual examination based scheme that expedites detection and localization of replicated frames in video sequences. Subjective and quantitative results of comprehensive tests on an elaborate dataset under diverse experimental set-ups substantiate the effectuality and robustness of the proposed system.

Hybrid particle image velocimetry with the combination of cross-correlation and optical flow method

Through a combination of cross-correlation and optical flow method (OFM), a novel technique can benefit from the strengths of each method while mitigating the flaws each individual method contains. The hybrid Particle Image Velocimetry (PIV) method utilizes the state-of-the-art cross-correlation method to account for the relatively large displacements of particles and refine the flow field using the high-resolution analysis of OFM. Image processing techniques such as interpolation, image shifting, and Gaussian filtering are crucial for integrating the cross-correlation technique with optical flow analysis. The accuracy of the hybrid PIV method was validated using standard simulated PIV images that encompassed various parameters encountered in PIV measurements. Each set of images was analyzed by the hybrid method and three other widely used correlation techniques to verify the accuracy. Results confirmed that the hybrid method is consistently more accurate than the other methods in generating the flow vectors, especially near the boundaries. Additionally, for cases dealing with large-sized particles or small displacements, the hybrid PIV method also attains more accurate results.

NEB in Analysis of Optical Flow 4 x 4 and 6 x 6-Patches

We apply the nudged elastic band technique to non-lineal high-dimensional datasets, we analyze spaces of 4 x 4 and 6 x 6 optical flow patches and detect their topological properties. We experimentally prove that subsets of 4 x 4 and 6 x 6 optical flow patches can be modeled a circle, which confirm some results of 4 x 4 and 6 x 6 optical flow patches by using a new method-NEB, and expend Adams et al's result to larger patches of optical flow.

Effective star tracking method based on optical flow analysis for star trackers.

Benefiting from rapid development of imaging sensor technology, modern optical technology, and a high-speed computing chip, the star tracker's accuracy, dynamic performance, and update rate have been greatly improved with low power consumption and miniature size. The star tracker is currently one of the most competitive attitude measurement sensors. However, due to restrictions of the optical imaging system, difficulties still exist in moving star spot detection and star tracking when in special motion conditions. An effective star tracking method based on optical flow analysis for star trackers is proposed in this paper. Spot-based optical flow, based on a gray gradient between two adjacent star images, is analyzed to distinguish the star spot region and obtain an accurate star spot position so that the star tracking can keep continuous under high dynamic conditions. The obtained star vectors and extended Kalman filter (EKF) are then combined to conduct an angular velocity estimation to ensure region prediction of the star spot; this can be combined with the optical flow analysis result. Experiment results show that the method proposed in this paper has advantages in conditions of large angular velocity and large angular acceleration, despite the presence of noise. Higher functional density and better performance can be achieved; thus, the star tracker can be more widely applied in small satellites, remote sensing, and other complex space missions.

From the Cover: High-Throughput Imaging of Cardiac Microtissues for the Assessment of Cardiac Contraction during Drug Discovery.

Cardiotoxicity is a common cause of attrition in preclinical and clinical drug development. Current in vitro approaches have two main limitations, they either are limited to low throughput methods not amendable to drug discovery or lack the physiological responses to allow an integrated risk assessment. A human 3D cardiac microtissue containing human-induced pluripotent stem cell-derived cardiomyocytes (hiPS-CMs), cardiac endothelial cells and cardiac fibroblast were used to assess their suitability to detect drug induced changes in cardiomyocyte contraction. These cardiac microtissues, have a uniform size, spontaneously beat, lack a hypoxic core, and contain key markers of each cell type. Application of field stimulation and measurement of cardiac contraction confirm cardiac microtissues to be a suitable model to investigate drug-induced changes in cardiomyocyte contractility. Using a bespoke image acquisition work flow and optical flow analysis method to test 29 inotroptic and 13 non-inotroptic compounds in vivo We report that cardiac microtissues provide a high-throughput experimental model that is both able to detect changes in cardiac contraction with a sensitivity and specificity of 80 and 91%, respectively, and provide insight into the direction of the inotropic response. Allowing improved in vitro cardiac contractility risk assessment. Moreover, our data provide evidence of the detection of this liability at therapeutically relevant concentrations with a throughput amenable to drug discovery.