Optical Flow Analysis Research Articles

Response properties of cat AMLS neurons to optic flow stimuli

Spiral and translation stimuli were used to investigate the response properties of cat AMLS (anteromedial lateral suprasylvian area) neurons to optic flow. The overwhelming majority of cells could be significantly excited by the two modes of stimuli and most responsive cells displayed obvious direction selectivity. It is the first time to find a visual area in mammalian brain preferring rotation stimuli. Two representative hypotheses are discussed here on the neural mechanism of optic flow analysis in visual cortex, and some new viewpoints are proposed to explain the experimental results.

Responses of neurons in the medial column of the inferior olive in pigeons to translational and rotational optic flowfields.

The responses of neurons in the medial column of the inferior olive to translational and rotational optic flow were recorded from anaesthetized pigeons. Panoramic translational or rotational flowfields were produced by mechanical devices that projected optic flow patterns onto the walls, ceiling and floor of the room. The axis of rotation/translation could be positioned to any orientation in three-dimensional space such that axis tuning could be determined. Each neuron was assigned a vector representing the axis about/along which the animal would rotate/translate to produce the flowfield that elicited maximal modulation. Both translation-sensitive and rotation-sensitive neurons were found. For neurons responsive to translational optic flow, the preferred axis is described with reference to a standard right-handed coordinate system, where +x, +y and +z represent rightward, upward and forward translation of the animal, respectively (assuming that all recordings were from the right side of the brain). t(+y) neurons were maximally excited in response to a translational optic flowfield that results from self-translation upward along the vertical (y) axis. t(-y) neurons also responded best to translational optic flow along the vertical axis but showed the opposite direction preference. The two remaining groups, t(-x+z) and t(-x-z) neurons, responded best to translational optic flow along horizontal axes that were oriented 45 degrees to the midline. There were two types of neurons responsive to rotational optic flow: rVA neurons preferred rotation about the vertical axis, and rH135c neurons preferred rotation about a horizontal axis at 135 degrees contralateral azimuth. The locations of marking lesions indicated a clear topographical organization of the six response types. In summary, our results reinforce that the olivo-cerebellar system dedicated to the analysis of optic flow is organized according to a reference frame consisting of three approximately orthogonal axes: the vertical axis, and two horizontal axes oriented 45 degrees to either side the midline. Previous research has shown that the eye muscles, vestibular semicircular canals and postural control system all share a similar spatial frame of reference.

Cardinal axes for radial and circular motion, revealed by summation and by masking

Both electro-physiological and psychophysical studies point to the existence of detectors specialised for the analysis of optic flow. However, it is unclear whether these detectors are tuned to specific ‘cardinal directions’ (such as radial and circular motion), or whether they respond equally to all directions of optic-flow motion, including intermediate spiral motions. Here summation and masking studies of motion coherence sensitivity are reported that suggest that optic flow may be tuned to radial and circular cardinal directions. Strong summation was found between two orthogonal directions of spiral motion, but much weaker summation between radial and circular motion. As orthogonal spiral motions always contain a common radial or circular component, the stronger summation for these motions implies that detectors are tuned to radial and circular directions. Similarly, the most effective masking stimuli (placed adjacent to but not superimposed on the test stimuli) tended to be those in the radial or circular directions, even for spiral targets, further suggesting that flow-field motion is detected and discriminated by mechanisms tuned to these ‘cardinal’ directions.

Spatiotemporal properties of fast and slow neurons in the pretectal nucleus lentiformis mesencephali in pigeons.

Neurons in the pretectal nucleus lentiformis mesencephali (LM) are involved in the analysis of optic flow that results from self-motion. Previous studies have shown that LM neurons have large receptive fields in the contralateral eye, are excited in response to largefield stimuli moving in a particular (preferred) direction, and are inhibited in response to motion in the opposite (anti-preferred) direction. We investigated the responses of LM neurons to sine wave gratings of varying spatial and temporal frequency drifting in the preferred and anti-preferred directions. The LM neurons fell into two categories. "Fast" neurons were maximally excited by gratings of low spatial [0.03-0.25 cycles/ degrees (cpd)] and mid-high temporal frequencies (0.5-16 Hz). "Slow" neurons were maximally excited by gratings of high spatial (0.35-2 cpd) and low-mid temporal frequencies (0.125-2 Hz). Of the slow neurons, all but one preferred forward (temporal to nasal) motion. The fast group included neurons that preferred forward, backward, upward, and downward motion. For most cells (81%), the spatial and temporal frequency that elicited maximal excitation to motion in the preferred direction did not coincide with the spatial and temporal frequency that elicited maximal inhibition to gratings moving in the anti-preferred direction. With respect to motion in the anti-preferred direction, a substantial proportion of the LM neurons (32%) showed bi-directional responses. That is, the spatiotemporal plots contained domains of excitation in addition to the region of inhibition. Neurons tuned to stimulus velocity across different spatial frequency were rare (5%), but some neurons (39%) were tuned to temporal frequency. These results are discussed in relation to previous studies of the responses of neurons in the accessory optic system and pretectum to drifting gratings and other largefield stimuli.

Quantitative study of dynamic behavior of cell monolayers during in vitro wound healing by optical flow analysis

Background In vitro wound healing assays are experimental models commonly used to analyze cell behavior during the migration process. A new approach is proposed for the quantification of cell motility based on an optical flow method. Methods We assumed that cell-population dynamics can be defined by an a priori affine-motion model. Identified model parameters are used as motion descriptors quantifying both elementary and complex cell movements, either at the wound margins or within the cell monolayer. Results When compared with the estimation of cell motility calculated from wound area temporal variation, it allows a more detailed and precise characterization of cell population movements. Comparative analysis of normal and cancerous cell lines revealed that typical measured velocities were about 2 μm/h and 7 μm/h for L929 and HeLa cells, respectively, at the beginning of the wound closure. The quantification of the effect of Hoechst 33342 on cell dynamics showed a similar behavior for control and stained cells within 20 h after wound scratching, but then a decreased velocity of stained cells. Conclusions The results demonstrate that this approach can be used to gain new insights into the dynamic changes induced by the extracellular environment and by anticancer drugs. Cytometry 41:19–30, 2000 © 2000 Wiley-Liss, Inc.

Scene-Based Shot Change Detection and Comparative Evaluation

A key step for managing a large video database is to partition the video sequences into shots. Past approaches to this problem tend to confuse gradual shot changes with changes caused by smooth camera motions. This is in part due to the fact that camera motion has not been dealt with in a more fundamental way. We propose an approach that is based on a physical constraint used in optical flow analysis, namely, the total brightness of a scene point across two frames should remain constant if the change across two frames is a result of smooth camera motion. Since the brightness constraint would be violated across a shot change, the detection can be based on detecting the violation of this constraint. It is robust because it uses only the qualitative aspect of the brightness constraint—detecting a scene change rather than estimating the scene itself. Moreover, by tapping on the significant know-how in using this constraint, the algorithm's robustness is further enhanced. Experimental results are presented to demonstrate the performance of various algorithms. It is shown that our algorithm is less likely to interpret gradual camera motions as shot changes, resulting in a better precision performance than most other algorithms. However, its performance deteriorates under large camera or object motions. A twin-threshold scheme is proposed to improve its robustness.

Projections from the nucleus of the basal optic root and nucleus lentiformis mesencephali to the inferior olive in pigeons (Columba livia)

The nucleus of the basal optic root (nBOR) of the accessory optic system (AOS) and the pretectal nucleus lentiformis mesencephali (LM) are involved in the analysis of optic flow and the generation of the optokinetic response. Previous studies have shown that the nBOR projects bilaterally to the medial column (mc) of the inferior olive (IO) and the LM projects to the ipsilateral mc. In the present study the retrograde tracer cholera toxin subunit B was injected into either the caudal or rostral mc. From all injections, retrogradely labeled cells were seen in the ipsilateral pretectum along the border of the medial and lateral subnuclei of the LM. Cells were also seen in bilaterally in the nBOR. On the contralateral side, a discrete group of cells was labeled in the rostral margin of the nBOR. These cells were localized in the dorsal portion of the nBOR proper and some were found in the adjacent nBOR dorsalis. On the ipsilateral side, a diffuse group of cells was seen in the caudal nBOR. Most of these cells were in the nBOR dorsalis and outside the nBOR complex in the area ventralis of Tsai and the reticular formation. From the injections into the caudal mc, a greater proportion of labeled cells was found in the LM, whereas a greater proportion of cells was found in the nBOR from the injections into the rostral mc. This differential projection from LM and nBOR to the caudal and rostral mc is consistent with the optic flow preferences of neurons in the mc, and a similar pattern of connectivity has been found in mammalian species.

Quantitative study of dynamic behavior of cell monolayers during in vitro wound healing by optical flow analysis

In vitro wound healing assays are experimental models commonly used to analyze cell behavior during the migration process. A new approach is proposed for the quantification of cell motility based on an optical flow method. We assumed that cell-population dynamics can be defined by an a priori affine-motion model. Identified model parameters are used as motion descriptors quantifying both elementary and complex cell movements, either at the wound margins or within the cell monolayer. When compared with the estimation of cell motility calculated from wound area temporal variation, it allows a more detailed and precise characterization of cell population movements. Comparative analysis of normal and cancerous cell lines revealed that typical measured velocities were about 2 microm/h and 7 microm/h for L929 and HeLa cells, respectively, at the beginning of the wound closure. The quantification of the effect of Hoechst 33342 on cell dynamics showed a similar behavior for control and stained cells within 20 h after wound scratching, but then a decreased velocity of stained cells. The results demonstrate that this approach can be used to gain new insights into the dynamic changes induced by the extracellular environment and by anticancer drugs.

A Common Frame of Reference for the Analysis of Optic Flow and Vestibular Information

A Common Frame of Reference for the Analysis of Optic Flow and Vestibular Information

Optic Flow Analysis for Self-Movement Perception

Optic Flow Analysis for Self-Movement Perception

An evolutionary system for recognition and tracking of synoptic-scale storm systems

An evolutionary system was developed for generation of complete tracks of northern midlatitude synoptic-scale storm systems based on optical flow and cloud motion analyses of global satellite-based datasets produced by the International Satellite Cloud Climatology Project (ISCCP). The tracking results were compared with low sea level pressure anomaly (SLPA) tracks obtained from the NASA Goddard Institute for Space Studies (GISS). The SLPA tracks were produced at GISS by analysis of meteorological, ground-based National Center for Environmental Prediction (NCEP) datasets. Results from the evolutionary system were also compared with results from using (a) the k-nearest neighbor rule ( k-NN) and (b) self-organizing maps (SOM) to determine correspondences between consecutive locations within a track. The consistency of our evolutionary storm tracking results with the behavior of the low sea level pressure anomaly tracks, the ability of our evolutionary system to generate and evaluate complete tracks, and the close comparison between the results obtained by the evolutionary, k-NN, and SOM analyses of the ISCCP-derived datasets at tracking steps in which proximity or optical flow information sufficed to determine movement, demonstrate the applicability and the potential of evolutionary systems for tracking midlatitude storm systems through low-resolution ISCCP cloud product datasets.

How stereovision interacts with optic flow perception: neural mechanisms

Optic flow, the global visual motion experienced during self-movement, supplies important navigational information. Optic flow analysis in the visual system is aided by several other visual and non-visual signals. Recent psychophysical findings demonstrate an interaction of optic flow perception and stereoscopic depth vision. Retinal disparity strongly affects an optic flow illusion, which can be related to the mechanisms of visual self-motion detection. To investigate the neuronal basis of this interaction, we tested several hypotheses by introducing different disparity contributions in a detailed neurobiological model of optic flow processing in monkey cortex. The disparity-dependent modification, which accounted best for the data suggests a specific contribution of a subset of stereoscopically modulated cortical neurons present in areas MT and MST.

Understanding mechanical motion: From images to behaviors

We present an algorithm for producing behavior descriptions of planar fixed axes mechanical motions from image sequences using a formal behavior language. The language, which covers the most important class of mechanical motions, symbolically captures the qualitative aspects of objects that translate and rotate along axes that are fixed in space. The algorithm exploits the structure of these motions to robustly recover the objects behaviors. It starts by identifying the independently moving objects, their motion parameters, and their variation with respect to time using normal optical flow analysis, iterative motion segmentation, and motion parameter estimation. It then produces a formal description of their behavior by identifying individual uniform motion events and simultaneous motion changes, and parsing them with a motion grammar. We demonstrate the algorithm on three sets of image sequences: mechanisms, everyday situations, and a robot manipulation scenario.

Optic flow illusion and single neuron behaviour reconciled by a population model.

Radial patterns of optic flow contain a centre of expansion that indicates the observer's direction of self-movement. When the radial pattern is viewed with transparently overlapping unidirectional motion, the centre of expansion appears to shift in the direction of the unidirectional motion [Duffy, C.J. & Wurtz, R.H. (1993) Vision Res., 33, 1481-1490]. Neurons in the medial superior temporal (MST) area of monkey cerebral cortex are thought to mediate optic flow analysis, but they do not shift their responses to parallel the illusion created by transparent overlap. The population-based model of optic flow analysis proposed by Lappe and Rauschecker replicates the illusory shift observed in perceptual studies [Lappe, M. & Rauschecker, J.P. (1995) Vision Res., 35, 1619-1631]. We analysed the behaviour of constituent neurons in the model, to gain insight into neuronal mechanisms underlying the illusion. Single model neurons did not show the illusory shift but rather graded variations of their response specificity. The shift required the aggregate response of the population. We compared the model's predictions about the behaviour of single neurons with the responses recorded from area MST. The predicted distribution of overlap effects agreed with that observed in area MST. The success of the population-based model in predicting the illusion and the neuronal behaviour suggests that area MST uses the graded responses of single neurons to create a population response that supports optic flow perception.

Optic Flow Selectivity in the Anterior Superior Temporal Polysensory Area, STPa, of the Behaving Monkey

Earlier studies of neurons in the anterior region of the superior temporal polysensory area (STPa) have demonstrated selectivity for visual motion using stimuli contaminated by nonmotion cues, including texture, luminance, and form. The present experiments investigated the motion selectivity of neurons in STPa in the absence of form cues using random dot optic flow displays. The responses of neurons were tested with translation, rotation, radial, and spiral optic flow displays designed to mimic the types of motion that occur during locomotion. Over half of the neurons tested responded significantly to at least one of these displays. On a cell by cell basis, 60% of the neurons tested responded selectively to rotation, radial, and spiral motion, whereas 20% responded selectively to translation motion. The majority of neurons responded maximally to single-component optic flow displays but was also significantly activated by the spiral displays that contained their preferred component. Moreover, there was a bias in the selectivity of the neurons for radial expansion motion. These results suggest that neurons within STPa are contributing to the analysis of optic flow. Furthermore, the preponderance of cells selective for radial expansion provides evidence that this area may be specifically involved in the processing of forward locomotion and/or looming stimuli. Finally, these results provide carefully controlled physiological evidence for an extension and specialization of the motion-processing pathway into the anterior temporal lobe.

On the quantum mechanics of optic flow and its application to driving in uncertain environments

The quantum mechanics approach is applied to the analysis of optic flow as a computational judgement model in order to develop a better understanding of car driver behaviour. It is argued that errors occur in the perception of distance, velocity and time because of a wave–image duality in the transfer of visual information and that this equates to fundamental uncertainty on the part of a driver between the variance in speed determination and foveal assignment. This quantum model, containing possibilities and probabilities, is contrasted to the ecological optic approach, which follows a classical view of deterministic assignments of information to trajectories from differentials in textural elements and gradients. Quantum mechanics can explain the occurrence of a number of motion-related perceptual phenomena. This includes the occlusion of images; driver fatigue; high speed adaptation and visual after-effect of motion; and the `white box' effect, where for an indifferent driver, limited illumination from night driving can increase speed and the underestimation of speed. These cases appear to happen when focus is bound to the road and the psychophysical energy of motion is discretely assigned beyond the retina. An ecological optic type approach may still be appropriate for low speed unbound vision in an energy continuum, where the peripheral field contains all the information for self-displacement assessment. Such modelling of the causes in the variance of driver behaviour has major implications for increasing driver safety and reducing road trauma.

Complex spike activity of Purkinje cells in the ventral uvula and nodulus of pigeons in response to translational optic flow.

The complex spike (CS) activity of Purkinje cells in the ventral uvula and nodulus of the vestibulocerebellum was recorded from anesthetized pigeons in response to translational optic flow. Translational optic flow was produced using a "translator" projector: a mechanical device that projected a translational optic flowfield onto the walls, ceiling, and floor of the room and encompassed the entire binocular visual field. CS activity was broadly tuned but maximally modulated in response to translational optic flow along a "best" axis. Each neuron was assigned a vector representing the direction in which the animal would need to translate to produce the optic flowfield that resulted in maximal excitation. The vector is described with reference to a standard right-handed coordinate system, where the vectors, +x, +y, and +z represent, rightward, upward, and forward translation of the animal, respectively. Neurons could be grouped into four response types based on the vector of maximal excitation. +y neurons were modulated maximally in response to a translational optic flowfield that results from self-motion upward along the vertical (y) axis. -y neurons also responded best to translational optic flow along the vertical axis but showed the opposite direction preference. The two remaining groups responded best to translational optic flow along horizontal axes: -x + z neurons and -x-z neurons. In summary, our results suggest that the olivocerebellar system dedicated to the analysis of translational optic flow is organized according to a reference frame consisting of three approximately orthogonal axes: the vertical axis, and two horizontal axes oriented 45 degrees to either side the midline. Previous research has shown that the rotational optic flow system, the eye muscles, the vestibular semicircular canals and the postural control system all share a similar spatial frame of reference.

Optic flow: A brain region devoted to optic flow analysis?

The visual motion – or optic flow – that results from an observer's own movement can indicate the direction of heading through the environment. Recent experiments have strengthened the argument that neurons in a specialized region of the cerebral cortex are critical for the analysis of this important class of visual stimuli.

Topographical organization of inferior olive cells projecting to translation and rotation zones in the vestibulocerebellum of pigeons

Previous electrophysiological studies in pigeons have shown that the vestibulocerebellum can be divided into two parasagittal zones based on responses to optic flow stimuli. The medial zone responds best to optic flow resulting from self-translation, whereas the lateral zone responds best to optic flow resulting from self-rotation. This information arrives from the retina via a projection from the accessory optic system to the medial column of the inferior olive. In this study we investigated inferior olive projections to translational and rotational zones of the vestibulocerebellum using the retrograde tracer cholera toxin subunit B. Extracellular recordings of Purkinje cell activity (complex spikes) in response to large-field visual stimuli were used to identify the injection sites. We found a distinct segregation of inferior olive cells projecting to translational and rotational zones of the vestibulocerebellum. Translation zone injections resulted in retrogradely labeled cells in the ventrolateral area of the medial column, whereas rotation zone injections resulted in retrogradely labeled cells in the dorsomedial region of the medial column. Motion of any object through space, including self-motion of organisms, can be described with reference to translation and rotation in three-dimensional space. Our results show that, in pigeons, the brainstem visual systems responsible for detecting optic flow are segregated into channels responsible for the analysis of translational and rotational optic flow in the inferior olive, which is only two synapses from the retina.

Optokinetic eye movements elicited by radial optic flow in the macaque monkey.

We recorded spontaneous eye movements elicited by radial optic flow in three macaque monkeys using the scleral search coil technique. Computer-generated stimuli simulated forward or backward motion of the monkey with respect to a number of small illuminated dots arranged on a virtual ground plane. We wanted to see whether optokinetic eye movements are induced by radial optic flow stimuli that simulate self-movement, quantify their parameters, and consider their effects on the processing of optic flow. A regular pattern of interchanging fast and slow eye movements with a frequency of 2 Hz was observed. When we shifted the horizontal position of the focus of expansion (FOE) during simulated forward motion (expansional optic flow), median horizontal eye position also shifted in the same direction but only by a smaller amount; for simulated backward motion (contractional optic flow), median eye position shifted in the opposite direction. We relate this to a change in Schlagfeld typically observed in optokinetic nystagmus. Direction and speed of slow phase eye movements were compared with the local flow field motion in gaze direction (the foveal flow). Eye movement direction matched well the foveal motion. Small systematic deviations could be attributed to an integration of the global motion pattern. Eye speed on average did not match foveal stimulus speed, as the median gain was only approximately 0.5-0.6. The gain was always lower for expanding than for contracting stimuli. We analyzed the time course of the eye movement immediately after each saccade. We found remarkable differences in the initial development of gain and directional following for expansion and contraction. For expansion, directional following and gain were initially poor and strongly influenced by the ongoing eye movement before the saccade. This was not the case for contraction. These differences also can be linked to properties of the optokinetic system. We conclude that optokinetic eye movements can be elicited by radial optic flow fields simulating self-motion. These eye movements are linked to the parafoveal flow field, i.e., the motion in the direction of gaze. In the retinal projection of the optic flow, such eye movements superimpose retinal slip. This results in complex retinal motion patterns, especially because the gain of the eye movement is small and variable. This observation has special relevance for mechanisms that determine self-motion from retinal flow fields. It is necessary to consider the influence of eye movements in optic flow analysis, but our results suggest that direction and speed of an eye movement should be treated differently.