Focus Of Expansion Research Articles

Effect of prism orientation on heading detection in optic flow

Recent studies of ambulatory behaviour indicate that people follow a curvilinear path when moving toward a target and wearing a prism. These results have been interpreted to indicate that heading judgments are based on moving toward a target not on optic flow. This interpretation assumes that if heading judgments were based on optic flow information they would not be altered by prism alignment. We tested this assumption. Thirteen observers made heading judgments using a relatively standard optic flow display. Observers were seated 50 cm from a display that subtended 44.6 arc deg in the horizontal dimension. The observers' task was to indicate if the focus of expansion (FoE) was to the right or left of centre. A double staircase was employed to determine right and left thresholds for the FoEs. Bias was the mean of the right and left thresholds. Precision was 1/2 the difference of the right and left thresholds. Observers repeated the judgments 3 times under 3 conditions: no prism and 2 prism conditions (15 diopters binocular, yoked base right and left). A 3× 3 Latin square design was used to avoid order effects. Wearing prisms had a statistically significant effect on subjects' bias (p < 0.05) without a statistically significant effect on their precision (p > 0.10). As prisms changed from 15° base left to 15° base right there was a shift to the right in the bias of observers' judgments of about 38 arc min (p < 0.05). 11 of the 13 observers showed this change. The precision of judgments of heading was unaffected. The difference between the precision of base left and right judgments averaged about 6 arc min. Prism orientation affects the bias of heading judgments in optic flow fields. Precision of heading judgments is unaffected.

A method of visual metrology from uncalibrated images

A method of measuring the height of any feature above a reference plane from a pair of uncalibrated images, separated by a (near) pure translation is presented. The output of the algorithm is a feature height, expressed as a fraction of the height of the camera above the reference plane. There are three contributions. Firstly a robust method of computing the dual epipole or focus of expansion (FOE) under pure translation is presented. Secondly, a novel reciprocal-polar (RP) image rectification scheme is presented, which allows planar image motion, expressed as a planar homography, to be accurately detected and recovered by 1D correlation. The technique can work even when there are no corner features on the reference plane and even over large image distortions caused by large camera motion, which would cause correlation techniques in the original image space to fail. Thirdly, we present a projective construct to enable measurement of the relative (or affine) feature height. Results show that our algorithm performs very well against outliers and noise. The mean of absolute error is 1.8 mm, and the mean of relative error is only 0.13% with two outliers removed.

Monocular obstacle detection using reciprocal-polar rectification

Our obstacle detection method is applicable to deliberative translation motion of a mobile robot and, in such motion, the epipole of each image of an image pair is coincident and termed the focus of expansion (FOE). We present an accurate method for computing the FOE and then we use this to apply a novel rectification to each image, called a reciprocal-polar (RP) rectification. When robot translation is parallel to the ground, as with a mobile robot, ground plane image motion in RP-space is a pure shift along an RP image scan line and hence can be recovered by a process of 1D correlation, even over large image displacements and without the need for corner matches. Furthermore, we show that the magnitude of these shifts follows a sinusoidal form along the second (orientation) dimension of the RP image. This gives the main result that ground plane motion over RP image space forms a 3D sinusoidal manifold. Simultaneous ground plane pixel grouping and recovery of the ground plane motion thus amounts to finding the FOE and then robustly fitting a 3D sinusoid to shifts of maximum correlation in RP space. The phase of the recovered sinusoid corresponds to the orientation of the vanishing line of the ground plane and the amplitude is related to the magnitude of the robot/camera translation. Recovered FOE, vanishing line and sinusoid amplitude fully define the ground plane motion (homography) across a pair of images and thus obstacles and ground plane can be segmented without any explicit knowledge of either camera parameters or camera motion.

A Vision Based Method for Aircraft Approach Angle Estimation

An algorithm for estimating aircraft approach angle using computer vision is proposed in this paper. During the landing of an aircraft, the descent of the aircraft may be approximately considered as a pure translation motion. In this case the epipole has the same coordinates in all images and is called the Focus-of-Expansion (FOE), and the vanishing line of the ground plane is termed the Horizon. Furthermore, how the Focus-of-Expansion and the Horizon are extracted from a few calibrated sequential images is first introduced, then the aircraft approach angle is derived from these parameters. Simulated and real experiments validate this algorithm.

Visually Perceived Eye Level and Horizontal Midline of the Body Trunk Influenced by Optic Flow

The eye level and the horizontal midline of the body trunk can serve, respectively, as references for judging the vertical and horizontal egocentric directions. We investigated whether the optic-flow pattern, which is the dynamic motion information generated when one moves in the visual world, can be used by the visual system to determine and calibrate these two references. Using a virtual-reality setup to generate the optic-flow pattern, we showed that judged elevation of the eye level and the azimuth of the horizontal midline of the body trunk are biased toward the positional placement of the focus of expansion (FOE) of the optic-flow pattern. Furthermore, for the vertical reference, prolonged viewing of an optic-flow pattern with lowered FOE not only causes a lowered judged eye level after removal of the optic-flow pattern, but also an overestimation of distance in the dark. This is equivalent to a reduction in the judged angular declination of the object after adaptation, indicating that the optic-flow information also plays a role in calibrating the extraretinal signals used to establish the vertical reference.

An illusory transformation of optic flow fields without local motion interactions

The focus of expansion (FOE) of a radially expanding optic flow pattern that is overlapped by unidirectional laminar flow is perceptually displaced in the direction of that laminar flow. There is continuing debate on whether this effect is due to local or global motion interactions. Here, we show psychophysically that under conditions without local motion transparency the illusion becomes weaker but can still be observed. In our experiments, the radial and laminar-flow fields were not presented with overlap but separately to the left and right halves of the visual field with a blank vertical strip of 15° horizontal width in between. The illusory shift observed in this condition cannot be explained by local motion interactions because (a) no transparent motion was present in the stimulus, and (b) the receptive fields of cortical cells involved in the analysis of local motion cross the vertical midline of the visual field to a limited extent. We conclude that global motion detectors that integrate motion from both halves of the visual field play a role in shifting the perceived position of the FOE and that local motion interactions may be sufficient, but are not necessary for the optic flow illusion to occur.

Computational analyses for illusory transformations in the optic flow field and heading perception in the presence of moving objects

When we see a stimulus of a radial flow field (the target flow) overlapped with a lateral flow field or another radial flow field, the focus of expansion (FOE) of the target radial flow appears to be shifted in a direction. Royden and Conti [(2003). A model using MT-like motion-opponent operators explains an illusory transformation in the optic flow field. Vision Research, 43, 2811–2826] argued that local motion subtraction is crucial for explanation of this phenomenon. The flow field which causes the illusory displacement of FOE was computationally analyzed. It was shown that the flow field is approximately a rigid-motion flow; the flow can be generated by simulating a situation where an observer moves toward a stationary scene. The heading direction for the observer corresponds to the perceived position of the FOE of the radial flow pattern. It implies that any algorithms which assume rigidity of the scene and recover veridical heading explain the bias in perceived FOE. There is no need for local motion subtraction in order to explain the phenomena. Furthermore, the flow for an observer’s translation in the presence of objects moving laterally or in depth was computationally analyzed. It was found that algorithms which minimizes standard error functions with less weights to the independently moving objects show similar biases in recovered heading to the bias of human observers. It implies that local motion subtraction is not necessary for explanation of the bias in perceived heading due to an object moving laterally or in depth, contrary to the argument of Royden [(2002). Computing heading in the presence of moving objects: a model that uses motion-opponent operators. Vision Research, 42, 3043–3058].

Perception of expanding optic flow patterns across the visual field

Purpose: To investigate the sensitivity to the direction of heading in expanding optic flow fields as a function of retinal location, with a view to determine the likely functional deficit in low vision patients with visual field loss. The results reported here are from a preliminary study investigating three normal vision observers. Methods: Sensitivity to the focus of expansion (FoE) was measured with 0.5 second expanding random Gaussian dot patterns as a function of retinal location. Accuracy was measured by positioning a cursor on the perceived FoE. Observers also discriminated whether the FoE was left or right of centre in an equivalent noise paradigm in which positional variance was applied to the FoE for each dot. Finally to determine FoE resolution, observers discriminated an interval containing two FoEs from an interval containing only one. Results: The accuracy of locating the FoE was independent of eccentricity and location of the expanding optic flow in the visual field. This was confirmed by the equivalent noise measurements which showed that internal noise did not vary with eccentricity but efficiency decreased somewhat. The separation between two FoEs had to increase with eccentricity before an observer could resolve them. Conclusions: Sensitivity to expanding optic flow patterns is generally invariant across the field for both identifying the position of the FoE and for integrating the motions of the dots defining it. The results suggest that mechanisms with large receptive fields pool directional information across the visual field. The loss in efficiency indicates that the number of elements integrated increases with eccentricity and this also accounts for the loss of resolution. The Wellcome Trust

Pseudo-linearizing collinearity constraint for accurate pose estimation from a single image

The goal of this paper is twofold: firstly, we propose a novel interpretation for collinearity in the process of camera pose estimation from given correspondences between a 3D model and its 2D projective image. In contrast with the existing interpretations for collinearity, the focus of expansion (FOE) theory is a special case of our novel interpretation for collinearity and besides the projection of camera position on the image plane, every image point can become a FOE. Secondly, we propose a novel method, based on the collinearity equation, for camera pose estimation from given point correspondences between a 3D model and its projective image. A comparative study based on both synthetic data and real images has shown that the novel algorithm is promising.

MSTd neuronal basis functions for the population encoding of heading direction.

Basis functions have been extensively used in models of neural computation because they can be combined linearly to approximate any nonlinear functions of the encoded variables. We investigated whether dorsal medial superior temporal (MSTd) area neurons use basis functions to simultaneously encode heading direction, eye position, and the velocity of ocular pursuit. Using optimal linear estimators, we first show that the head-centered and eye-centered position of a focus of expansion (FOE) in optic flow, pursuit direction, and eye position can all be estimated from the single-trial responses of 144 MSTd neurons with an average accuracy of 2-3 degrees, a value consistent with the discrimination thresholds measured in humans and monkeys. We then examined the format of the neural code for the head-centered position of the FOE, eye position, and pursuit direction. The basis function hypothesis predicts that a large majority of cells in MSTd should encode two or more signals simultaneously and combine these signals nonlinearly. Our analysis shows that 95% of the neurons encode two or more signals, whereas 76% code all three signals. Of the 95% of cells encoding two or more signals, 90% show nonlinear interactions between the encoded variables. These findings support the notion that MSTd may use basis functions to represent the FOE in optic flow, eye position, and pursuit.

The role of vision in maintaining heading direction: effects of changing gaze and optic flow on human gait.

How is heading direction maintained in human gait? This question was investigated with respect to the role of optic flow and in the context of different movement strategies. While walking on a treadmill the deviation from the ideal straight path was measured in terms of lateral sway induced by a lateral gaze shift (by looking at a moving visual target). The role of the focus of expansion (FOE) within a radially expanding optic flow pattern was investigated by varying its relative velocity of expansion from 0- to 4-fold (the equivalent of walking speed), thus increasing the perceptibility of FOE. If FOE was a relevant cue for maintaining heading direction, a reduction of lateral sway amplitude was expected with increasing flow velocity. The presence of a radially expanding flow pattern did not reduce lateral sway. Lateral sway was least when the visual background remained stable without any flow pattern. Increasing the velocity of the flow pattern resulted in an increase in lateral sway. If the relative velocity of the flow pattern was raised beyond that corresponding to walking speed, lateral sway amplitude approached the maximal values observed in the dark. In all experiments, sway amplitude increased linearly with the increasing excursion of the visual target. Different strategies to perform the gaze shift (eye or head turns) only resulted in minor differences in lateral sway amplitude. The results show that gaze shifts during locomotion induce lateral sway, which depends upon the presence, and characteristics, of background optic flow. Under the present conditions, the FOE within the flow field seems not to be a dominant cue to control heading. However, the systematic increase in lateral sway induced by high flow velocities indicates that motion parallax has an effect on heading during locomotion.

Effect of Color Coding on Misperception of Heading from Superimposed Optic Flow

If dynamic sensor imagery is overlaid on the view seen through the windscreen using a HUD, and if the sensor is misaligned with the pilot's eye, the resulting motion transparency can lead to systematic errors in perceiving the direction of heading. We examined whether coding misaligned, dynamic imagery in distinct colors aids pre-attentive segregation of transparent optical flow, thus eliminating errors in heading perception. Observers were shown displays overlaying radially expanding optic flow patterns with foci of expansion (FOEs) separated along the horizontal meridian and indicated the perceived direction of the FOE of one flow pattern by moving an on-screen cross-hair. We found significant repulsion errors when the angular separation between the FOEs exceeded four degrees. Further, coding the flow patterns in distinct colors did not eliminate the repulsion errors. Distinctive color coding alone appears to be insufficient for overcoming errors in heading perception induced by motion transparency.

MST responses to pursuit across optic flow with motion parallax.

Self-movement creates the patterned visual motion of optic flow with a focus of expansion (FOE) that indicates heading direction. During pursuit eye movements, depth cues create a retinal flow field that contains multiple FOEs, potentially complicating heading perception. Paradoxically, human heading perception during pursuit is improved by depth cues. We have studied medial superior temporal (MST) neurons to see whether their heading selectivity is also improved under these conditions. The responses of 134 MST neurons were recorded during the presentation of optic flow stimuli containing one or three speed-defined depth planes. During pursuit, multiple depth-plane stimuli evoked larger responses (71% of neurons) and stronger heading selectivity (70% of neurons). Responses to the three speed-defined depth-planes presented separately showed that most neurons (54%) preferred one of the planes. Responses to multiple depth-plane stimuli were larger than the averaged responses to the three component planes, suggesting enhancing interactions between depth-planes. Thus speed preferences create selective responses to one of many depth-planes in the retinal flow field. The presence of multiple depth-planes enhances those responses. These properties might improve heading perception during pursuit and contribute to relative depth perception.

Extracting Structure from Optical Flow Using the Fast Error Search Technique

In this paper, we present a globally optimal and computationally efficient technique for estimating the focus of expansion (FOE) of an optical flow field, using fast partial search. For each candidate location on a discrete sampling of the image area, we generate a linear system of equations for determining the remaining unknowns, viz. rotation and inverse depth. We compute the least squares error of the system without actually solving the equations, to generate an error surface that describes the goodness of fit across the hypotheses. Using Fourier techniques, we prove that given an N × N flow field, the FOE, and subsequently rotation and structure, can be estimated in {\cal O}(N^2 log N) operations. Since the resulting system is linear, bounded perturbations in the data lead to bounded errors. We support the theoretical development and proof of our technique with experiments on synthetic and real data. Through a series of experiments on synthetic data, we prove the correctness, robustness and operating envelope of our algorithm. We demonstrate the utility of our technique by applying it for detecting obstacles from a monocular sequence of images.

Model-based analytical FOE determination

An analytical procedure for the estimation of the focus of expansion (FOE) location is introduced. The method is applied to a vector field which has been obtained from the analysis of the relative translational movement of a rigid body with respect to the acquiring camera (monocular system). A polynomial model is fitted to the vector field; the order and shape of the model are determined by the kinetic analysis of the Euclidean camera centered coordinate system in which the camera is moving, and some hypothesis concerning the approximation of the inverse of the depth function 1/Z(x, y) . Two basic properties of complex differentiation theory are applied, assuming that each vector of the optic flow is a real-valued complex function of the form v(x, y)=v x (x, y)+ i v y (x, y) . These properties provide the set of points for which both components of the vector field v x (x, y) and v y (x, y) are harmonic and, as a subset, the set of points for which both components are zero. The use of perspective projection states that the FOE is the locus where v x (x, y)=0=v y (x, y) and, therefore, where the conditions stated by both properties meet.

Dynamical Estimation of Camera Motion Parameters Using Focus of Expansion

A dynamical vehicle-mounted camera motion parameters' estimation method with the introduction of camera's Focus of Expansion (FOE) is presented here. Compared with other approaches like using line matching of road lane-marks or tracking the horizon, proposed method purely uses points matching between the adjacent frames without need of any previous knowledge or recognition results. This method is able to determine camera's rotation and translation parameters by using only 3 pairs of matching points and thus more suitable for real-time applications. Experiments on image sequences of real road scenes show the effectiveness and precision of proposed method.

MST neuronal responses to heading direction during pursuit eye movements.

As you move through the environment, you see a radial pattern of visual motion with a focus of expansion (FOE) that indicates your heading direction. When self-movement is combined with smooth pursuit eye movements, the turning of the eye distorts the retinal image of the FOE but somehow you still can perceive heading. We studied neurons in the medial superior temporal area (MST) of monkey visual cortex, recording responses to FOE stimuli presented during fixation and smooth pursuit eye movements. Almost all neurons showed significant changes in their FOE selective responses during pursuit eye movements. However, the vector average of all the neuronal responses indicated the direction of the FOE during both fixation and pursuit. Furthermore, the amplitude of the net vector increased with increasing FOE eccentricity. We conclude that neuronal population encoding in MST might contribute to pursuit-tolerant heading perception.

Determining Motion Parameters for V ehicle-Mounted Camera Using Focus of Expansion

Camera's motion parameters play a vital role in the research of autonomous vehicle's vision navigation and dynamic image sequences analysis. We propose here a fast motion parameters' estimation method with the use of stationary camera's Focus of Expansion (FOE). Contrary to general methods like using optical flow and feature matching, our approach is able to determine camera's rotation and translation parameters by using only 3 pairs of matching points. A fast matching algorithm of feature points extracted by color gradient edge detector (CGD) and localized contour sequences (LCS) is also presented in this paper. Experiments on image sequences of synthetic and real road scenes showed the of fectiveness and precision of our approach.

Guidance of locomotion on foot uses perceived target location rather than optic flow

What visual information do we use to guide movement through our environment? Self-movement produces a pattern of motion on the retina, called optic flow. During translation, the direction of movement (locomotor direction) is specified by the point in the flow field from which the motion vectors radiate – the focus of expansion (FoE) [1–3]. If an eye movement is made, however, the FoE no longer specifies locomotor direction [4], but the ‘heading’ direction can still be judged accurately [5]. Models have been proposed that remove confounding rotational motion due to eye movements by decomposing the retinal flow into its separable translational and rotational components ([6,7] are early examples). An alternative theory is based upon the use of invariants in the retinal flow field [8]. The assumption underpinning all these models (see also [9–11]), and associated psychophysical [5,12,13] and neurophysiological studies [14–16], is that locomotive heading is guided by optic flow. In this paper we challenge that assumption for the control of direction of locomotion on foot. Here we have explored the role of perceived location by recording the walking trajectories of people wearing displacing prism glasses. The results suggest that perceived location, rather than optic or retinal flow, is the predominant cue that guides locomotion on foot.

Global induced motion and visual stability in an optic flow illusion.

When an expansion flow field of moving dots is overlapped by planar motion, observers perceive an illusory displacement of the focus of expansion (FOE) in the direction of the planar motion (Duffy and Wurtz, Vision Research, 1993;33:1481-1490). The illusion may be a consequence of induced motion, wherein an induced component of motion relative to planar dots is added to the motions of expansion dots to produce the FOE shift. While such a process could be mediated by local 'center-surround' receptive fields, the effect could also be due to a higher level process which detects and subtracts large-field planar motion from the flow field. We probed the mechanisms underlying this illusion by adding varying amounts of rotation to the expansion stimulus, and by varying the speed and size of the planar motion field. The introduction of rotation into the stimulus produces an illusory shift in a direction perpendicular to the planar motion. Larger FOE shifts were perceived for greater speeds and sizes of planar motion fields, although the speed effect saturated at high speeds. While the illusion appears to share a common mechanism with center-surround induced motion, our results also point to involvement of a more global mechanism that subtracts coherent planar motion from the flow field. Such a process might help to maintain visual stability during eye movements.