Global Image Motion Research Articles

Asymmetric retinal direction tuning predicts optokinetic eye movements across stimulus conditions.

Across species, the optokinetic reflex (OKR) stabilizes vision during self-motion. OKR occurs when ON direction-selective retinal ganglion cells (oDSGCs) detect slow, global image motion on the retina. How oDSGC activity is integrated centrally to generate behavior remains unknown. Here, we discover mechanisms that contribute to motion encoding in vertically tuned oDSGCs and leverage these findings to empirically define signal transformation between retinal output and vertical OKR behavior. We demonstrate that motion encoding in vertically tuned oDSGCs is contrast-sensitive and asymmetric for oDSGC types that prefer opposite directions. These phenomena arise from the interplay between spike threshold nonlinearities and differences in synaptic input weights, including shifts in the balance of excitation and inhibition. In behaving mice, these neurophysiological observations, along with a central subtraction of oDSGC outputs, accurately predict the trajectories of vertical OKR across stimulus conditions. Thus, asymmetric tuning across competing sensory channels can critically shape behavior.

Motion Vision: A New Mechanism in the Mammalian Retina.

In animal eyes, the detection of slow global image motion is crucial to preventing blurry vision. A new study reveals how a mammalian global motion detector achieves this through 'space-time wiring' at its dendrites.

Neuronal Architecture of a Visual Center that Processes Optic Flow

Animals use global image motion cues to actively stabilize their position by compensatory movements. Neurons in the zebrafish pretectum distinguish different optic flow patterns, e.g., rotation and translation, to drive appropriate behaviors. Combining functional imaging and morphological reconstruction of single cells, we revealed critical neuroanatomical features of this sensorimotor transformation. Terminals of direction-selective retinal ganglion cells (DS-RGCs) are located within the pretectal retinal arborization field 5 (AF5), where they meet dendrites of pretectal neurons with simple tuning to monocular optic flow. Translation-selective neurons, which respond selectively to optic flow in the same direction for both eyes, are intermingled with these simple cells but do not receive inputs from DS-RGCs. Mutually exclusive populations of pretectal projection neurons innervate either the reticular formation or the cerebellum, which in turn control motor responses. We posit that local computations in a defined pretectal circuit transform optic flow signals into neural commands driving optomotor behavior. VIDEO ABSTRACT.

Algorithm of global image motion detection based on remote images and engineering parameters

相邻两片成像传感器的拼接区能够形成具有相同目标景物的图像, 是提炼像移信息的关键, 具有相同目标景物的图像并非贯穿于每次成像任务全程, 无相同目标景物期间的像移估计问题仍然存在。提出基于遥感图像及工程参数的全局像移探测算法, 试图对包含无相同目标景物期间在内的像移进行全覆盖探测。根据拼接区成像特点, 将成像过程分为无相同目标景物和有相同目标景物两个阶段:在无相同目标景物阶段, 利用工程参数计算像移的低频分量, 建立基于拼接区图像和像移低频分量的目标函数, 衡量像移曲线与低频分量之间的偏离程度; 采用最速下降法搜索偏离度最小值点, 作为无相同目标景物期间像移的最优估计; 推导有相同目标景物期间的像移计算公式, 利用无相同目标景物期间的估计值求解像移。以XX-1号空间光学遥感器为实验对象, 检测到0.133 Hz、7 pixels左右的像移, 可探测的时间范围包含最初226 ms的无相同目标景物阶段, 且显著削弱了盲点处的像移测量偏差。实验结果证明, 提出的算法能够对包含无相同目标景物期间的全局像移进行有效测量。

Image-Guided Nanopositioning Scheme for SEM

Positioning of micro-nanoobjects inside a scanning electron microscope (SEM) for manipulation is a key and challenging task to perform. Often it is performed by skilled operators via teleoperation, which is tedious and lacks repeatability. In this paper, rendering this task as an image-guided problem, we present a frequency domain scheme for automatic control of positioning platform movements. The designed controller uses the relative global image motion computed using the frequency spectral information of the images as visual signal and can provide control up to five degrees of freedom. The proposed approach is validated in simulations as well as experimentally using a high-resolution piezo-positioning platform mounted inside a SEM vacuum chamber. The obtained results quantify the performance of the proposed nanopositioning scheme.

Local processing in neurites of VGluT3-expressing amacrine cells differentially organizes visual information.

Neurons receive synaptic inputs on extensive neurite arbors. How information is organized across arbors and how local processing in neurites contributes to circuit function is mostly unknown. Here, we used two-photon Ca2+ imaging to study visual processing in VGluT3-expressing amacrine cells (VG3-ACs) in the mouse retina. Contrast preferences (ON vs. OFF) varied across VG3-AC arbors depending on the laminar position of neurites, with ON responses preferring larger stimuli than OFF responses. Although arbors of neighboring cells overlap extensively, imaging population activity revealed continuous topographic maps of visual space in the VG3-AC plexus. All VG3-AC neurites responded strongly to object motion, but remained silent during global image motion. Thus, VG3-AC arbors limit vertical and lateral integration of contrast and location information, respectively. We propose that this local processing enables the dense VG3-AC plexus to contribute precise object motion signals to diverse targets without distorting target-specific contrast preferences and spatial receptive fields.

Inhibitory Control of Feature Selectivity in an Object Motion Sensitive Circuit of the Retina

Object motion sensitive (OMS) W3-retinal ganglion cells (W3-RGCs) in mice respond to local movements in a visual scene but remain silent during self-generated global image motion. The excitatory inputs that drive responses of W3-RGCs to local motion were recently characterized, but which inhibitory neurons suppress W3-RGCs' responses to global motion, how these neurons encode motion information, and how their connections are organized along the excitatory circuit axis remains unknown. Here, we find that a genetically identified amacrine cell (AC) type, TH2-AC, exhibits fast responses to global motion and slow responses to local motion. Optogenetic stimulation shows that TH2-ACs provide strong GABAA receptor-mediated input to W3-RGCs but only weak input to upstream excitatory neurons. Cell-type-specific silencing reveals that temporally coded inhibition from TH2-ACs cancels W3-RGC spike responses to global but not local motion stimuli and, thus, controls the feature selectivity of OMS signals sent to the brain.

Joint Encoding of Object Motion and Motion Direction in the Salamander Retina.

The retina contains specialized cells for motion processing. Among the retinal ganglion cells, which form the output neurons of the retina, some are known to report the direction of a moving stimulus (direction-selective cells), and others distinguish the motion of an object from a moving background. But little is known about how information about local object motion and information about motion direction interact. Here, we report that direction-selective ganglion cells can be identified in the salamander retina, where their existence had been unclear. Furthermore, there are two independent systems of direction-selective cells, and one of these combines direction selectivity with sensitivity to local motion. The output of these cells could assist in tracking moving objects and estimating their future position.

Automated measurement of spectral sensitivity of motion vision during optokinetic behavior

Optokinetic behavior is a commonly used paradigm to conveniently determine sensitivities of motion vision in animals. The optokinetic reflex (OKR) compensates global image motion on the retina by head and eye movements. It can reliably be elicited without training the test animal. In most OKR behavior experiments the animal is stimulated either with a rotating drum containing a pattern of vertical stripes or with computer monitors displaying a moving stripe pattern. In many of these studies, sensitivity thresholds are measured based on the subjective judgment of the experimenter. In this paper, we describe an alternative method to induce and measure OKR behavior. Our setup consists of a metal drum into which a slide is projected via a 360° panoramic mirror, using a spectrally filtered LED as light source. The slide is mounted on a motor-driven turntable whose rotation leads to a horizontal movement of the stimulus on the drum’s wall, accordingly. By this means a spatial, temporal, and spectral well-defined and flexible panoramic stimulation is achieved. The animal’s head movements are video recorded under infrared illumination and tracked online. We introduce an objective criterion to automatically determine sensitivity thresholds based on the correlation of the animal’s head angle with the stimulus position. We exemplarily used this setup for an experiment that could not be performed with the state-of-the-art setup consisting of four monitors—the measurement of spectral sensitivity thresholds of the OKR behavior in turtles.

Feasibility and reproducibility of biventricular volumetric assessment of cardiac function during exercise using real‐time radial k‐t SENSE magnetic resonance imaging

To assess the feasibility and reproducibility of real-time radial k-t sensitivity encoding (SENSE) magnetic resonance imaging (MRI) for biventricular volumetric assessment during exercise. In all, 12 healthy young adults underwent MRI at rest and during supine exercise at three different workload intensities. Biventricular volumes and function were assessed with 1) a radial k-t SENSE real-time sequence and 2) a scanner vendor supplied (standard) real-time sequence. Global image quality, motion fidelity, and agreement in right ventricular (RV) and left ventricular (LV) stroke volume (SV) as a surrogate measure for accuracy were assessed. Exercise MR was repeated within 1 month for assessment of reproducibility. Imaging scores were superior for radial real-time k-t SENSE images (P < 0.001). Agreement in RV and LV SV during exercise was better with radial k-t real-time (SD of difference +/-3.43 vs. +/-8.97 mL; P < 0.001). Agreement in cardiac output (CO) in the same subject at two different imaging sessions was better for radial k-t SENSE. This was significant for the CO calculated for the RV (SD of difference +/-0.6 vs. +/-0.95 L/min; P = 0.01) and LV (+/-0.45 vs. +/-0.92 L/min; P < 0.001). Radial k-t SENSE real-time imaging represents a feasible and reproducible imaging technique for biventricular assessment during exercise.

Real-time Assessment of Right and Left Ventricular Volumes and Function in Patients with Congenital Heart Disease by Using High Spatiotemporal Resolution Radial k-t SENSE

The purpose of this study was to compare ventricular volumes in patients with congenital heart disease measured by using (a) a cardiac gated sequence, (b) a standard real-time sequence, and (c) a radial real-time k-space and time (k-t) sensitivity encoding (SENSE) sequence. The local research ethics committee approved this study, and written consent was obtained from all participants. Of 40 patients with congenital heart disease, ventricular volumes were measured by using the three sequences. Global image quality and motion fidelity were scored and compared with a Wilcoxon signed rank test. Image contrast, edge sharpness, and summed perimeters (the total length of the endocardial tracings for a given ventricle at systole and diastole) were quantified and compared by using paired t tests. Ventricular volumes were compared with paired t tests, Bland-Altman analysis, and correlation coefficients. Global image quality, motion fidelity, image contrast, edge sharpness, and summed perimeters were all greater for radial real-time k-t SENSE imaging compared with standard real-time imaging (P < .05). However, the gated acquisitions were significantly superior to radial real-time k-t SENSE (P < .05). For cardiac gated versus radial k-t real-time acquisitions, there was no difference between right ventricular (RV) volumes and ejection fraction (EF) (P > .15). There was a small difference in left ventricular (LV) end-diastolic volume (EDV) and thus, LV stroke volume and EF (P < .05). For cardiac gated versus standard real-time acquisitions, both RV and LV EDV and thus, stroke volume and EF were significantly lower (P < .05). Ventricular volumes and function can be accurately quantified by using radial k-t SENSE real-time imaging.

Capture of attention by scene-relative object movement during self-movement

If the observer is not moving object movement is uniquely signalled by retinal image motion. Under such circumstances a moving object captures attention. During self-movement the global image motion that results from movement of the eyes may mask the image motion that signals object movement. We investigated whether a moving object captures attention in this case. Through stereo glasses, observers viewed 24 objects arranged in a 3-D volume. Four of the objects were potential target objects that could change shape. During viewing the camera viewpoint was translated and rotated to simulate self-movement. Observers were required to identify a shape change as rapidly as possible. On every 35th trial an object within the scene would unexpectedly move at approximately the same time as the shape change. When the object that changed shape was also the object that moved, response time was shorter that when a different object moved; this is the hallmark of attentional capture. Therefore we conclude that during self-movement, unexpected object movement captures attention.

Affine Motion Based CMOS Distortion Analysis and CMOS Digital Image Stabilization

The CMOS image distortion is due to the rolling shutter in CMOS image sensors (CISs) and it can be more exaggerated when a CIS camera moves rapidly. Several methods have been proposed to remove CMOS distortions made by the translational motion. But, in this paper, we propose the affine motion based CMOS distortion correction method combined with digital image stabilization. To remove CMOS distortions due to the affine global image motion, CMOS distortion model is proposed to explain the effect of the affine global image motion on the CMOS distortion in CISs. To improve CMOS video's visuals, we propose CMOS digital image stabilization to remove the jittering motions in the new image sequence obtained by our correction method. In addition, to reduce the computational time and the outlier effect, a reliable feature selection method is proposed to be used in the affine global image motion estimation. In the experiment results, we show that the proposed CMOS distortion correction method is more general than previous ones. Also, we show that our stabilization method can improve CMOS video's visuals running in real-time.

CMOS Digital Image Stabilization

In this paper, we present a CMOS digital image stabilization algorithm based on the characteristics of a rolling shutter camera. Due to the rolling shuttering mechanism of a CMOS sensor, a CMOS video frame shows CMOS distortions which are not observed in a CCD video frame, and previous video stabilization techniques cannot handle these distortions properly even though they can make a visually stable CMOS video sequence. In our proposed algorithm, we first suggest a CMOS distortion model. This model is based on a rolling shutter mechanism which provides a solution to solve the CMOS distortion problem. Next, we estimate the global image motion and the CMOS distortion transformation directly from the homography between CMOS frames. Using the two transformations, we remove CMOS distortions as well as jittering motions in a CMOS video. In the experimental results, we demonstrate that our proposed algorithm can handle the CMOS distortion problem more effectively as well as the jittering problem in a CMOS video compared to previous CCD-based digital image stabilization techniques.

Linear global mosaics for underwater surveying

An important feature for autonomous underwater vehicles equipped with video cameras in survey missions, is the ability to quickly generate a wide area view of the sea floor. This paper presents a method for the fast creation of globally consistent video mosaics. A closed-form solution for the estimation of the global image motion is presented. It uses a least-squares criteria over a residual vector which is linear on the homography parameters. Aiming at real-time operation, a fast implementation is described using recursive least-squares, which permits the creation of globally consistent mosaics during video acquisition. The application to underwater imagery is illustrated by the creation of video mosaics capable of being used for surveying or autonomous navigation.

Correction of image instability in confocal microscopy using image realignment: Effects on the analysis of intracellular calcium

Using confocal microscopy, we have examined the increases in [Ca 2+] i evoked by sodium channel toxins in cells labelled with the fluorescent dye INDO-1. We describe a new image analysis method that improves the detection of region-specific, toxin-induced patterns of change of intracellular calcium. This method is based on correction of global image motion followed by calculation of the strength of correlation between calcium changes in “seed” or reference pixels chosen to represent different regions of cells and those in other regions of the image. When the selected “seed” pixel was chosen to be in either varicosities or neurites, correlations were detected in the same regions of other cells as well as in the soma, indicating specific but spatially distinct patterns of behaviour. Control images (without changes in [Ca 2+] i) did not reveal significant interpixel correlations. The ability to recognize correlated patterns of calcium change in different regions of cells was greatly improved by correction for global motion.

Segregation of object and background motion in the retina.

An important task in vision is to detect objects moving within a stationary scene. During normal viewing this is complicated by the presence of eye movements that continually scan the image across the retina, even during fixation. To detect moving objects, the brain must distinguish local motion within the scene from the global retinal image drift due to fixational eye movements. We have found that this process begins in the retina: a subset of retinal ganglion cells responds to motion in the receptive field centre, but only if the wider surround moves with a different trajectory. This selectivity for differential motion is independent of direction, and can be explained by a model of retinal circuitry that invokes pooling over nonlinear interneurons. The suppression by global image motion is probably mediated by polyaxonal, wide-field amacrine cells with transient responses. We show how a population of ganglion cells selective for differential motion can rapidly flag moving objects, and even segregate multiple moving objects.

Method and apparatus for generating and displaying panoramic ultrasound images

An image registration based method and a corresponding apparatus are provided for creating composite, panoramic images, in which consecutive ultrasonic image frames are correlated in order to derive transducer motion information. The individual images can be divided into several smaller sub-image regions and a fast and robust image motion detection algorithm is used to measure the sub-image motion and then to determine an initial estimation of the local motion vectors. The initial estimation is combined with two measurement parameters by use of a fuzzy logic technique to derive a final estimation of the local motion vectors. The final local motion vectors are then used to estimate global image motion. Compound images are then diplayed based on the derived global image motion. The speed of motion of the transducer is also estimated and a "speedometer" is displayed to help the user move the transducer within efficient speed ranges. Image frames may also be selectively "zoomed" in on and displayed in an undistorted manner. Distance scales are generated for both the normally curved panoramic image and also the undistorted frame images. A frame guide is also displayed to help the user guide the transducer to area of greatest interest. Color-coded velocity or power information may also be superimposed on the panoramic image, and spectral Doppler data may also be displayed at selected points in the panoramic image. A memory map is preferably also displayed to indicate to the user how much image memory is remaining during a scan.

Method and apparatus for generating large compound ultrasound image

An image registration method and apparatus is provided in which consecutive ultrasonic image frames are correlated in order to derive transducer motion information. Since ultrasonic images from real-time scanning operations are highly correlated from one frame to another, the individual images can be divided into several smaller sub-image regions and a very fast and robust image motion detection algorithm utilized to measure the sub-image motion. This algorithm uses a fast adaptive coarse/fine minimum-sum-absolute-difference (MSAD) search strategy to compute an initial estimation of the local motion vectors. The initial estimation is combined with two measurement parameters by use of a fuzzy logic technique to derive a final estimation of the local motion vectors. The final local motion vectors are applied to a least-squares (L-S) process in order to estimate global image motion. Finally, a fast display technique generates compound images based on the derived global image motion.