Local Motion Detectors Research Articles

Neural mechanisms to incorporate visual counterevidence in self-movement estimation

In selecting appropriate behaviors, animals should weigh sensory evidence both for and against specific beliefs about the world. For instance, animals measure optic flow to estimate and control their own rotation. However, existing models of flow detection can be spuriously triggered by visual motion created by objects moving in the world. Here, we show that stationary patterns on the retina, which constitute evidence against observer rotation, suppress inappropriate stabilizing rotational behavior in the fruit fly Drosophila. In silico experiments show that artificial neural networks (ANNs) that are optimized to distinguish observer movement from external object motion similarly detect stationarity and incorporate negative evidence. Employing neural measurements and genetic manipulations, we identified components of the circuitry for stationary pattern detection, which runs parallel to the fly's local motion and optic-flow detectors. Our results show how the fly brain incorporates negative evidence to improve heading stability, exemplifying how a compact brain exploits geometrical constraints of the visual world.

Bioinspired figure-ground discrimination via visual motion smoothing

Flies detect and track moving targets among visual clutter, and this process mainly relies on visual motion. Visual motion is analyzed or computed with the pathway from the retina to T4/T5 cells. The computation of local directional motion was formulated as an elementary movement detector (EMD) model more than half a century ago. Solving target detection or figure-ground discrimination problems can be equivalent to extracting boundaries between a target and the background based on the motion discontinuities in the output of a retinotopic array of EMDs. Individual EMDs cannot measure true velocities, however, due to their sensitivity to pattern properties such as luminance contrast and spatial frequency content. It remains unclear how local directional motion signals are further integrated to enable figure-ground discrimination. Here, we present a computational model inspired by fly motion vision. Simulations suggest that the heavily fluctuating output of an EMD array is naturally surmounted by a lobula network, which is hypothesized to be downstream of the local motion detectors and have parallel pathways with distinct directional selectivity. The lobula network carries out a spatiotemporal smoothing operation for visual motion, especially across time, enabling the segmentation of moving figures from the background. The model qualitatively reproduces experimental observations in the visually evoked response characteristics of one type of lobula columnar (LC) cell. The model is further shown to be robust to natural scene variability. Our results suggest that the lobula is involved in local motion-based target detection.

Direction selectivity of the retinotectal system of fish: Findings based on microelectrode extracellular recordings of the tectum opticum

Vision in fish plays an important role in different forms of visually guided behavior. The visual system of fish is available for research by different methods; it is a convenient experimental model for studying and understanding the mechanisms of vision in general. Responses of retinal direction-selective (DS) ganglion cells (GCs) are recorded extracellularly from their axon terminals in the superficial layers of the tectum opticum (TO). They can be divided into three distinct groups according to the preferred directions of stimulus movement: caudorostral, dorsoventral and ventrodorsal. Each of these groups comprises both ON and OFF units in equal proportions. Relatively small receptive fields (3-8?) and fine spatial resolution characterize retinal DS units as local motion detectors. Conversely, the responses of direction-selective tectal neurons (DS TNs) are recorded at two different tectal levels, deeper than the zone of retinal DS afferents. They are characterized by large receptive fields (up to 60?) and are indifferent to any sign of contrast, i.e., they can be considered as ON-OFF-type units. Four types of ON-OFF DS TNs preferring different directions of motion have been recorded. The preferred directions of three types of DS TNs match the preferred directions of three types of DS GCs. Matching the three preferred directions of ON and OFF DS GCs and ON-OFF DS TNs has allowed us to hypothesize that the GCs with caudorostral, ventrodorsal and dorsoventral preferences are input neurons for the corresponding types of DS TNs. On the other hand, the rostrocaudal preference in the fourth type of DS TNs, recorded exclusively in the deep tectal zone, is an emergent property of the TO. In this review, our findings are compared with the results of other authors examining direction selectivity in the fish retinotectal system.

Disentangling of Local and Wide-Field Motion Adaptation.

Motion adaptation has been attributed in flying insects a pivotal functional role in spatial vision based on optic flow. Ongoing motion enhances in the visual pathway the representation of spatial discontinuities, which manifest themselves as velocity discontinuities in the retinal optic flow pattern during translational locomotion. There is evidence for different spatial scales of motion adaptation at the different visual processing stages. Motion adaptation is supposed to take place, on the one hand, on a retinotopic basis at the level of local motion detecting neurons and, on the other hand, at the level of wide-field neurons pooling the output of many of these local motion detectors. So far, local and wide-field adaptation could not be analyzed separately, since conventional motion stimuli jointly affect both adaptive processes. Therefore, we designed a novel stimulus paradigm based on two types of motion stimuli that had the same overall strength but differed in that one led to local motion adaptation while the other did not. We recorded intracellularly the activity of a particular wide-field motion-sensitive neuron, the horizontal system equatorial cell (HSE) in blowflies. The experimental data were interpreted based on a computational model of the visual motion pathway, which included the spatially pooling HSE-cell. By comparing the difference between the recorded and modeled HSE-cell responses induced by the two types of motion adaptation, the major characteristics of local and wide-field adaptation could be pinpointed. Wide-field adaptation could be shown to strongly depend on the activation level of the cell and, thus, on the direction of motion. In contrast, the response gain is reduced by local motion adaptation to a similar extent independent of the direction of motion. This direction-independent adaptation differs fundamentally from the well-known adaptive adjustment of response gain according to the prevailing overall stimulus level that is considered essential for an efficient signal representation by neurons with a limited operating range. Direction-independent adaptation is discussed to result from the joint activity of local motion-sensitive neurons of different preferred directions and to lead to a representation of the local motion direction that is independent of the overall direction of global motion.

Heterogeneous Temporal Contrast Adaptation in Drosophila Direction-Selective Circuits.

In visual systems, neurons adapt both to the mean light level and to the range of light levels, or the contrast. Contrast adaptation has been studied extensively, but it remains unclear how it is distributed among neurons in connected circuits, and how early adaptation affects subsequent computations. Here, we investigated temporal contrast adaptation in neurons across Drosophila's visual motion circuitry. Several ON-pathway neurons showed strong adaptation to changes in contrast over time. One of these neurons, Mi1, showed almost complete adaptation on fast timescales, and experiments ruled out several potential mechanisms for its adaptive properties. When contrast adaptation reduced the gain in ON-pathway cells, it was accompanied by decreased motion responses in downstream direction-selective cells. Simulations show that contrast adaptation can substantially improve motion estimates in natural scenes. The benefits are larger for ON-pathway adaptation, which helps explain the heterogeneous distribution of contrast adaptation in these circuits.

Motion-Induced Scotoma.

We investigated artificial scotomas created when a moving object instantaneously crossed a gap, jumping ahead and continuing its otherwise smooth motion. Gaps of up to 5.1 degrees of visual angle, presented at 18° eccentricity, either closed completely or appeared much shorter than when the same gap was crossed by two-point apparent motion, or crossed more slowly, mimicking occlusion. Prolonged exposure to motion trajectories with a gap in most cases led to further shrinking of the gap. The same gap-shrinking effect has previously been observed in touch. In both sensory modalities, it implicates facilitation among codirectional local motion detectors and motion neurons with receptive fields larger than the gap. Unlike stimuli that simply deprive a receptor surface of input, suggesting it is insentient, our motion pattern skips a section in a manner that suggests a portion of the receptor surface has been excised, and the remaining portions stitched back together. This makes it a potentially useful tool in the experimental study of plasticity in sensory maps.

A Robust and Efficient Video Anti-Shaking Algorithm for Low-End Smartphone Platforms

Feature phones or low-end smart phones are an important part of our daily life. While lacking the powerful embedded camera and processing capability of the high-end smart phone, the feature phones with the low-end camera and processing capability are still widely used owing to its lower cost. Consequently, image and video processing algorithms are needed for many intelligent applications on the low-end smart phones. In this paper, we propose a novel robust and efficient video anti-shaking algorithm for the low-end smartphone platform. It provides an integrated application for the low-end smartphone platform, which includes a novel global motion estimation method, and a rapid and efficient local motion detection method. The overall framework is proposed considering the limited camera perception quality and limited mobile processing capability of the feature phone. The proposed algorithm can not only obtain a good visual effect but also be implemented in real-time for the low-end mobile platforms with very limited computational resource. Experimental results demonstrate that the proposed method can achieve a similar video stabilization effect to the Optical Image Stabilization hardware in high-end smartphones.

Full reconstruction of large lobula plate tangential cells in Drosophila from a 3D EM dataset.

With the advent of neurogenetic methods, the neural basis of behavior is presently being analyzed in more and more detail. This is particularly true for visually driven behavior of Drosophila melanogaster where cell-specific driver lines exist that, depending on the combination with appropriate effector genes, allow for targeted recording, silencing and optogenetic stimulation of individual cell-types. Together with detailed connectomic data of large parts of the fly optic lobe, this has recently led to much progress in our understanding of the neural circuits underlying local motion detection. However, how such local information is combined by optic flow sensitive large-field neurons is still incompletely understood. Here, we aim to fill this gap by a dense reconstruction of lobula plate tangential cells of the fly lobula plate. These neurons collect input from many hundreds of local motion-sensing T4/T5 neurons and connect them to descending neurons or central brain areas. We confirm all basic features of HS and VS cells as published previously from light microscopy. In addition, we identified the dorsal and the ventral centrifugal horizontal, dCH and vCH cell, as well as three VSlike cells, including their distinct dendritic and axonal projection area.

Optic tectal superficial interneurons detect motion in larval zebrafish

ABSTRACTDetection of moving objects is an essential skill for animals to hunt prey, recognize conspecifics and avoid predators. The zebrafish, as a vertebrate model, primarily uses its elaborate visual system to distinguish moving objects against background scenes. The optic tectum (OT) receives and integrates inputs from various types of retinal ganglion cells (RGCs), including direction-selective (DS) RGCs and size-selective RGCs, and is required for both prey capture and predator avoidance. However, it remains largely unknown how motion information is processed within the OT. Here we performed in vivo whole-cell recording and calcium imaging to investigate the role of superficial interneurons (SINs), a specific type of optic tectal neurons, in motion detection of larval zebrafish. SINs mainly receive excitatory synaptic inputs, exhibit transient ON- or OFF-type of responses evoked by light flashes, and possess a large receptive field (RF). One fifth of SINs are DS and classified into two subsets with separate preferred directions. Furthermore, SINs show size-dependent responses to moving dots. They are efficiently activated by moving objects but not static ones, capable of showing sustained responses to moving objects and having less visual adaptation than periventricular neurons (PVNs), the principal tectal cells. Behaviorally, ablation of SINs impairs prey capture, which requires local motion detection, but not global looming-evoked escape. Finally, starvation enhances the gain of SINs’ motion responses while maintaining their size tuning and DS. These results indicate that SINs serve as a motion detector for sensing and localizing sized moving objects in the visual field.

Sequential Nonlinear Filtering of Local Motion Cues by Global Motion Circuits

Many animals guide their movements using optic flow, the displacement of stationary objects across the retina caused by self-motion. How do animals selectively synthesize a global motion pattern from its local motion components? To what extent does this feature selectivity rely on circuit mechanisms versus dendritic processing? Here we used invivo calcium imaging to identify pre- and postsynaptic mechanisms for processing local motion signals in global motion detection circuits in Drosophila. Lobula plate tangential cells (LPTCs) detect global motion by pooling input from local motion detectors, T4/T5 neurons. We show that T4/T5 neurons suppress responses to adjacent local motion signals whereas LPTC dendrites selectively amplify spatiotemporal sequences of local motion signals consistent with preferred global patterns. We propose that sequential nonlinear suppression and amplification operations allow optic flow circuitry to simultaneously prevent saturating responses to local signals while creating selectivity for global motion patterns critical to behavior.

Robust real-time extraction of respiratory signals from PET list-mode data

Respiratory motion, which typically cannot simply be suspended during PET image acquisition, affects lesions’ detection and quantitative accuracy inside or in close vicinity to the lungs. Some motion compensation techniques address this issue via pre-sorting (‘binning’) of the acquired PET data into a set of temporal gates, where each gate is assumed to be minimally affected by respiratory motion. Tracking respiratory motion is typically realized using dedicated hardware (e.g. using respiratory belts and digital cameras). Extracting respiratory signals directly from the acquired PET data simplifies the clinical workflow as it avoids handling additional signal measurement equipment. We introduce a new data-driven method ‘combined local motion detection’ (CLMD). It uses the time-of-flight (TOF) information provided by state-of-the-art PET scanners in order to enable real-time respiratory signal extraction without additional hardware resources.CLMD applies center-of-mass detection in overlapping regions based on simple back-positioned TOF event sets acquired in short time frames. Following a signal filtering and quality-based pre-selection step, the remaining extracted individual position information over time is then combined to generate a global respiratory signal. The method is evaluated using seven measured FDG studies from single and multiple scan positions of the thorax region, and it is compared to other software-based methods regarding quantitative accuracy and statistical noise stability.Correlation coefficients around 90% between the reference and the extracted signal have been found for those PET scans where motion affected features such as tumors or hot regions were present in the PET field-of-view. For PET scans with a quarter of typically applied radiotracer doses, the CLMD method still provides similar high correlation coefficients which indicates its robustness to noise. Each CLMD processing needed less than 0.4 s in total on a standard multi-core CPU and thus provides a robust and accurate approach enabling real-time processing capabilities using standard PC hardware.

A UHDTV2 Full-Resolution 60/120-Hz Multiformat Portable Camera System

Japan Broadcasting Corp. (NHK) is accelerating the development of full-featured ultrahigh-definition (UHD) broadcasting equipment ( $7680\times4320$ , RGB 4:4:4, 12-bit, 120-Hz, high dynamic range and wide color gamut). We have developed a 60/120-Hz multiformat portable single-chip camera system using a 133-Mp CMOS image sensor. The weight of the camera head is less than one-seventh that of conventional three-chip camera systems and the size of the camera control unit (CCU) is reduced to 3RU. It can use both commercial 35-mm full-frame lenses and Super 35-mm lenses by using lens adapters. It employs a compact 100 Gbit/s optical transceiver to transmit signals between the camera head and the CCU. In 60-Hz operation, all the lines of the sensor are read progressively and a full-resolution UHDTV2 image is captured. In 120-Hz operation, even and odd lines in the video domain are read alternatively every 1/120 s by a 2:1 interline scan, and line interpolation with local motion detection is performed to configure the image. Modulation transfer function in the vertical direction without motion was 40% at 3200 TV lines, which was the same as that in 60-Hz operation. The signal-to-noise ratio of the 120-Hz interline video was 58 dB, which can be improved by 1 dB in areas without motion.

Local motion adaptation enhances the representation of spatial structure at EMD arrays.

Neuronal representation and extraction of spatial information are essential for behavioral control. For flying insects, a plausible way to gain spatial information is to exploit distance-dependent optic flow that is generated during translational self-motion. Optic flow is computed by arrays of local motion detectors retinotopically arranged in the second neuropile layer of the insect visual system. These motion detectors have adaptive response characteristics, i.e. their responses to motion with a constant or only slowly changing velocity decrease, while their sensitivity to rapid velocity changes is maintained or even increases. We analyzed by a modeling approach how motion adaptation affects signal representation at the output of arrays of motion detectors during simulated flight in artificial and natural 3D environments. We focused on translational flight, because spatial information is only contained in the optic flow induced by translational locomotion. Indeed, flies, bees and other insects segregate their flight into relatively long intersaccadic translational flight sections interspersed with brief and rapid saccadic turns, presumably to maximize periods of translation (80% of the flight). With a novel adaptive model of the insect visual motion pathway we could show that the motion detector responses to background structures of cluttered environments are largely attenuated as a consequence of motion adaptation, while responses to foreground objects stay constant or even increase. This conclusion even holds under the dynamic flight conditions of insects.

Neural mechanisms underlying sensitivity to reverse-phi motion in the fly.

Optical illusions provide powerful tools for mapping the algorithms and circuits that underlie visual processing, revealing structure through atypical function. Of particular note in the study of motion detection has been the reverse-phi illusion. When contrast reversals accompany discrete movement, detected direction tends to invert. This occurs across a wide range of organisms, spanning humans and invertebrates. Here, we map an algorithmic account of the phenomenon onto neural circuitry in the fruit fly Drosophila melanogaster. Through targeted silencing experiments in tethered walking flies as well as electrophysiology and calcium imaging, we demonstrate that ON- or OFF-selective local motion detector cells T4 and T5 are sensitive to certain interactions between ON and OFF. A biologically plausible detector model accounts for subtle features of this particular form of illusory motion reversal, like the re-inversion of turning responses occurring at extreme stimulus velocities. In light of comparable circuit architecture in the mammalian retina, we suggest that similar mechanisms may apply even to human psychophysics.

A Motion Detection Algorithm Using Local Phase Information.

Previous research demonstrated that global phase alone can be used to faithfully represent visual scenes. Here we provide a reconstruction algorithm by using only local phase information. We also demonstrate that local phase alone can be effectively used to detect local motion. The local phase-based motion detector is akin to models employed to detect motion in biological vision, for example, the Reichardt detector. The local phase-based motion detection algorithm introduced here consists of two building blocks. The first building block measures/evaluates the temporal change of the local phase. The temporal derivative of the local phase is shown to exhibit the structure of a second order Volterra kernel with two normalized inputs. We provide an efficient, FFT-based algorithm for implementing the change of the local phase. The second processing building block implements the detector; it compares the maximum of the Radon transform of the local phase derivative with a chosen threshold. We demonstrate examples of applying the local phase-based motion detection algorithm on several video sequences. We also show how the locally detected motion can be used for segmenting moving objects in video scenes and compare our local phase-based algorithm to segmentation achieved with a widely used optic flow algorithm.

Direction-selective units in goldfish retina and tectum opticum- review and new aspects.

The output units of fish retina, i.e., the retinal ganglion cells (detectors), send highly processed information to the primary visual centers of the brain, settled in the midbrain formation tectum opticum (TO). Axons of different fish motion detectors terminate in different tectal levels. In the superficial layer of TO, axons of direction-selective ganglion cells (DS GCs) are terminated. Single unit responses of the DS GCs were recorded in intact fish from their axon terminals in TO. Goldfish DS GCs projecting to TO were shown to comprise six physiological types according to their selectivity to sign of stimulus contrast (ON and OFF units) and their preferred directions: three directions separated by 120[Formula: see text]. These units, characterized by relatively small receptive fields and remarkable spatial resolution should be classified as local motion detectors. In addition to the retinal DS GCs, other kinds of DS units were extracellularly recorded in the superficial and deep sublaminae of tectum. Some features of their responses suggested that they originated from tectal neurons (TNs). Contrary to DS GCs which are characterized by small RFs and use separate ON and OFF channels, DS TNs have extra-large RFs and ON-OFF type responses. DS TNs were shown to select four preferred directions. Three of them are compatible with those already selected on the retinal level. Complementary to them, the fourth DS TN type with rostro-caudal preference (lacking in the retina) has been revealed. Possible functional interrelations between DS GCs and DS TNs are discussed.

A Class of Visual Neurons with Wide-Field Properties Is Required for Local Motion Detection

Visual motion cues are used by many animals to guide navigation across a wide range of environments. Long-standing theoretical models have made predictions about the computations that compare light signals across space and time to detect motion. Using connectomic and physiological approaches, candidate circuits that can implement various algorithmic steps have been proposed in the Drosophila visual system. These pathways connect photoreceptors, via interneurons in the lamina and the medulla, to direction-selective cells in the lobula and lobula plate. However, the functional architecture of these circuits remains incompletely understood. Here, we use a forward genetic approach to identify the medulla neuron Tm9 as critical for motion-evoked behavioral responses. Using in vivo calcium imaging combined with genetic silencing, we place Tm9 within motion-detecting circuitry. Tm9 receives functional inputs from the lamina neurons L3 and, unexpectedly, L1 and passes information onto the direction-selective T5 neuron. Whereas the morphology of Tm9 suggested that this cell would inform circuits about local points in space, we found that the Tm9 spatial receptive field is large. Thus, this circuit informs elementary motion detectors about a wide region of the visual scene. In addition, Tm9 exhibits sustained responses that provide a tonic signal about incoming light patterns. Silencing Tm9 dramatically reduces the response amplitude of T5 neurons under a broad range of different motion conditions. Thus, our data demonstrate that sustained and wide-field signals are essential for elementary motion processing.

Differences in primary visual cortex predict performance in local motion detection in deaf and hearing adults

Differences in primary visual cortex predict performance in local motion detection in deaf and hearing adults

Local motion detectors are required for the computation of expansion flow-fields.

ABSTRACTAvoidance of predators or impending collisions is important for survival. Approaching objects can be mimicked by expanding flow-fields. Tethered flying fruit flies, when confronted with an expansion flow-field, reliably turn away from the pole of expansion when presented laterally, or perform a landing response when presented frontally. Here, we show that the response to an expansion flow-field is independent of the overall luminance change and edge acceleration. As we demonstrate by blocking local motion-sensing neurons T4 and T5, the response depends crucially on the neural computation of appropriately aligned local motion vectors, using the same hardware that also controls the optomotor response to rotational flow-fields.

Noise-robust recognition of wide-field motion direction and the underlying neural mechanisms in Drosophila melanogaster.

Appropriate and robust behavioral control in a noisy environment is important for the survival of most organisms. Understanding such robust behavioral control has been an attractive subject in neuroscience research. Here, we investigated the processing of wide-field motion with random dot noise at both the behavioral and neuronal level in Drosophila melanogaster. We measured the head yaw optomotor response (OMR) and the activity of motion-sensitive neurons, horizontal system (HS) cells, with in vivo whole-cell patch clamp recordings at various levels of noise intensity. We found that flies had a robust sensation of motion direction under noisy conditions, while membrane potential changes of HS cells were not correlated with behavioral responses. By applying signal classification theory to the distributions of HS cell responses, however, we found that motion direction under noise can be clearly discriminated by HS cells, and that this discrimination performance was quantitatively similar to that of OMR. Furthermore, we successfully reproduced HS cell activity in response to noisy motion stimuli with a local motion detector model including a spatial filter and threshold function. This study provides evidence for the physiological basis of noise-robust behavior in a tiny insect brain.