Optic Flow Research Articles

Long-term liver lesion tracking in contrast-enhanced ultrasound videos via a siamese network with temporal motion attention.

Propose: Contrast-enhanced ultrasound has shown great promises for diagnosis and monitoring in a wide range of clinical conditions. Meanwhile, to obtain accurate and effective location of lesion in contrast-enhanced ultrasound videos is the basis for subsequent diagnosis and qualitative treatment, which is a challenging task nowadays. Methods: We propose to upgrade a siamese architecture-based neural network for robust and accurate landmark tracking in contrast-enhanced ultrasound videos. Due to few researches on it, the general inherent assumptions of the constant position model and the missing motion model remain unaddressed limitations. In our proposed model, we overcome these limitations by introducing two modules into the original architecture. We use a temporal motion attention based on Lucas Kanade optic flow and Karman filter to model the regular movement and better instruct location prediction. Moreover, we design a pipeline of template update to ensure timely adaptation to feature changes. Results: Eventually, the whole framework was performed on our collected datasets. It has achieved the average mean IoU values of 86.43% on 33 labeled videos with a total of 37,549 frames. In terms of tracking stability, our model has smaller TE of 19.2 pixels and RMSE of 27.6 with the FPS of 8.36 ± 3.23 compared to other classical tracking models. Conclusion: We designed and implemented a pipeline for tracking focal areas in contrast-enhanced ultrasound videos, which takes the siamese network as the backbone and uses optical flow and Kalman filter algorithm to provide position prior information. It turns out that these two additional modules are helpful for the analysis of CEUS videos. We hope that our work can provide an idea for the analysis of CEUS videos.

Case report of the evidence of a spontaneous Reverse Pulfrich effect in monovision after cataract surgery

BackgroundCataracts affect the optics of the eye in terms of absorption, blur, and scattering. When cataracts are unilateral, they cause differences between the eyes that can produce visual discomfort and harm binocular vision. These interocular differences can also induce differences in the processing speed of the eyes that may cause a spontaneous Pulfrich effect, a visual illusion provoking important depth misperceptions. Interocular differences in light level, like those present in unilateral cataracts, can cause the Classic Pulfrich effect, and interocular differences in blur, like those present in monovision, a common correction for presbyopia, can cause the Reverse Pulfrich effect. The visual system may be able to adapt, or not, to the new optical condition, depending on the degree of the cataract and the magnitude of the monovision correction.Case presentationHere, we report a unique case of a 45-year-old patient that underwent unilateral cataract surgery resulting in a monovision correction of 2.5 diopters (D): left eye emmetropic after the surgery compensated with a monofocal intraocular lens and right eye myopic with a spherical equivalent of -2.50 D. This patient suffered severe symptoms in binocular vision, which can be explained by a spontaneous Pulfrich effect (a delay measured of 4.82 ms, that could be eliminated with a 0.19 optical density filter). After removing the monovision with clear lens extraction in the second eye, symptoms disappeared. We demonstrate that, at least in this patient, both Classic and Reverse Pulfrich effects coexist after unilateral cataract surgery and that can be readapted by reverting the interocular differences. Besides, we report that the adaptation/readaptation process to the Reverse Pulfrich effect happens in a timeframe of weeks, as opposed to the Classic Pulfrich effect, known to have timeframes of days. Additionally, we used the illusion measured in the laboratory to quantify the relevance of the spontaneous Pulfrich effect in different visual scenarios and tasks, using geometrical models and optic flow algorithms.ConclusionsMeasuring the different versions of the Pulfrich effect might help to understand the visual discomfort reported by many patients after cataract surgery or with monovision and could guide compensation or intervention strategies.

Spatial updating in virtual reality for reproducing object locations in vista space-Boundaries, landmarks, and idiothetic cues.

Keeping track of locations across self-motion is possible by continuously updating spatial representations or by encoding and later instantaneously retrieving spatial representations. In virtual reality (VR), sensory cues to self-motion used in continuous updating are typically reduced. In passive translation compared to real walking in VR, optic flow is available but body-based (idiothetic) cues are missing. With both kinds of translation, boundaries and landmarks as static visual cues can be used for instantaneous updating. In two experiments, we let participants encode two target locations, one of which had to be reproduced by pointing after forward translation in immersive VR (HMD). We increased sensory cues to self-motion in comparison to passive translation either by strengthening optic flow or by real walking. Furthermore, we varied static visual cues in the form of boundaries and landmarks inside boundaries. Increased optic flow and real walking did not reliably increase performance suggesting that optic flow even in a sparse environment was sufficient for continuous updating or that merely instantaneous updating took place. Boundaries and landmarks, however, did support performance as quantified by decreased bias and increased precision, particularly if they were close to or even enclosed target locations. Thus, enriched spatial context is a viable method to support spatial updating in VR and synthetic environments (teleoperation). Spatial context does not only provide a static visual reference in offline updating and continuous allocentric self-location updating but also, according to recent neuroscientific evidence on egocentric bearing cells, contributes to continuous egocentric location updating as well.

Distinguishing Glial Tissue from Optic Disc in Bergmeister's Papilla Using OCT Angiography

A 55-year-old asymptomatic woman was found to have bilateral abnormalities of the optic nerve during an ophthalmic examination. Ocular examination revealed visual acuity of 20/20 in both eyes with normal anterior segment. Fundus examination showed grayish-white glial tissue covering the optic nerve (A) as confirmed by infrared image (B) from OCT. Cross-sectional OCT image (C) exhibited hyperreflective membrane overlying the optic disc, and adhesions to adjacent optic papilla structure (white arrow). The corresponding OCT angiography B-scan with flow overlay (D) showed hyperreflective tissue without any flow signal, contrasting with the optic disc's normal blood flow signal (Magnified version of Figure A-D is available online at www.aaojournal.org).

Wide-spread brain activation and reduced CSF flow during avian REM sleep

Mammalian sleep has been implicated in maintaining a healthy extracellular environment in the brain. During wakefulness, neuronal activity leads to the accumulation of toxic proteins, which the glymphatic system is thought to clear by flushing cerebral spinal fluid (CSF) through the brain. In mice, this process occurs during non-rapid eye movement (NREM) sleep. In humans, ventricular CSF flow has also been shown to increase during NREM sleep, as visualized using functional magnetic resonance imaging (fMRI). The link between sleep and CSF flow has not been studied in birds before. Using fMRI of naturally sleeping pigeons, we show that REM sleep, a paradoxical state with wake-like brain activity, is accompanied by the activation of brain regions involved in processing visual information, including optic flow during flight. We further demonstrate that ventricular CSF flow increases during NREM sleep, relative to wakefulness, but drops sharply during REM sleep. Consequently, functions linked to brain activation during REM sleep might come at the expense of waste clearance during NREM sleep.

71‐1: Data‐Driven and Optics‐Inspired Decomposition of Global Pupil Swim in VR/AR for an Improved Perception Model of Motion Discomfort

Users can observe dynamic distortion (or global pupil swim) during the usage of VR/AR headsets. Our earlier study correlated pupil swim to selected optic flow patterns corresponding to motion and mathematically modeled discomfort. This study decomposes global pupil swim as a linear sum of orthogonal basis patterns based on Zernike polynomials for improved applicability and for an improved perception model.

Modeling the development of cortical responses in primate dorsal ("where") pathway to optic flow using hierarchical neural field models.

Although there is a plethora of modeling literature dedicated to the object recognition processes of the ventral ("what") pathway of primate visual systems, modeling studies on the motion-sensitive regions like the Medial superior temporal area (MST) of the dorsal ("where") pathway are relatively scarce. Neurons in the MST area of the macaque monkey respond selectively to different types of optic flow sequences such as radial and rotational flows. We present three models that are designed to simulate the computation of optic flow performed by the MST neurons. Model-1 and model-2 each composed of three stages: Direction Selective Mosaic Network (DSMN), Cell Plane Network (CPNW) or the Hebbian Network (HBNW), and the Optic flow network (OF). The three stages roughly correspond to V1-MT-MST areas, respectively, in the primate motion pathway. Both these models are trained stage by stage using a biologically plausible variation of Hebbian rule. The simulation results show that, neurons in model-1 and model-2 (that are trained on translational, radial, and rotational sequences) develop responses that could account for MSTd cell properties found neurobiologically. On the other hand, model-3 consists of the Velocity Selective Mosaic Network (VSMN) followed by a convolutional neural network (CNN) which is trained on radial and rotational sequences using a supervised backpropagation algorithm. The quantitative comparison of response similarity matrices (RSMs), made out of convolution layer and last hidden layer responses, show that model-3 neuron responses are consistent with the idea of functional hierarchy in the macaque motion pathway. These results also suggest that the deep learning models could offer a computationally elegant and biologically plausible solution to simulate the development of cortical responses of the primate motion pathway.

Retinal waves align the concentric orientation map in mouse superior colliculus to the center of vision

Neurons in the mouse superior colliculus (SC) are arranged in a concentric orientation map, which is aligned to the center of vision and the optic flow experienced by the mouse. The origin of this map remains unclear. Here, we propose that spontaneous retinal waves during development provide a scaffold to establish the concentric orientation map within the SC and its alignment to the optic flow. We test this hypothesis by modeling the orientation-tuned SC neurons that receive ON/OFF retinal inputs. Our model suggests that the propagation direction bias of stage III retinal waves, together with OFF-delayed responses, shapes the spatial organization of the orientation map. The OFF delay establishes orientation-tuned neurons by segregating their ON/OFF receptive subfields, the wave-like activities form the concentric pattern, and the direction biases align the map to the center of vision. Together, retinal waves may play an instructive role in establishing functional properties of single SC neurons and their spatial organization within maps.

Integration of optic flow into the sky compass network in the brain of the desert locust.

Flexible orientation through any environment requires a sense of current relative heading that is updated based on self-motion. Global external cues originating from the sky or the earth's magnetic field and local cues provide a reference frame for the sense of direction. Locally, optic flow may inform about turning maneuvers, travel speed and covered distance. The central complex in the insect brain is associated with orientation behavior and largely acts as a navigation center. Visual information from global celestial cues and local landmarks are integrated in the central complex to form an internal representation of current heading. However, it is less clear how optic flow is integrated into the central-complex network. We recorded intracellularly from neurons in the locust central complex while presenting lateral grating patterns that simulated translational and rotational motion to identify these sites of integration. Certain types of central-complex neurons were sensitive to optic-flow stimulation independent of the type and direction of simulated motion. Columnar neurons innervating the noduli, paired central-complex substructures, were tuned to the direction of simulated horizontal turns. Modeling the connectivity of these neurons with a system of proposed compass neurons can account for rotation-direction specific shifts in the activity profile in the central complex corresponding to turn direction. Our model is similar but not identical to the mechanisms proposed for angular velocity integration in the navigation compass of the fly Drosophila.

Visual perception of travel distance for self-motion through crowds.

Humans can use visual motion to estimate the distance they have traveled. In static environments, optic flow generated by self-motion provides a pattern of expanding motion that is used for the estimation of travel distance. When the environment is populated by other people, their biological motion destroys the one-to-on correspondence between optic flow and travel distance. We investigated how observers estimate travel distance in a crowded environment. In three conditions, we simulated self-motion through a crowd of standing, approaching, or leading point-light walkers. For a standing crowd, optic flow is a veridical signal for distance perception. For an approaching crowd, the visual motion is the sum of the self-motion-induced optic flow and the optic flow produced by the approaching walkers. If only optic flow were to be used, travel distance estimates would be too high because of the approaching direction of the crowd toward the observer. If, on the other hand, cues from biological motion could be used to estimate the speed of the crowd, then the excessive optic from the approaching crowd flow might be compensated. In the leading crowd condition, in which walkers of the crowd keep their distance from the observer as they walk along with the observer, no optic flow is produced. In this condition, travel distance estimation would have to rely solely on biological motion information. We found that distance estimation was quite similar across these three conditions. This suggests that biological motion information can be used (a) to compensate for excessive optic flow in the approaching crowd condition and (b) to generate distance information in the leading crowd condition.

Heading perception from optic flow occurs at both perceptual representation and working memory stages with EEG evidence

Psychophysical studies have demonstrated that heading perception from optic flow occurs in perceptual and post-perceptual stages. The post-perception stage is a complex concept, containing working memory. The current study examined whether working memory was involved in heading perception from optic flow by asking participants to conduct a heading perception task and recording their scalp EEG. On each trial, an optic flow display was presented, followed by a blank display. Participants were then asked to report their perceived heading. We know that participants would tend to automatically forget previous headings when they learned that previously presented headings were unrelated to the current heading perception to save cognitive resources. As a result, we could not decode previous headings from the EEG data of current trials. More importantly, if we successfully decoded previous headings when the blank display (optic flow) was presented, then working memory (perceptual representation stage) was involved in heading perception. Our results showed that the decoding accuracy was significantly higher than the chance level when the optic flow and blank displays were presented. Therefore, the current study provided electrophysiological evidence that heading perception from optic flow occurred in the perceptual representation and working memory stages, against the previous perceptual claim.

Quantitative Measurements of Macular and Optic Nerve Head Blood Flow Parameters Following Cataract Surgery in Eye Department, Red Cross Tertiary General Hospital, Athens, Greece

Objective: To evaluate the effect of uncomplicated cataract surgery on macular and optic nerve head blood flow with Optical Coherence Tomography Angiography (OCT-A). Retinal vascular changes and macular vessel remodeling following cataract surgery is an area of interest, as the effect, cause and duration of such changes remain debatable.
 Setting: Eye Department, Red Cross Tertiary General Hospital, Athens, Greece.
 Methods: In this study, we recruited cataract surgery candidates who had regular post operative visits and underwent longitudinal OCT-A examinations before and after surgery.
 Results: Macular vessel density increased in the post operative visits both in the superficial (SCP) and deep capillary network (DCP) (P<0.001) and showed a persistent effect over 3 months follow-up. Macular thickness increased over the first postoperative month, but returned to preoperative values at 3 months. Foveal avascular zone (FAZ) measurements and vessel density at radial peripapillary capillaries (RPC) were unaffected.
 Conclusions: OCT-A enables a quantitative analysis of blood flow by regular patient monitoring. In this study, we found that after cataract surgery, macular blood flow increased significantly. The reason for this could be attributed to post operative inflammatory process or vascular remodeling due to change in retinal metabolic demands.

Variable Angular Rate Measurement for a Spacecraft Based on the Rolling Shutter Mode of a Star Tracker

Angular rate is a piece of useful information for the attitude control of a spacecraft. The star tracker as a space optical sensor can be used to measure the angular rate of a spacecraft. In this paper, a novel approach is proposed to improve the measurement accuracy of the angular rate during spacecraft rotation. The electronic rolling shutter (RS) imaging mode of the complementary metal-oxide semiconductor (CMOS) image sensor in a star tracker is applied to obtain much higher sampling frequency for reducing the change of the angular rate between the sampling interval. The optic flow vector on the imaging plane is approximated within the second order using three successive star images to reflect the nonlinear effect from the variable angular rate. The experiment is performed to demonstrate the advantage of the new approach for variable angular rate measurement.

Parallel motion vision pathways in the brain of a tropical bee

Spatial orientation is a prerequisite for most behaviors. In insects, the underlying neural computations take place in the central complex (CX), the brain’s navigational center. In this region different streams of sensory information converge to enable context-dependent navigational decisions. Accordingly, a variety of CX input neurons deliver information about different navigation-relevant cues. In bees, direction encoding polarized light signals converge with translational optic flow signals that are suited to encode the flight speed of the animals. The continuous integration of speed and directions in the CX can be used to generate a vector memory of the bee’s current position in space in relation to its nest, i.e., perform path integration. This process depends on specific, complex features of the optic flow encoding CX input neurons, but it is unknown how this information is derived from the visual periphery. Here, we thus aimed at gaining insight into how simple motion signals are reshaped upstream of the speed encoding CX input neurons to generate their complex features. Using electrophysiology and anatomical analyses of the halictic bees Megalopta genalis and Megalopta centralis, we identified a wide range of motion-sensitive neurons connecting the optic lobes with the central brain. While most neurons formed pathways with characteristics incompatible with CX speed neurons, we showed that one group of lobula projection neurons possess some physiological and anatomical features required to generate the visual responses of CX optic-flow encoding neurons. However, as these neurons cannot explain all features of CX speed cells, local interneurons of the central brain or alternative input cells from the optic lobe are additionally required to construct inputs with sufficient complexity to deliver speed signals suited for path integration in bees.

A virtual reality study investigating the train illusion.

The feeling of self-movement that occurs in the absence of physical motion is often referred to as vection, which is commonly exemplified using the train illusion analogy (TIA). Limited research exists on whether the TIA accurately exemplifies the experience of vection in virtual environments (VEs). Few studies complemented their vection research with participants' qualitative feedback or by recording physiological responses, and most studies used stimuli that contextually differed from the TIA. We investigated whether vection is experienced differently in a VE replicating the TIA compared to a VE depicting optic flow by recording subjective and physiological responses. Additionally, we explored participants' experience through an open question survey. We expected the TIA environment to induce enhanced vection compared to the optic flow environment. Twenty-nine participants were visually and audibly immersed in VEs that either depicted optic flow or replicated the TIA. Results showed optic flow elicited more compelling vection than the TIA environment and no consistent physiological correlates to vection were identified. The post-experiment survey revealed discrepancies between participants' quantitative and qualitative feedback. Although the dynamic content may outweigh the ecological relevance of the stimuli, it was concluded that more qualitative research is needed to understand participants' vection experience in VEs.

Dynamical latent state computation in the male macaque posterior parietal cortex

Success in many real-world tasks depends on our ability to dynamically track hidden states of the world. We hypothesized that neural populations estimate these states by processing sensory history through recurrent interactions which reflect the internal model of the world. To test this, we recorded brain activity in posterior parietal cortex (PPC) of monkeys navigating by optic flow to a hidden target location within a virtual environment, without explicit position cues. In addition to sequential neural dynamics and strong interneuronal interactions, we found that the hidden state - monkey’s displacement from the goal - was encoded in single neurons, and could be dynamically decoded from population activity. The decoded estimates predicted navigation performance on individual trials. Task manipulations that perturbed the world model induced substantial changes in neural interactions, and modified the neural representation of the hidden state, while representations of sensory and motor variables remained stable. The findings were recapitulated by a task-optimized recurrent neural network model, suggesting that task demands shape the neural interactions in PPC, leading them to embody a world model that consolidates information and tracks task-relevant hidden states.

The effects of optic flow on postural stability: Influence of age and fall risk

BackgroundOptic flow provides dynamic information relating to body position and motion with respect to visual frames of reference. This study investigated the effects of optic flow stimuli presented in four directions on postural stability in young and older adults. MethodsTwenty-five young (20–40 years) and 51 older (≥65 years) people participated in this study, with the older group classified into low fall risk (n = 27), and high fall risk (n = 24) sub-groups. While standing in a dark room, participants viewed static scattered white dots for 30 s, followed by 30 s periods of optic flow consisting of white dots “moving” in one of four flow directions, randomised: radial expansion and contraction, circular anti-clockwise and clockwise. Centre of pressure (CoP) position, postural sway in anteroposterior (AP) and mediolateral (ML) axes, and muscle activity of tibialis anterior (TA), gastrocnemius medialis (GM) and tensor fascia latae (TFL) were recorded. ResultsAcross groups, the four optic flow stimuli induced increased AP sway and three of the four optic flow stimuli induced increased ML sway, with concomitant increases in muscle activity, indicating optic flow stimuli induced a generalised destabilising, rather than a direction-specific, effect. Only one optic flow condition (radial contraction) induced a change in average CoP position, and this may reflect the adoption of a protective stance position to avoid a backward fall. Optic flow destabilised postural control more in older people compared with younger people, and radial expansion stimuli destabilised ML postural control more in the older high fall risk group compared with the older low fall risk group. ConclusionOptic flow stimuli have a generalised destabilising effect on postural control across groups as shown by non-directional specific increases in postural sway and muscular activity. Optic flow stimuli have a greater impact on postural stability in older compared with younger adults and this is more pronounced in the ML plane for older people at increased risk of falls.

OCTA evaluation of posterior ocular blood flow in patients after COVID-19 infection without pneumonia

We aimed to demonstrate the changes in optic nerve and retinal microvascular structures with Optical Coherence Tomography Angiography (OCTA) in patients who had COVID-19 infection. Prospective study. Retina, choroid and optic nerve head microvascular flow and vascular densities of both groups were measured by OCTA. OCTA measurements of 122 right eyes of a total of 122 patients, including 72 patients in the COVID-19 group and 50 patients in the control group, were included in the study. The Deep Capillary Plexus (DCP) Flow Area (FA) in the COVID-19 group was 1.42±0.23mm2, in the control group was 1.50±0.15mm2, Choriocapillary Plexus FA was 1.89±0.04mm2 in the COVID-19 group, was 1.91±0.05mm2 in the control group, and a statistically significant difference was found between the two groups (respectively; P=0.03; P=0.02). DCP Whole Vascular density (VD) values were 56.76±4.16% in the COVID-19 group, 58.28±3.88% in the control group; difference between the values was statistically significant (P=0.04). There was no statistically significant difference between the two groups in terms of optic nerve head flow areas and other parameters examined according to quadrants. Results show that the retinal microcirculation in subjects with mild disease is affected. Even if the disease is mild, patients may need to be followed up for retinal changes that may develop in the future.

Does optic flow provide information about actions?

Optic flow, the pattern of light generated in the visual field by motion of objects and the observer's body, serves as information that underwrites perception of events, actions, and affordances. This visual pattern informs the observer about their own actions in relation to their surroundings, as well as those of others. This study explored the limits of action detection for others as well as the role of optic flow. First-person videos were created using camera recordings of the actor's perspective as they performed various movements (jumping jacks, jumping, squatting, sitting, etc.). In three experiments participants attempted to detect the action from first-person video footage using open ended responses (Experiment 1), forced-choice responses (Experiment 2), and a match-to-sample paradigm (Experiment 3). It was discovered that some actions are more difficult to detect than others. When the task was challenging (Experiment 1) athletes were more accurate, but this was not the case in Experiments 2 and 3. All actions were identified above chance level across viewpoints, suggesting that invariant information was detected and used to perform the task.

Social signal learning of the waggle dance in honey bees.

Honey bees use a complex form of spatial referential communication. Their "waggle dance" communicates the direction, distance, and quality of a resource to nestmates by encoding celestial cues, retinal optic flow, and relative food value into motion and sound within the nest. We show that correct waggle dancing requires social learning. Bees without the opportunity to follow any dances before they first danced produced significantly more disordered dances with larger waggle angle divergence errors and encoded distance incorrectly. The former deficit improved with experience, but distance encoding was set for life. The first dances of bees that could follow other dancers showed neither impairment. Social learning, therefore, shapes honey bee signaling, as it does communication in human infants, birds, and multiple other vertebrate species.