Visual Input Research Articles

Prediction by Young Autistic Children from Visual and Spoken Input.

Recent theoretical accounts suggest that differences in the processing of probabilistic events underlie the core and associated traits of autism spectrum disorder (ASD). These theories hypothesize that autistic individuals are differentially impacted by disruptions in probabilistic input relative to neurotypical peers. According to this view, autistic individuals assign disproportionate weight to prediction errors such that novel input is overweighted relative to the aggregation of prior input; this is referred to as 'hyperplasticity' of learning. Prediction among autistic individuals has primarily been examined in nonverbal, visual contexts with older children and adults. The present study examined 32 autistic and 32 cognitively-matched neurotypical (NT) children's ability to generate predictions and adjust to changes in predictive relationships in auditory stimuli using two eye gaze tasks. In both studies, children were trained and tested on an auditory-visual cue which predicted the location of a reward stimulus. In Experiment 1 the cue was non-linguistic (instrumental sound) whereas in Experiment 2 the cue was linguistically-relevant (speaker gender). In both experiments, the cue-reward contingency was switched after the first block of trials, and predictive behavior was evaluated across a second block of trials. Results: Analyses of children's looking behavior revealed similar performance in both groups on the non-linguistic task (Exp. 1). In the linguistically-relevant task (Exp. 2), predictive looking was less disrupted by the contingency switch for autistic children than NT children. Results suggest that autistic children may demonstrate hyperplastic learning in linguistically-relevant contexts, relative to NT peers.

The effect of juggling on the proprioceptive and attentional abilities among older women.

Age-related changes in attentional abilities can lead to a decline in body segment awareness in space. However, studies have reported that physical activity can improve proprioception among older adults, although proven activities with this potential are limited. Juggling is a promising activity for enhancing proprioception, as it requires high levels of attention and sensory precision. The first hypothesis posited that a juggling intervention would positively impact ipsilateral and contralateral elbow joint position matching without visual input. The second hypothesis suggested a correlation between cognitive abilities and joint position sense efficiency. A total of 20 older women (mean age: 69.95 ± 4.58) participated in a repeated-measures study using a Latin square design. Measurements were taken at three time points (baseline, post-juggling, and control). Ipsilateral and contralateral elbow joint position matchings without visual or verbal feedback of accuracy were used to assess proprioception. Attention and reaction time variables were measured using the Vienna Test System protocols. Although significant changes were observed between baseline and subsequent time points in joint position sense accuracy, no specific effect of juggling was detected. Low and medium correlations were found between decision time and the variability of choice reaction time with contralateral accuracy. For ipsilateral accuracy, a relationship was observed only with handedness. No correlations were found between attention test scores and joint position sense accuracy. The study did not demonstrate a significant effect of juggling on position-matching ability. However, cognitive abilities such as decision speed and the stability of choice reaction time may play a role in enhancing position-matching in older women. ClinicalTrials.gov, identifier NCT06108713.

Text-Guided Multi-Class Multi-Object Tracking for Fine-Grained Maritime Rescue

The rapid development of remote sensing technology has provided new sources of data for marine rescue and has made it possible to find and track survivors. Due to the requirement of tracking multiple survivors at the same time, multi-object tracking (MOT) has become the key subtask of marine rescue. However, there exists a significant gap between fine-grained objects in realistic marine rescue remote sensing data and the fine-grained object tracking capability of existing MOT technologies, which mainly focuses on coarse-grained object scenarios and fails to track fine-grained instances. Such a gap limits the practical application of MOT in realistic marine rescue remote sensing data, especially when rescue forces are limited. Given the promising fine-grained classification performance of recent text-guided methods, we delve into leveraging labels and attributes to narrow the gap between MOT and fine-grained maritime rescue. We propose a text-guided multi-class multi-object tracking (TG-MCMOT) method. To handle the problem raised by fine-grained classes, we design a multi-modal encoder by aligning external textual information with visual inputs. We use decoding information at different levels, simultaneously predicting the category, location, and identity embedding features of objects. Meanwhile, to improve the performance of small object detection, we also develop a data augmentation pipeline to generate pseudo-near-infrared images based on RGB images. Extensive experiments demonstrate that our TG-MCMOT not only performs well on typical metrics in the maritime rescue task (SeaDronesSee dataset), but it also effectively tracks open-set categories on the BURST dataset. Specifically, on the SeaDronesSee dataset, the Higher Order Tracking Accuracy (HOTA) reached a score of 58.8, and on the BURST test dataset, the HOTA score for the unknown class improved by 16.07 points.

Connectomic reconstruction predicts visual features used for navigation

Many animals use visual information to navigate1–4, but how such information is encoded and integrated by the navigation system remains incompletely understood. In Drosophila melanogaster, EPG neurons in the central complex compute the heading direction5 by integrating visual input from ER neurons6–12, which are part of the anterior visual pathway (AVP)10,13–16. Here we densely reconstruct all neurons in the AVP using electron-microscopy data17. The AVP comprises four neuropils, sequentially linked by three major classes of neurons: MeTu neurons10,14,15, which connect the medulla in the optic lobe to the small unit of the anterior optic tubercle (AOTUsu) in the central brain; TuBu neurons9,16, which connect the AOTUsu to the bulb neuropil; and ER neurons6–12, which connect the bulb to the EPG neurons. On the basis of morphologies, connectivity between neural classes and the locations of synapses, we identify distinct information channels that originate from four types of MeTu neurons, and we further divide these into ten subtypes according to the presynaptic connections in the medulla and the postsynaptic connections in the AOTUsu. Using the connectivity of the entire AVP and the dendritic fields of the MeTu neurons in the optic lobes, we infer potential visual features and the visual area from which any ER neuron receives input. We confirm some of these predictions physiologically. These results provide a strong foundation for understanding how distinct sensory features can be extracted and transformed across multiple processing stages to construct higher-order cognitive representations.

An Audio-Visual Speech Separation Model Inspired by Cortico-Thalamo-Cortical Circuits.

Audio-visual approaches involving visual inputs have laid the foundation for recent progress in speech separation. However, the optimization of the concurrent usage of auditory and visual inputs is still an active research area. Inspired by the cortico-thalamo-cortical circuit, in which the sensory processing mechanisms of different modalities modulate one another via the non-lemniscal sensory thalamus, we propose a novel cortico-thalamo-cortical neural network (CTCNet) for audio-visual speech separation (AVSS). First, the CTCNet learns hierarchical auditory and visual representations in a bottom-up manner in separate auditory and visual subnetworks, mimicking the functions of the auditory and visual cortical areas. Then, inspired by the large number of connections between cortical regions and the thalamus, the model fuses the auditory and visual information in a thalamic subnetwork through top-down connections. Finally, the model transmits this fused information back to the auditory and visual subnetworks, and the above process is repeated several times. The results of experiments on three speech separation benchmark datasets show that CTCNet remarkably outperforms existing AVSS methods with considerably fewer parameters. These results suggest that mimicking the anatomical connectome of the mammalian brain has great potential for advancing the development of deep neural networks.

Ensemble memory of a scene interacts with current perception regardless of attentional requirements.

How do we maintain a rich and stable perceptual experience across the entire visual scene, even when we are focusing on a subset of visual inputs? The current study explored this question by investigating whether the visual system processes summary statistics of multiple features regardless of task relevance, and how they interact with subsequent perception. To test the processing of multifeature summary statistics under different attentional requirements, we presented multiple Gabor patches with heterogeneous orientations/colors and asked participants to attend to a single feature dimension (Experiments 1 and 3) or a single item (Experiment 2) for the memory task. During the memory maintenance period (before memory response), we asked the participants to perform a discrimination task (Experiments 1 and 2) or a boundary localization task (Experiment 3) to test how the memory of the ensemble representation alters the subsequent perceptual experience. We found evidence for obligatory processing of scene summary statistics presented for the memory task, which interacted with the subsequent perceptual sensitivity. Specifically, not only summary statistics relevant but also those of task-irrelevant feature (Experiments 1 and 3) and outside the focus of attention (Experiment 2) were encoded and bidirectionally interacted with subsequent perception. These results suggest obligatory processing of summary statistics of a scene, which may allow rich and stable visual experience by integrating temporally adjacent visual inputs. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Lack of visual experience leads to severe distortions in the hand representation of the body model

This study investigates the impact of vision on the maintenance of hand representation in the implicit body model, particularly focusing on congenitally blind individuals. To address this, we performed a hand landmark localization task on blind individuals who lacked visual experience of their bodies and compared their performance to normally sighted and normally sighted but blindfolded participants. Through measurements of finger lengths, hand width, and shape index, we demonstrate that blind participants exhibit significantly greater distortions in their hand representation compared to sighted and blindfolded controls. Notably, blind individuals displayed a marked overestimation of hand width and an underestimation of finger lengths, particularly in digits D2, D3, and D4. Surprisingly, blind subjects with partial vision displayed more severe distortions than those with no residual vision. Furthermore, our findings reveal that late-blind participants exhibit similar levels of distortion as congenitally blind individuals, suggesting an extended period of susceptibility to the lack of visual input in shaping body representations. The Reverse Distortion (RD) hypothesis provides a plausible explanation for these distortions, suggesting that compensatory mechanisms occur within the body model to counteract the anisotropic cortical representations. Our results support this hypothesis: blind individuals have expanded cortical representations processing tactile information, so this could lead to more pronounced distortions in their hand representation of the body model. This underscores the importance of visual input in modulating body representations. Overall, our study highlights the malleability of body representations and the intricate interplay between sensory inputs and cortical processing in shaping the implicit body model.

Right visual field advantage in orientation discrimination is influenced by biased suppression

Visual input is not equally processed over space. In recent years, a right visual field advantage during free walking and standing in orientation discrimination and contrast detection task was reported. The current study investigated the underlying mechanism of the previously reported right visual field advantage. It particularly tested if the advantage is driven by a stronger suppression of distracting input from the left visual field or improved processing of targets from the right visual field. Combing behavioural and electrophysiological measurements in a mobile EEG and augmented reality setup, human participants (n = 30) in a standing and a walking condition performed a line orientation discrimination task with stimulus eccentricity and distractor status being manipulated. The right visual field advantage, as demonstrated in accuracy and reaction time, was influenced by the distractor status. Specifically, the right visual field advantage was only observed when the target had an incongruent line orientation with the distractor. Neural data further showed that the right visual field advantage was paralleled by a strong modulation of neural activity in the right hemisphere (i.e. contralateral to the distractor). A significant positive correlation between this right hemispheric event related potential (ERP) and behavioural measures (accuracy and reaction time) was found exclusively for trials in which a target was presented on the right and an incongruent distractor was presented on the left. The right hemispheric ERP component further predicted the strength of the right visual field advantage. Notably, the lateralised brain activity and the right visual field advantage were both independent of stimulus eccentricity and the movement state of participants. Overall, our findings suggest an important role of spatially biased suppression of left distracting input in the right visual field advantage as found in orientation discrimination.

Computational model of layer 2/3 in mouse primary visual cortex explains observed visuomotor mismatch response.

Activity in layer 2/3 of the mouse primary visual cortex has been shown to depend both on visual input and the mouse's locomotion. Moreover, this activity is altered by a mismatch between the observed visual flow and the predicted visual flow from locomotion. Here, I present a simple computational model that explains previously reported recordings from layer 2/3 neurons in mice. In my model, layer 2/3 encodes the velocity difference between the estimate from visual flow and the prediction from locomotion using a neural population code. Moreover, I describe a hypothesized mechanism for how the brain may carry out computations of variables encoded in population codes. This mechanism may point to a general principle for computing any mathematical function in the brain.

Neuro-cognitive effects of degraded visibility on illusory body ownership

Based on visuo-tactile stimulation, the rubber hand illusion induces a sense of ownership for a dummy hand. Manipulating the visibility of the dummy hand during the stimulation influences cognitive aspects of the illusion, suggesting that the related brain activity may be influenced too. To test this, we analyzed brain activity (fMRI), subjective ratings, and skin conductance from 45 neurotypical participants undergoing a modified rubber hand illusion protocol where we manipulated the visibility (high, medium, and low) of a virtual hand, not the brush (virtual hand illusion; VHI). To further investigate the impact of visibility manipulations on VHI-related secondary effects (i.e. vicarious somatosensation), we recorded brain activity and skin conductance during a vicarious pain protocol (observation of painful stimulations of the virtual hand) that occurred after the VHI procedure. Results showed that, during both the VHI and vicarious pain periods, the activity of distinct visual, somatosensory, and motor brain regions was modulated by (i) visibility manipulations, (ii) coherence between visual and tactile stimulation, and (iii) time of visuo-tactile stimulation. Accordingly, embodiment-related subjective ratings of the perceived illusion were specifically influenced by visibility manipulations. These findings suggest that visibility modifications can impact the neural and cognitive effects of illusory body ownership, in that when visibility decreases the illusion is perceived as weaker and the brain activity in visual, motor, and somatosensory regions is overall lower. We interpret this evidence as a sign of the weight of vision on embodiment processes, in that the cortical and subjective aspects of illusory body ownership are weakened by a degradation of visual input during the induction of the illusion.

Simultaneous Triboelectric and Mechanoluminescence Sensing Toward Self‐Powered Applications

AbstractSimultaneous phenomena of triboelectricity and mechanoluminescence (ML) acquire vital insights into the mechanics of charge separation and recombination, as well as the relationship between mechanical stress and light emission. In the present work, polydimethylsiloxane (PDMS) and ZnS:Cu particle‐based composites are fabricated, which have good ML characteristics and can generate electricity via contact electrification. ML, in conjunction with a triboelectric nanogenerator (TENG), contributes by producing power from mechanical operations while also giving vital visual input in the form of light emission. This dual capability improves user awareness and efficiency in a variety of applications, making mechanical systems and wearable devices easier to monitor and optimize. To accomplish this, a single‐electrode mode silver (Ag) nanowires embedded PDMS‐ZnS: Cu‐based TENG device is developed and achieved an electrical output of 60 V, 395 nA, and 15 nC by using a linear motor. Furthermore, the combined ML and TENG device is employed in various cases of safety monitoring. This integration provides self‐powered devices that detect mechanical stress, delivering real‐time warnings and illumination signals for increased safety and communication in demanding conditions such as SOS signaling, underwater driving, deep mining, and sports.

Conversations in Content and Context: Can We Teach AI to Chat About Drawings?

An innovative chatbot incorporates a drawing tool allowing users to draw pictures that symbolise the nature of their chronic pain. Rather than simply ask 'What did you draw?" what if the chatbot could engage the user to provide insight into symbols and images related to pain and suffering? The system was tested on images that included features of clinical interest in a first step toward a chatbot able to 'see' and discuss visual input. Such a chatbot has the potential for retrospective and prospective applications. *Contains childhood sexual abuse references.

From Sound to Meaning: Navigating Wernicke's Area in Language Processing.

Wernicke's area, a critical brain region associated with language comprehension, was first identified by Carl Wernicke in the late 19th century. Situated in the left hemisphere's posterior superior temporal gyrus, this area is essential for processing auditory and visual language inputs. It integrates semantic and syntactic information, playing a key role in meaningful communication. The development of Wernicke's area during infancy and childhood is marked by rapid growth and refinement influenced by early language exposure and environmental stimuli. Neuroplasticity, the brain's ability to reorganize and adapt, is crucial for recovery from language impairments such as Wernicke's aphasia. This capacity for reorganization includes synaptic plasticity and axonal sprouting, which facilitate recovery through targeted rehabilitation and enriched environments. Recent research utilizing advanced neuroimaging and neuroanatomical tracing techniques has elucidated the connectivity of Wernicke's area with other language-related regions, such as Broca's area. Functional studies have revealed its specialized roles in processing different aspects of language, including phonological, semantic, and syntactic features. Moreover, investigations into language disorders and potential therapeutic interventions underscore the importance of harnessing neuroplasticity for effective treatment. Emerging technologies, such as non-invasive brain stimulation and multimodal imaging, offer promising avenues for further exploration of Wernicke's area and its role in language functions. These innovations hold the potential to enhance our understanding of language processing and improve therapeutic strategies for language impairments. In conclusion, Wernicke's area is central to language comprehension, and genetic and environmental factors influence its development. Understanding neuroplasticity and leveraging advanced research technologies can significantly advance our ability to address language-related disorders and enhance patient outcomes.

Multi-modal remote perception learning for object sensory data.

When it comes to interpreting visual input, intelligent systems make use of contextual scene learning, which significantly improves both resilience and context awareness. The management of enormous amounts of data is a driving force behind the growing interest in computational frameworks, particularly in the context of autonomous cars. The purpose of this study is to introduce a novel approach known as Deep Fused Networks (DFN), which improves contextual scene comprehension by merging multi-object detection and semantic analysis. To enhance accuracy and comprehension in complex situations, DFN makes use of a combination of deep learning and fusion techniques. With a minimum gain of 6.4% in accuracy for the SUN-RGB-D dataset and 3.6% for the NYU-Dv2 dataset. Findings demonstrate considerable enhancements in object detection and semantic analysis when compared to the methodologies that are currently being utilized.

The effects of emotional arousal on pupil size depend on luminance

Pupillometry is widely used to measure arousal states. The primary functional role of the pupil, however, is to respond to the luminance of visual inputs. We previously demonstrated that cognitive effort-related arousal interacted multiplicatively with luminance, with the strongest pupillary effects of arousal occurring at low-to-mid luminances (< 37 cd/m2), implying a narrow range of conditions ideal for assessing cognitive arousal-driven pupillary differences. Does this generalize to other forms of arousal? To answer this, we assessed luminance-driven pupillary response functions while manipulating emotional arousal, using well-established visual and auditory stimulus sets. At the group level, emotional arousal interacted with the pupillary light response differently from cognitive arousal: the effects occurred primarily at much lower luminances (< 20 cd/m2). Analyses at the individual-participant level revealed qualitatively distinct patterns of modulation, with a sizable number of individuals displaying no arousal response to the visual or auditory stimuli, regardless of luminance. Together, our results suggest that effects of arousal on pupil size are not monolithic: different forms of arousal exert different patterns of effects. More practically, our findings suggest that lower luminances create better conditions for measuring pupil-linked arousal, and when selecting ambient luminance levels, consideration of the arousal manipulation and individual differences is critical.

The cross-script cognate effect in spoken and written second language production: A study based on Chinese-English bilinguals.

Previous studies using cognates with the same writing system have found cognate facilitation effect in the lexical processes of spoken and typewritten productions and cognate interference effect in the sub-lexical process of typewritten production. This study focused on cross-script cognates, Chinese-English, which have different writing systems, and explored cognate effects based on the input and output modalities by using a Chinese-English translation task. Experiment 1 was under visual input modality and investigated the cross-script cognate effect in all three output modalities: spoken, typewritten and handwritten. Results revealed a cognate facilitation effect in the lexical processes across all three output modalities. However, it showed a cognate facilitation effect rather than a cognate interference effect in the sub-lexical process of typewritten production. Experiment 2 was under auditory input modality and focused on exploring cross-script cognate effect on typewritten and handwritten modalities, finding a consistent result on cognate effects with Experiment 1. Both experiments showed higher accuracy for cognates, and there was no significant difference in cgnate effect between visual and auditory inputs. In summary, these findings indicated that the use of cross-script cognates could effectively mitigate cognate interference. While spoken, handwritten and typewritten production share lexical processes, differences emerge in sub-lexical processes, with spoken production being less influenced by orthography. Furthermore, combining the results of Experiments 1 and 2, typewritten production may lean towards the phonological route while handwritten production may favour the direct lexical-orthographic route in the sub-lexical processes.

Experience Dependence of Alpha Rhythms and Neural Dynamics in the Mouse Visual Cortex.

The role of experience in the development and maintenance of emergent network properties such as cortical oscillations and states is poorly understood. To define how early-life experience affects cortical dynamics in the visual cortex of adult, head-fixed mice, we examined the effects of two forms of blindness initiated before eye opening and continuing through recording: (1) bilateral loss of retinal input (enucleation) and (2) degradation of visual input (eyelid suture). Neither form of deprivation fundamentally altered the state-dependent regulation of firing rates or local field potentials. However, each deprivation caused unique changes in network behavior. Laminar analysis revealed two different generative mechanisms for low-frequency synchronization: one prevalent during movement and the other during quiet wakefulness. The former was absent in enucleated mice, suggesting a mouse homolog of human alpha oscillations. In addition, neurons in enucleated animals were less correlated and fired more regularly, but no change in mean firing rate. Eyelid suture decreased firing rates during quiet wakefulness, but not during movement, with no effect on neural correlations or regularity. Sutured animals showed a broadband increase in depth EEG power and an increased occurrence, but reduced central frequency, of narrowband gamma oscillations. The complementary-rather than additive-effects of lid suture and enucleation suggest that the development of emergent network properties does not require vision but is plastic to modified input. Our results suggest a complex interaction of internal set points and experience determines mature cortical activity, with low-frequency synchronization being particularly susceptible to early deprivation.

Multisensory working memory capture of attention.

During visual search, representations in working memory (WM) can guide the deployment of attention toward memory-matching visual input. Although previous studies have demonstrated that multisensory interactions facilitate WM and visual search, it remains unclear whether multisensory interaction influences attentional capture by WM. To address this issue, the present study adopted a dual-task paradigm, pairing a visual search task with a WM task, in which the memory modality was manipulated to be either visual or audiovisual. The results revealed that memory-driven attentional capture was observed under the visual and the audiovisual condition. Additionally, the capture effects and response time (RT) costs under the audiovisual condition were weaker than those under the visual condition, even on the trials with the earliest RTs. Furthermore, RT benefits under the audiovisual condition were comparable with those under the visual condition. These findings suggest that multisensory interactions can enhance cognitive control, leading to robust strategic effects and improved search performance. In this process, cognitive control tends to suppress the attentional capture by WM-matching distractors rather than enhance the attentional capture by WM-matching targets. The present study offers new insights into the influence of multisensory interactions on attentional capture by WM.

The Role of the Cervical Spine in Dizziness.

The neck can be implicated in dizziness via several, separate, discreet mechanisms. Proprioceptive cervicogenic dizziness (CGD) is the most common and reflects the cervical spine's important role, along with visual and vestibular input, for sensorimotor control. Impaired cervical proprioception can lead to symptoms such as dizziness, unsteadiness, visual disturbances, and altered sensorimotor control, and treatment directed toward cervical musculoskeletal and sensorimotor control is efficacious to improve this in individuals with cervical musculoskeletal disorders. Despite this, CGD is difficult to diagnose. Many people present with both neck pain and dizziness, and often the onset of both follows head and neck trauma, but neither necessarily implicate the neck as the cause of dizziness. Further, people often present with mixed forms of dizziness. Thus, rather than diagnosing CGD, it might be more important to consider the potential for the neck to have no (nil, co-morbid cervical condition) or some (minor, major, or compensatory) role in dizziness. Determining the precise role of the cervical spine role in dizziness requires a skilled interview and examination for cervical musculoskeletal and related sensorimotor impairments and relevant testing of other potential causes. A combination and cluster of test outcomes in addition to comparing responses to specific tests when the cervical afferents are stimulated and not stimulated will be important. Considering the role of the neck in dizziness will allow a balanced approach in assessment and management to allow timely, effective intervention to be provided to the large number of individuals presenting with neck pain and dizziness (Supplemental Digital Content, available at: http://links.lww.com/JNPT/A484 ).

Clinical recognition of the role of the cervical spine in signs and symptoms of altered sensorimotor control

Together with visual and vestibular input the cervical spine is vital for sensorimotor control of head and eye movement control, general body postural stability adjustments and co-ordination.Altered cervical input in persons with neck disorders can lead to signs and symptoms of impaired sensorimotor control across and within several domains. Clinical assessment for differential diagnosis and to direct management of cervical related altered sensorimotor control is clearly required in many patients. This applies not only to patients with neck disorders but also in patients with, for example vestibular disorders or concussion, presenting with signs and symptoms of altered sensorimotor control where the cervical spine may have a role to play.This paper explores the research and current knowledge in relation to clinical measures of cervical related sensorimotor control. The feasibility, responsiveness, comparison to gold standards and potential of clinical measures of cervical related sensorimotor control to assist in differential diagnosis are considered where relevant. Future research directions should examine the full complement of psychometric properties of tests and consideration of any relationships of these measures to pain, dizziness, trauma and functional implications. Development of other measures as well as use in assessing response post management are also important directions.These clinical measures, along with a skilled interview and cervical musculoskeletal examination will enable clinicians to recognise and manage impaired cervical sensorimotor control in patients with neck disorders as well as determine the role of the cervical spine in many patients presenting with signs and symptoms of altered sensorimotor control.