Visual Processing System Research Articles

APM: Adaptive parameter multiplexing for class incremental learning

In recent developments within the domain of image classification, deep neural networks (DNNs) have attracted considerable scholarly interest and have been extensively trained using data in closed environments. Such training methodologies contrast sharply with the inherently open, progressive, and adaptive processes of the natural visual system, leading to emergent challenges. Among these, catastrophic forgetting is notable, where the network acquisition of new class information precipitates the erosion of previously established knowledge. Additionally, the network encounters the stability-plasticity dilemma, necessitating a delicate equilibrium between assimilating novel classes and retaining existing ones. To address these issues, we propose a novel incremental learning model, termed Adaptive Parameter Multiplexing (APM), which incorporates a cross-class parameter adaptive incremental strategy. Central to our methodology is the conceptualization of parameter multiplexing or incremental as a learnable optimization problem, enabling the model to autonomously evaluate and decide on the necessity for parameter adjustment throughout its training lifecycle. This framework is designed to enhance the ability of the network to extract features for new class categories effectively through incremental parameters while simultaneously employing parameter multiplexing to augment storage optimization. Our model is underpinned by a dual strategy of coarse-grained and fine-grained parameter multiplexing, guided by a learnable score that dynamically assesses the appropriateness of parameter multiplexing versus incremental updates, facilitating an optimized balance for incremental model performance and storage. In addition, we have integrated a novel regularization loss mechanism for the learnable score to optimize storage efficiency. The effectiveness of APM is empirically validated through rigorous testing on benchmark datasets, including ImageNet100, CIFAR100, CIFAR10, and CUB200. The experimental outcomes indicate that, with a trace amount of parameter increase, our model achieves significant enhancements in classification performance across both new and previously established classes, thereby surpassing existing benchmarks set by state-of-the-art algorithms in the field.

A prospective multi-center study quantifying visual inattention in delirium using generative models of the visual processing stream

The visual attentional deficits in delirium are poorly characterized. Studies have highlighted neuro-anatomical abnormalities in the visual processing stream but fail at quantifying these abnormalities at a functional level. To identify these deficits, we undertook a multi-center eye-tracking study where we recorded 210 sessions from 42 patients using a novel eye-tracking system that was made specifically for free-viewing in the (ICU); each session lasted 10 min and was labeled with the delirium status of the patient using the Confusion Assessment Method in ICU (CAM-ICU). To analyze this data, we formulate the task of visual attention as a hierarchical generative process that yields a probabilistic distribution of the location of the next fixation. This distribution can then be compared to the measured patient fixation producing a correctness score which is tallied compared across delirium status. This analysis demonstrated that the visual processing system of patients suffering from delirium is functionally restricted to a statistically significant degree. This is the first study to explore the potential mechanisms underpinning visual inattention in delirium and suggests a new target of future research into a disease process that affects one in four hospitalized patients with severe short and long-term consequences.

Higher-contrast images are better remembered during naturalistic encoding

It is unclear whether memory for images of poorer visibility (as low contrast or small size) will be lower due to weak signals elicited in early visual processing stages, or perhaps better since their processing may entail top-down processes (as effort and attention) associated with deeper encoding. We have recently shown that during naturalistic encoding (free viewing without task-related modulations), for image sizes between 3°–24°, bigger images stimulating more visual system processing resources at early processing stages are better remembered. Similar to size, higher contrast leads to higher activity in early visual processing. Therefore, here we hypothesized that during naturalistic encoding, at critical visibility ranges, higher contrast images will lead to higher signal-to-noise ratio and better signal quality flowing downstream and will thus be better remembered. Indeed, we found that during naturalistic encoding higher contrast images were remembered better than lower contrast ones (~ 15% higher accuracy, ~ 1.58 times better) for images at 7.5–60 RMS contrast range. Although image contrast and size modulate early visual processing very differently, our results further substantiate that at poor visibility ranges, during naturalistic non-instructed visual behavior, physical image dimensions (contributing to image visibility) impact image memory.

Shape from dots: a window into abstraction processes in visual perception

IntroductionA remarkable phenomenon in perception is that the visual system spontaneously organizes sets of discrete elements into abstract shape representations. We studied perceptual performance with dot displays to discover what spatial relationships support shape perception.MethodsIn Experiment 1, we tested conditions that lead dot arrays to be perceived as smooth contours vs. having vertices. We found that the perception of a smooth contour vs. a vertex was influenced by spatial relations between dots beyond the three points that define the angle of the point in question. However, there appeared to be a hard boundary around 90° such that any angle 90° or less was perceived as a vertex regardless of the spatial relations of ancillary dots. We hypothesized that dot arrays whose triplets were perceived as smooth curves would be more readily perceived as a unitary object because they can be encoded more economically. In Experiment 2, we generated dot arrays with and without such “vertex triplets” and compared participants’ phenomenological reports of a unified shape with smooth curves vs. shapes with angular corners. Observers gave higher shape ratings for dot arrays from curvilinear shapes. In Experiment 3, we tested shape encoding using a mental rotation task. Participants judged whether two dot arrays were the same or different at five angular differences. Subjects responded reliably faster for displays without vertex triplets, suggesting economical encoding of smooth displays. We followed this up in Experiment 4 using a visual search task. Shapes with and without vertex triplets were embedded in arrays with 25 distractor dots. Participants were asked to detect which display in a 2IFC paradigm contained a shape against a distractor with random dots. Performance was better when the dots were sampled from a smooth shape than when they were sampled from a shape with vertex triplets.Results and discussionThese results suggest that the visual system processes dot arrangements as coherent shapes automatically using precise smoothness constraints. This ability may be a consequence of processes that extract curvature in defining object shape and is consistent with recent theory and evidence suggesting that 2D contour representations are composed of constant curvature primitives.

A histological and diceCT-derived 3D reconstruction of the avian visual thalamofugal pathway

Amniotes feature two principal visual processing systems: the tectofugal and thalamofugal pathways. In most mammals, the thalamofugal pathway predominates, routing retinal afferents through the dorsolateral geniculate complex to the visual cortex. In most birds, the thalamofugal pathway often plays the lesser role with retinal afferents projecting to the principal optic thalami, a complex of several nuclei that resides in the dorsal thalamus. This thalamic complex sends projections to a forebrain structure called the Wulst, the terminus of the thalamofugal visual system. The thalamofugal pathway in birds serves many functions such as pattern discrimination, spatial memory, and navigation/migration. A comprehensive analysis of avian species has unveiled diverse subdivisions within the thalamic and forebrain structures, contingent on species, age, and techniques utilized. In this study, we documented the thalamofugal system in three dimensions by integrating histological and contrast-enhanced computed tomography imaging of the avian brain. Sections of two-week-old chick brains were cut in either coronal, sagittal, or horizontal planes and stained with Nissl and either Gallyas silver or Luxol Fast Blue. The thalamic principal optic complex and pallial Wulst were subdivided on the basis of cell and fiber density. Additionally, we utilized the technique of diffusible iodine-based contrast-enhanced computed tomography (diceCT) on a 5-week-old chick brain, and right eyeball. By merging diceCT data, stained histological sections, and information from the existing literature, a comprehensive three-dimensional model of the avian thalamofugal pathway was constructed. The use of a 3D model provides a clearer understanding of the structural and spatial organization of the thalamofugal system. The ability to integrate histochemical sections with diceCT 3D modeling is critical to better understanding the anatomical and physiologic organization of complex pathways such as the thalamofugal visual system.

Visuo-Cognitive Phenotypes in Early Multiple Sclerosis: A Multisystem Model of Visual Processing

Background: Cognitive impairment can emerge in the earliest stages of multiple sclerosis (MS), with heterogeneity in cognitive deficits often hindering symptom identification and management. Sensory–motor dysfunction, such as visual processing impairment, is also common in early disease and can impact neuropsychological task performance in MS. However, cognitive phenotype research in MS does not currently consider the relationship between early cognitive changes and visual processing impairment. Objectives: This study explored the relationship between cognition and visual processing in early MS by adopting a three-system model of afferent sensory, central cognitive and efferent ocular motor visual processing to identify distinct visuo-cognitive phenotypes. Methods: Patients with clinically isolated syndrome and relapsing–remitting MS underwent neuro-ophthalmic, ocular motor and neuropsychological evaluation to assess each visual processing system. The factor structure of ocular motor variables was examined using exploratory factor analysis, and phenotypes were identified using latent profile analysis. Results: Analyses revealed three ocular-motor constructs (cognitive control, cognitive processing speed and basic visual processing) and four visuo-cognitive phenotypes (early visual changes, efferent-cognitive, cognitive control and afferent-processing speed). While the efferent-cognitive phenotype was present in significantly older patients than was the early visual changes phenotype, there were no other demographic differences between phenotypes. The efferent-cognitive and cognitive control phenotypes had poorer performance on the Symbol Digit Modalities Test compared to that of other phenotypes; however, no other differences in performance were detected. Conclusion: Our findings suggest that distinct visual processing deficits in early MS may differentially impact cognition, which is not captured using standard neuropsychological evaluation. Further research may facilitate improved symptom identification and intervention in early disease.

Detection of lanes, obstacles and drivable areas for self-driving cars using multifusion perception metrics

<p>Autonomous vehicles have been a recent trend and active research area from the onset of machine learning and deep learning algorithms. Computer vision and deep learning techniques have simplified the operations of continuous monitoring and decision-making capabilities of autonomous vehicles. A navigation system is facilitated by a visual system, where sensors and collectors process input in form of images or videos, and the navigation system will be making certain decisions to adhere to the safety of drivers and passers-by. This research article contemplates the model of obstacle detection, lane detection, and how the vehicle is supposed to act in terms of autonomous driving situation. This situation should resemble human driving conditions and should ensure maximum safety to both the stakeholders. A unified neural network for detecting lanes, objects, obstacles and to advise the driving speed is defined in this architecture. As far as autonomous driving is considered, these target elements are considered to be the predominant areas of focus for autonomous driving vehicles. Since capturing the images or videos have to be performed in real-time scenarios and processing them for relevant decision making have to be completed at a swift pace, a concept of context tensors is introduced in the decoders for discriminating the tasks based on priority. Every task is associated with the other tasks and also the decision-making process and hence this architecture will continue to learn every day. From the obtained results, it is evident that multitask networks can be improved using the proposed method in terms of accuracy, decision-making capability and reduced computational time. This model investigates the performance using Berkeley deep drive datasets which are considered to be a challenging dataset.</p>

Sensorimotor Dysfunction Following Anterior Cruciate Ligament Reconstruction- an Afferent Perspective: A Scoping Review.

Sensorimotor dysfunction is thought to occur following anterior cruciate ligament (ACL) injury which may have implications on future reinjury risk. Dysfunction has been demonstrated within the efferent component of the sensorimotor system. However, no reviews have examined the two main components of the afferent system: the visual and somatosensory systems. This study aimed to report differences in function (central processing and local processing) within the (1) somatosensory and (2) visual systems between individuals following anterior cruciate ligament reconstruction (ACLR) and healthy controls (between-subject). The study also aimed to report differences in function within the two systems between the two limbs of an individual following ACLR (within-subject). Scoping review. A search was conducted in PubMed, SPORTDiscus, CINAHL, Medline and Embase up until September 2021. Level I-IV studies assessing somatosensory and visual systems were included if they compared ACLR limbs to the uninjured contralateral limb (within-subject) or a healthy control limb (between-group). The function of somatosensory and visual systems was assessed across both central processing (processing of information in the central cortex) and local processing (all other assessments outside of central processing of information). Seventy studies were identified (52 somatosensory, 18 visual). Studies examining somatosensory central processing demonstrated significant differences; 66% of studies exhibited within-subject differences and 100% of the studies exhibited between-group differences. Studies examining local somatosensory processing had mixed findings; 40% of the 'joint position sense (JPS)' and 'threshold to detect motion (TTDM)' studies showed significant within-subject differences (JPS=0.8°-3.8° and TTDPM=0.2°-1.4°) and 42% demonstrated significant between-group differences (JPS=0.4°-5° and TTDPM=0.3°-2.8°). Eighty-three percent of visual central processing studies demonstrated significant dysfunction between-groups with no studies assessing within-subject differences. Fifty percent of the studies examining local visual processing demonstrated a significant between-group difference. Significant differences in central processing exist within somatosensory and visual systems following ACLR. There is mixed evidence regarding local somatosensory and visual processing. Increased compensation by the visual system and local visual processing dysfunction may occur in conjunction with somatosensory dysfunction.

Fractal Fluency: Processing of Fractal Stimuli Across Sight, Sound, and Touch.

People are continually exposed to the rich complexity generated by the repetition of fractal patterns at different size scales. Fractals are prevalent in natural scenery and also in patterns generated by artists and mathematicians. In this chapter, we will investigate the powerful significance of fractals for the human senses. In particular, we propose that fractals with mid-range complexity play a unique role in our visual experiences because the visual system has adapted to these prevalent natural patterns. This adaptation is evident at multiple stages of the visual system, ranging from data acquisition by the eye to processing of this data in the higher visual areas of the brain. Based on these results, we will discuss a fluency model in which the visual system processes mid-complexity fractals with relative ease. This fluency optimizes the observer's capabilities (such as enhanced attention and pattern recognition) and generates an aesthetic experience accompanied by a reduction in the observer's physiological stress levels. In addition to reviewing people's responses to viewing fractals, we will compare these responses to recent research focused on fractal sounds and fractal surface textures. We will extend our fractal fluency model to allow for stimuli across multiple senses.

Emotion-gaze interaction affects time-to-collision estimates, but not preferred interpersonal distance towards looming faces.

Estimating the time until impending collision (time-to-collision, TTC) of approaching or looming individuals and maintaining a comfortable distance from others (interpersonal distance, IPD) are commonly required in daily life and contribute to survival and social goals. Despite accumulating evidence that facial expressions and gaze direction interactively influence face processing, it remains unclear how these facial features affect the spatiotemporal processing of looming faces. We examined whether facial expressions (fearful vs. neutral) and gaze direction (direct vs. averted) interact on the judgments of TTC and IPD for looming faces, based on the shared signal hypothesis that fear signals the existence of threats in the environment when coupled with averted gaze. Experiment 1 demonstrated that TTC estimates were reduced for fearful faces compared to neutral ones only when the concomitant gaze was averted. In Experiment 2, the emotion-gaze interaction was not observed in the IPD regulation, which is arguably sensitive to affective responses to faces. The results suggest that fearful-averted faces modulate the cognitive extrapolation process of looming motion by communicating environmental threats rather than by altering subjective fear or perceived emotional intensity of faces. The TTC-specific effect may reflect an enhanced defensive response to unseen threats implied by looming fearful-averted faces. Our findings provide insight into how the visual system processes facial features to ensure bodily safety and comfortable interpersonal communication in dynamic environments.

Electrophysiological study of visual processing in children with cochlear implants

Electrophysiological studies of congenitally deaf children and adults have reported atypical visual evoked potentials (VEPs) which have been associated with both behavioral enhancements of visual attention as well as poorer performance and outcomes in tests of spoken language speech processing. This pattern has often been interpreted as a maladaptive consequence of early auditory deprivation, whereby a remapping of auditory cortex by the visual system ultimately reduces resources necessary for optimal rehabilitative outcomes of spoken language acquisition and use. Making use of a novel electrophysiological paradigm, we compare VEPs in children with severe to profound congenital deafness who received a cochlear implant(s) prior to 31 months (n = 28) and typically developing age matched controls (n = 28). We observe amplitude enhancements and in some cases latency differences in occipitally expressed P1 and N1 VEP components in CI-using children as well as an early frontal negativity, N1a. We relate these findings to developmental factors such as chronological age and spoken language understanding. We further evaluate whether VEPs are additionally modulated by auditory stimulation. Collectively, these data provide a means to examine the extent to which atypical VEPs are consistent with prior accounts of maladaptive cross-modal plasticity. Our results support a view that VEP changes reflect alterations to visual-sensory attention and saliency mechanisms rather than a re-mapping of auditory cortex. The present data suggests that early auditory deprivation may have temporally prolonged effects on visual system processing even after activation and use of cochlear implant.

Immune-related visual dysfunction in thyroid eye disease: a combined orbital and brain neuroimaging study.

To investigate the pathological interplay between immunity and the visual processing system (VPS) in thyroid eye disease (TED). A total of 24 active patients (AP), 26 inactive patients (IP) of TED, and 27 healthy controls (HCs) were enrolled. Orbital magnetic resonance imaging (MRI) and resting-state functional MRI (rs-fMRI) were conducted for each participant. Multiple MRI parameters of the intraorbital optic nerve (ON) were assessed. The amplitude of low-frequency fluctuations (ALFF) and regional homogeneity (ReHo) were calculated. Correlation analyses were carried out on the above parameters and clinical characteristics. Visual functioning scores differentiated between the AP and IP groups. The ON subarachnoid space and ON sheath diameter were significantly higher in AP than in IP. Six vision-related brain regions were identified in TED patients compared with HCs, including right calcarine (CAL.R), right cuneus (CUN.R), left postcentral gyrus (PoCG.L), right middle temporal gyrus (MTG.R), left superior frontal gyrus (SFG.L), and left caudate (CAU.L). The brain activity of MTG.R, SFG.L, and CAU.L differentiated between the AP and IP groups. The correlation analysis revealed a close association among the vision-related brain regions, MRI parameters of ON, and clinical characteristics in AP and IP, respectively. Combined orbital and brain neuroimaging revealed abnormalities of the VPS in TED, which had a close correlation with immune statuses. Vision-related brain regions in TED might be possibly altered by peripheral immunity via a direct or indirect approach. The discovery of this study explained the disparity of visual dysfunction in TED patients with different immune statuses. With the uncovered neuroimaging markers, early detection and intervention of visual dysfunction could be achieved and potentially benefit TED patients. • Patients with different immune statuses of thyroid eye disease varied in the presentation of visual dysfunction. • The combined orbital and brain neuroimaging study identified six altered vision-related brain regions, which had a significant correlation with the MRI parameters of the intraorbital optic nerve and immunological characteristics. • Peripheral immunity might possibly give rise to alterations in the central nervous system part of the visual processing system via a direct or indirect approach.

The effects of modular process models on gaze patterns - A follow-up investigation about modularization in process model literacy

Advanced organizational processes today require many process-relevant artifacts to be formally represented in process models. As these processes become more complex, process designers face the challenge of maintaining high comprehensibility while adequately visualizing process-relevant information. Modularization has been introduced to address the increasing complexity and information overload of process models, tackling this issue. While research acknowledges the importance of process model comprehensibility in order to benefit from the merits of such models, there still needs to be more clarity about the benefits of process model modularization. Recent research has focused on the human visual system and investigated visual behavior in the context of process model literacy. Continuing this branch of research, this paper analyses gaze behavior when comprehending modularized process models. A follow-up eye-tracking study was conducted to comprehend differently modularized BPMN 2.0 process models (i.e., vertical, horizontal, orthogonal). The results indicated that process model readers comprehend modularization approaches differently. Vertical modularization was perceived as the best comprehensible approach, whereas horizontal received the least acceptance. Considered eye movement performance measures confirm prior observation that attention is deployed more effectively in vertical modularization. Based upon the findings, the paper proposes a revised theoretical model from prior work that relates visual tasks to the execution of visual routines, explaining comprehension and information processing in modularized process models. The theoretical model elaborates how the visual system process and comprehends information obtained from modularized process models. Furthermore, the paper derived implications for modeling and practical applications, pointing out solid representational points and shortcomings of each modularization approach regarding how modularized process models are read and comprehended.

Faces in scenes attract rapid saccades.

During natural vision, the human visual system has to process upcoming eye movements in parallel to currently fixated stimuli. Saccades targeting isolated faces are known to have lower latency and higher velocity, but it is unclear how this generalizes to the natural cycle of saccades and fixations during free-viewing of complex scenes. To which degree can the visual system process high-level features of extrafoveal stimuli when they are embedded in visual clutter and compete with concurrent foveal input? Here, we investigated how free-viewing dynamics vary as a function of an upcoming fixation target while controlling for various low-level factors. We found strong evidence that face- versus inanimate object-directed saccades are preceded by shorter fixations and have higher peak velocity. Interestingly, the boundary conditions for these two effects are dissociated. The effect on fixation duration was limited to face saccades, which were small and followed the trajectory of the preceding one, early in a trial. This is reminiscent of a recently proposed model of perisaccadic retinotopic shifts of attention. The effect on saccadic velocity, however, extended to very large saccades and increased with trial duration. These findings suggest that multiple, independent mechanisms interact to process high-level features of extrafoveal targets and modulate the dynamics of natural vision.

PS-Net: Human perception-guided segmentation network for EM cell membrane.

Cell membrane segmentation in electron microscopy (EM) images is a crucial step in EM image processing. However, while popular approaches have achieved performance comparable to that of humans on low-resolution EM datasets, they have shown limited success when applied to high-resolution EM datasets. The human visual system, on the other hand, displays consistently excellent performance on both low and high resolutions. To better understand this limitation, we conducted eye movement and perceptual consistency experiments. Our data showed that human observers are more sensitive to the structure of the membrane while tolerating misalignment, contrary to commonly used evaluation criteria. Additionally, our results indicated that the human visual system processes images in both global-local and coarse-to-fine manners. Based on these observations, we propose a computational framework for membrane segmentation that incorporates these characteristics of human perception. This framework includes a novel evaluation metric, the perceptual Hausdorff distance (PHD), and an end-to-end network called the PHD-guided segmentation network (PS-Net) that is trained using adaptively tuned PHD loss functions and a multiscale architecture. Our subjective experiments showed that the PHD metric is more consistent with human perception than other criteria, and our proposed PS-Net outperformed state-of-the-art methods on both low and high resolution EM image datasets as well as other natural image datasets. The code and dataset can be found at https://github.com/EmmaSRH/PS-Net. Supplementary Information for this article is available online.

Physical reservoir computing using vertically aligned graphene/diamond photomemristors

Reservoir computing is one of the most promising machine learning architectures and could allow highly efficient, high-speed processing of time-series data. Physical reservoir computing based on various physical phenomena that exhibit complicated dynamics has been widely investigated in recent years. The present work demonstrates vertically aligned graphene/diamond junctions (photomemristors) could be employed for physical reservoir computing involving image recognition of single digits. Exceptional image recognition performance of 92% was obtained due to their complex photoconducting behaviors. This work is expected to assist in the realization of novel visual information processing systems using photomemristors that mimic human brain functions.

Haar Wavelet-Based Classification Method for Visual Information Processing Systems

Nowadays, the systems for visual information processing are significantly extending their application field. Moreover, an unsolved problem for such systems is that the classification procedure has often-conflicting requirements for performance and classification reliability. Therefore, the goal of the article is to develop the wavelet method for classifying the systems for visual information processing by evaluating the performance and informativeness of the adopted classification solutions. This method of classification uses the Haar wavelet functions with training and calculates the ranges of changes in the coefficients of the separating surfaces. The authors proposed to select the ranges of changes in these coefficients by employing the Shannon entropy formula for measuring the information content. A case study proved that such a method will significantly increase the speed of detecting the intervals of coefficient values. In addition, this enables us to justify the choice of the width of the ranges for the change of coefficients, solving the contradiction between the performance and reliability of the classifier.

Smooth pursuit eye movements contribute to anticipatory force control during mechanical stopping of moving objects.

When stopping a closing door or catching an object, humans process the motion of inertial objects and apply reactive limb force over short period to interact with them. One way in which the visual system processes motion is through extraretinal signals associated with smooth pursuit eye movements (SPEMs). We conducted three experiments to investigate how SPEMs contribute to anticipatory and reactive hand force modulation when interacting with a virtual object moving in the horizontal plane. We hypothesized that SPEM signals are critical for timing motor responses, anticipatory control of hand force, and task performance. Participants held a robotic manipulandum and attempted to stop an approaching simulated object by applying a force impulse (area under force-time curve) that matched the object's virtual momentum upon contact. We manipulated the object's momentum by varying either its virtual mass or its speed under free gaze or constrained gaze conditions. We examined gaze variables, the timing of hand motor responses, anticipatory force control, and overall task performance. Our results show that when participants were fixated at a designated location instead of following objects with SPEM, anticipatory modulation of hand force before contact decreased. However, constraining gaze by asking participants to fixate did not seem to affect the timing of the motor response or the task performance. Together, these results suggest that SPEMs may be important for anticipatory control of hand force before contact and may also play a critical role in anticipatory stabilization of limb posture when humans interact with moving objects.NEW & NOTEWORTHY We show for the first time that smooth pursuit eye movements (SPEMs) play a role in the modulation of anticipatory control of hand force to stabilize posture against contact forces. SPEMs are critical for tracking moving objects, facilitate processing motion of moving objects, and are impacted during aging and in many neurological disorders, such as Alzheimer's disease and multiple sclerosis. These results provide a novel basis to probe how changes in SPEMs could contribute to deficient limb motor control in older adults and patients with neurological disorders.

Neuromorphic visual artificial synapse in-memory computing systems based on GeOx-coated MXene nanosheets

Artificial synapses with light signal perception capability offer the ability to neuromorphic visual signal processing system on demand. In light of the excellent optical and electrical characteristics, the low-dimensional materials have become one of the most favorable candidates of the key component for optoelectronic artificial synapses. Previously, our group originally proposed the synthesis of germanium oxide-coated MXene nanosheets. In this work, we further applied this technology into the optoelectronic synaptic thin-film transistors for the first time. The devices exhibited the adjustable postsynaptic current behaviors under the visible light inputs. Moreover, the potentiation and depression operation modes of the devices further improved the application potential of the devices in mimicking biological synapses. Regulated by the wavelength of incident lights, the proposed artificial synapse could effectively help detect the target area of the image. Eventually, we further showed the results of the devices in the projects of neural network computing task. The long-term potentiation/depression characteristics of the conductance were applied to the synaptic weight matrix for image identification and path recognition tasks. By adding knowledge transfer in the process of recognition, the epoch required for convergence has been greatly reduced. The result of high noise tolerance revealed the great potential of the proposed transistors in establishing high-efficiency and robustness hardware neuromorphic systems for in-memory computing.

Baseline free water within the visual processing system predicts future psychosis in Parkinson disease.

As psychosis is associated with decreased quality of life, increased institutionalization, and mortality in Parkinson disease (PD), it is essential to identify individuals at risk for future psychosis. This longitudinal study aimed to investigate whether diffusion tensor imaging (DTI) metrics of white matter hold independent utility for predicting future psychosis in PD, and whether they could be combined with clinical predictors to improve the prognostication of PD psychosis. This study included 123 newly diagnosed PD patients collected in the Parkinson's Progression Markers Initiative. Tract-based spatial statistics were used to compare baseline DTI metrics between PD patients who developed psychosis and those who did not during follow-up. Binary logistic regression analyses were performed to identify the clinical and white matter markers predictive of psychosis. Among DTI measures, both higher baseline whole brain (odds ratio [OR] = 1.711, p= 0.016) free water (FW) and visual processing system (OR=1.680, p< 0.001) FW were associated with an increased risk of future psychosis. Baseline FW remained a significant indicator of future psychosis in PD after controlling for clinical predictors. Moreover, the accuracy of prediction of psychosis using clinical predictors alone (area under the curve [AUC] = 0.742, 95% confidence interval [CI] = 0.655-0.816) was significantly improved by the addition of the visual processing system FW (AUC=0.856, 95% CI=0.781-0.912; Delong method, p= 0.022). Baseline FW of the visual processing system incurs an independent risk of future psychosis in PD, thus providing an opportunity for multiple-modality marker models to include a white matter marker.