Visual Research Articles

VPRF: Visual Perceptual Radiance Fields for Foveated Image Synthesis.

Neural radiance fields (NeRF) has achieved revolutionary breakthrough in the novel view synthesis task for complex 3D scenes. However, this new paradigm struggles to meet the requirements for real-time rendering and high perceptual quality in virtual reality. In this paper, we propose VPRF, a novel visual perceptual based radiance fields representation method, which for the first time integrates the visual acuity and contrast sensitivity models of human visual system (HVS) into the radiance field rendering framework. Initially, we encode both the appearance and visual sensitivity information of the scene into our radiance field representation. Then, we propose a visual perceptual sampling strategy, allocating computational resources according to the HVS sensitivity of different regions. Finally, we propose a sampling weight-constrained training scheme to ensure the effectiveness of our sampling strategy and improve the representation of the radiance field based on the scene content. Experimental results demonstrate that our method renders more efficiently, with higher PSNR and SSIM in the foveal and salient regions compared to the state-of-the-art FoV-NeRF. The results of the user study confirm that our rendering results exhibit high-fidelity visual perception.

Visualizing the obscure- Chromophore-assisted retinal break detection.

Finding a retinal break which is responsible for retinal detachment is a critical step in the surgical treatment of this condition. In spite of improvement in visualization systems in pars plana vitrectomy, identifying the break which is the source of subretinal fluid (SRF) could pose a challenge, especially in complex retinal detachments or re-detachments with extensive laser scars. The current technique of break identification involves direct detection of the break via peripheral indentation or using perfluorocarbon liquid (PFCL) to displace SRF from the posterior retina to the periphery and visualizing the proteinaceous SRF egress from the retinal break. In retinal detachments, if no peripheral breaks are visualized, surgeons tend to make a posterior draining retinotomy and drain SRF, the necessity of which can be confirmed after subretinal brilliant blue green (BBG) dye injection. In addition, in already vitrectomized eyes where it can be difficult to identify SRF egressing via the break, subretinal BBG dye provides a color contrast that enhances break visualization. A method to identify these occult breaks was required, which is described in this video. Hence, the idea of BBG dye injection into the subretinal space with a 41G subretinal needle was thought of. PFCL would be placed over the posterior pole, which would displace the dye to the periphery, and this would disperse out into the vitreous cavity into a blue plume via the break, thereby leading to its identification. Subretinal dye injection is useful for occult retinal break detection as it provides a color contrast. A posterior draining retinotomy can be avoided. It is also useful in recurrent retinal detachments. We describe the use of BBG, which is the least toxic dye available, and report the use of this technique in Indian eyes. https://youtu.be/JGXQjNV9asw.

Scene-aware Foveated Rendering.

We propose a new scene-aware foveated rendering method, which incorporates the scene awareness and characteristics of the human visual system into the mapping-based foveated rendering framework. First, we generate the conservative visual importance map that encodes the visual features of the scene, visual acuity, and gaze motion. Second, we construct the pixel size control map using a convolution kernel method. Third, we utilize the pixel size control map to guide the foveated rendering. At last, a temporal coherent refinement strategy is used to maintain the smooth foveated rendering for the adjacent frames. Compared to the state-of-the-art mapping-based foveated rendering methods using the same compression ratio, our method achieves smaller MSE, higher PSNR, and SSIM in the fovea, periphery, salient regions, and the whole image. We also conducted user studies, and the results proved that the perceptual quality of our method has a high visual similarity with the around truth rendered with the full resolution.

Subjective visual vertical/horizontal and video head impulse test in dyslexic children.

Different studies have tried to establish a relationship between dyslexia and the vestibular system function. Subjective Visual Vertical/Horizontal (SVV and SVH) and Video Head Impulse Test (VHIT) are useful for studying the vestibular system and can be easily performed in children. Our aim was to evaluate the vestibular function in dyslexic children by SVV/SVH and VHIT. We enrolled 18 dyslexic children (10M/8F; mean age 10.7 ± 2.3 years; range 7-14 years) and 18 age-matched children with typical development of learning abilities. All children performed VHIT, SVV and SVH. We found normal gain and symmetry of vestibulo-ocular-reflex both in dyslexic and typically developing children. Fifteen out of 18 dyslexic children (83.3%) showed a difference of at least one amongst SVV or SVH. The mean value of SVV was 2.3° and the mean value of SVH was 2.6°. Statistical analysis showed a significant difference between typically developing and dyslexic children for both SVV and SVH. We confirm a relationship between dyslexia and the alteration of SVV and SVH. Our results could be related to the pathogenetic hypothesis of a visual processing impairment related to a dysfunction of the magnocellular pathway or to a general deficit related to a multimodal cortical network.

Pixel-Level Precision Saccade Control of Arbitrary 3D Spatial Points in Active Binocular Vision Systems

Pixel-Level Precision Saccade Control of Arbitrary 3D Spatial Points in Active Binocular Vision Systems

Retinal Damage and Visual Network Reconfiguration Defines Visual Function Recovery in Optic Neuritis.

Recovery of vision after acute optic neuritis (AON) is critical to improving the quality of life of people with demyelinating diseases. The objective of the study was to prospectively assess the changes in visual acuity, retinal layer thickness, and cortical visual network in patients with AON to identify the predictors of permanent visual disability. We studied a prospective cohort of 88 consecutive patients with AON with 6-month follow-up using high and low-contrast (2.5%) visual acuity, color vision, retinal thickness from optical coherence tomography, latencies and amplitudes of multifocal visual evoked potentials, mean deviation of visual fields, and diffusion-based structural (n = 53) and functional (n = 19) brain MRI to analyze the cortical visual network. The primary outcome was 2.5% low-contrast vision, and data were analyzed with mixed-effects and multivariate regression models. We found that after 6 months, low-contrast vision and quality of vision remained moderately impaired. The thickness of the ganglion cell layer at baseline was a predictor of low-contrast vision 6 months later (ß = 0.49 [CI 0.11-0.88], p = 0.012). The structural cortical visual network at baseline predicted low-contrast vision, the best predictors being the betweenness of the right parahippocampal cortex (ß = -036 [CI -0.66 to 0.06], p = 0.021), the node strength of the right V3 (ß = 1.72 [CI 0.29-3.15], p = 0.02), and the clustering coefficient of the left intraparietal sulcus (ß = 57.8 [CI 12.3-103.4], p = 0.015). The functional cortical visual network at baseline also predicted low-contrast vision, the best predictors being the betweenness of the left ventral occipital cortex (ß = 8.6 [CI: 4.03-13.3], p = 0.009), the node strength of the right intraparietal sulcus (ß = -2.79 [CI: -5.1-0.4], p = 0.03), and the clustering coefficient of the left superior parietal lobule (ß = 501.5 [CI 50.8-952.2], p = 0.03). The assessment of the visual pathway at baseline predicts permanent vision disability after AON, indicating that damage is produced early after disease onset and that it can be used for defining vision impairment and guiding therapy.

Machine vision system for automatic defect detection of ultrasound probes

Machine vision system for automatic defect detection of ultrasound probes

Validation of two simple scores to identify increased risk of transthyretin amyloid cardiomyopathy in heart failure with preserved ejection fraction

Abstract Background Transthyretin Amyloid Cardiomyopathy (ATTR-CM) is an emerging underdiagnosed cause of heart failure (HF). 99mTc-3, 3-diphosphono-1,2-propanodicarboxylic acid (DPD) allows ATTR-CM diagnosis in the absence of histology. Recently two simple clinical scores to identify increased risk of ATTR-CM in patients with heart failure and preserved left ventricular ejection fraction have been developed and validated. Purpose To assess the prevalence of ATTR-CM in a Mediterranean cohort of patients at risk, and to externally validate the Mayo ATTR-CM and T-Amylo scores. We also explore its diagnostic accuracy improvement by adding N-terminal pro-B-type natriuretic peptide (NT-proBNP). Methods Retrospective cohort study of consecutive patients with suspected cardiac amyloidosis referred to DPD between March 2017 and November 2020. Scans were reported using a visual grading system (0 to 3 increasingly; 2-3 positive) on planar images. Patients with clonal dyscrasia or those without rule-out tests (serum free light chain assay, and serum and urine protein electrophoresis with immunofixation) were excluded. Discrimination was obtained using the area under the curve (AUC) of the receiver operating characteristic (ROC) curve based on the ATTR-CM score [High risk: score ≥ 6. Components: age (60-69 = 2 points; 70-79 = 3 points; ≥80 = 4 points), male sex = 2 points, hypertension = -1 point, ejection fraction <60% = 1 point, posterior wall thickness ≥12 mm = 1 point, relative wall thickness (septal + posterior wall thickness/left ventricular end diastolic diameter) >0.57 = 2 points], and T-amylo score [High risk: score > 6. Components: age ≥ 80 years = 1 point, interventricular septum thickness ≥16 mm = 2 points, low QRS interval voltage = 2 points, male gender = 3 points, and carpal tunnel syndrome = 3 points. Calibration was assessed with the Hosmer-Lemeshow goodness-of-fit test. Results A total of 200 patients were included with a mean age of 78.5 ± 11 years, 45.5% male, left ventricular ejection fraction 58% ± 12 and a median NT-proBNP of 2310 pg/mL (Table 1). 81 (40.5%) and 10 (5.0%) patients were categorized as a high-risk group due to ATTR-CM score value ≥6 and simplified T-Amylo score >6, respectively. Prevalence of ATTR-CM was 17.5%. The AUC for the Mayo ATTR-CM and T-Amylo scores in our cohort were 0.83 (95% CI 0.75-0.90) and 0.79 (95% CI 0.69-0.88) respectively, with an appropriated calibration (Figure 1). The addition of NT-proBNP to the scores did not increase the AUC (p=0.19 and p=0.08), with a Net Reclassification Improvement (NRI) of 0.055 (95% CI - 0.110 to 0.206) and 0.033 (95% CI - 0.113 to 0.224) respectively. Conclusion These two simple clinical scores are valid and reliable tools for ATTR-CM diagnosis in our Mediterranean cohort. Added to clinical suspicion, can improve the diagnostic workup for ATTR-CM, irrespective of NT-proBNP values.

Bidirectional Photovoltage-Driven Oxide Transistors for Neuromorphic Visual Sensors.

Biological vision is one of the most important parts of the human perception system. However, emulating biological visuals is challenging because it requires complementary photoexcitation and photoinhibition. Here, the study presents a bidirectional photovoltage-driven neuromorphic visual sensor (BPNVS) that is constructed by monolithically integrating two perovskite solar cells (PSCs) with dual-gate ion-gel-gated oxide transistors. PSCs act as photoreceptors, converting external visual stimuli into electrical signals, whereas oxide transistors generate neuromorphic signal outputs that can be adjusted to produce positive and negative photoresponses. This device mimics the human vision system's ability to recognize colored and color-mixed patterns. The device achieves a static color recognition accuracy of 96% by utilizing the reservoir computing system for feature extraction. The BPNVS mem-reservoir chip is also proposed for handing object movement and dynamic color recognition. This work is a significant step forward in neuromorphic sensing and complex pattern recognition.

A VOx based optoelectronic memristor for application in visual perception

Abstract While biological vision systems excel at in-memory processing with low power consumption, traditional silicon-based vision chips struggle with high energy demands. This gap motivates the exploration of alternative materials for artificial intelligence applications. This paper presents a VOx-based optoelectronic synaptic memristive device. The proposed artificial synaptic device ITO/VOx /Pt mimics biological functions such as potentiation (P), depression (D), long-term memory (LTM), short-term memory (STM), and paired-pulse facilitation (PPF). The PPF index, standing at 105%, suggests a favorable pattern in short-term memory function. The device served as synapses within a spiking neural network (SNN) showing an achievable pattern classification accuracy of 88.68%, highlighting the potential of the VOx synaptic device for pattern classification tasks. The proposed VOx-based synaptic devices could be a promising platform for efficient pattern recognition and visual perception applications.

Motion Recognition Based on Deep Learning Algorithm

Advanced scientific and technological measurement methods are the basis of sports research and the progress of sports research methods. It promotes the continuous improvement of sports technology to meet athletes’ pursuit of excellent sports achievements. Therefore, the method of sports scientific research must be changed from the traditional empirical training mode to the programmed training method, from the qualitative analysis of the training effect to the nuanced analysis of the training process. Based on this, this paper uses in-depth learning to study sports motion capture. First, two-dimensional human joints are extracted based on Mask R-CNN. Then, the 3D human motion skeleton is constructed by using the binocular vision system, and the Mask R-CNN human pose estimation algorithm is optimized. On this basis, a sports motion capture system is designed, and the system’s accuracy is verified. The error of the depth information obtained by the sports capture system constructed in this paper is less than 3% in the experiment of about 2[Formula: see text]m. It has strong practicability.

Anatomical Connectivity Constrains Dynamic Functional Connectivity among Neural Systems: Implications for Cognition and Behavior

Abstract Leslie Ungerleider had a tremendous impact across many different areas of cognitive neuroscience. Her ideas and her approach, as well as her findings, will continue to impact the field for generations to come. One of the most impactful aspects of her approach was her focus on the ways that anatomical connections constrain functional communications among brain regions. Furthermore, she emphasized that changes in these functional communications, whether from lesions to the anatomical connections or temporary modulations of the efficacy of information transmission resulting from selective attention, have consequences for cognition and behavior. By necessity, this short review cannot cover the vast amount of research that contributed to or benefited from Leslie's work. Rather, we focus on one line of research that grew directly from some of Leslie's early work and her mentoring on these important concepts. This research and the many other lines of research that arose from these same origins has helped develop our understanding of the visual system, and cognitive systems more generally, as collections of highly organized, specialized, dynamic, and interacting subsystems.

A new role for excitation in the retinal direction-selective circuit.

A key feature of the receptive field of neurons in the visual system is their centre-surround antagonism, whereby the centre and the surround exhibit responses of opposite polarity. This organization is thought to enhance visual acuity, but whether and how such antagonism plays a role in more complex processing remains poorly understood. Here, we investigate the role of centre and surround receptive fields in retinal direction selectivity by exposing posterior-preferring On-Off direction-selective ganglion cells (pDSGCs) to adaptive light and recording their response to globally moving objects. We reveal that light adaptation leads to surround expansion in pDSGCs. The pDSGCs maintain their original directional tuning in the centre receptive field, but present the oppositely tuned response in their surround. Notably, although inhibition is the main substrate for retinal direction selectivity, we found that following light adaptation, both the centre- and surround-mediated responses originate from directionally tuned excitatory inputs. Multi-electrode array recordings show similar oppositely tuned responses in other DSGC subtypes. Together, these data attribute a new role for excitation in the direction-selective circuit. This excitation carries an antagonistic centre-surround property, possibly designed to sharpen the detection of motion direction in the retina. KEY POINTS: Receptive fields of direction-selective retinal ganglion cells expand asymmetrically following light adaptation. The increase in the surround receptive field generates a delayed spiking phase that is tuned to the null direction and is mediated by excitation. Following light adaptation, excitation rules the computation in the centre receptive field and is tuned to the preferred direction. GABAergic and glycinergic inputs modulate the null-tuned delayed response differentially. Null-tuned delayed spiking phases can be detected in all types of direction-selective retinal ganglion cells. Light adaptation exposes a hidden directional excitation in the circuit, which is tuned to opposite directions in the centre and surround receptive fields.

Early visual modulation and selection predict saccadic timing during visual search: An ERP study.

Saccadic eye movements, a critical aspect of real-world visual behavior, are preceded by an initial accumulation of visual information followed by the selection of a single location to move one's eyes. However, it is currently unclear how each of these stages uniquely affects saccadic timing. In this study, participants searched for a contour integration target while EEG was used to measure posterior cortical activity between search display onset and first saccade initiation. The goal was to determine whether saccade timing could be attributed to differences in early ERP amplitudes, with the P1 reflecting the magnitude of early perceptual information accumulation and the N1 reflecting the strength of selection leading to the saccadic decision. EOG was used to measure saccade timing, and trials were divided into fast, middle, and slow bins. The N1 response was smallest in the slow saccade tertile, relative to both the fast and middle tertiles, suggesting weak selection. In contrast, the P1 response was largest for this same slow saccadic tertile relative to the middle saccadic tertile, suggesting vigorous information accumulation. Therefore, delays in saccadic behavior may occur when the visual system is overwhelmed with visual input, thus increasing the time to reach a saccadic decision. These findings reconcile models of eye movement behavior which often prioritize either the impact of information accrual or selection, rather than regarding both as an integrated whole.

Effect of attention on ensemble perception: Comparison between exogenous attention, endogenous attention, and depth.

Ensemble perception is an important ability of human beings that allows one to extract summary information for scenes and environments that contain information that far exceeds the processing limit of the visual system. Although attention has been shown to bias ensemble perception, two important questions remain unclear: (1) whether direct manipulations on different types of spatial attention could produce similar effects on ensembles and (2) whether factors potentially influencing the attention distribution, such as depth perception, could evoke an indirect effect of attention on ensemble representation. This study aims to address these questions. In Experiments 1 and 2, two types of precues were used to evoke exogenous and endogenous attention, respectively, and the ensemble color perceptions were examined. We found that both exogenous and endogenous attention biased ensemble representation towards the attended items, and the latter produced a greater effect. In Experiments 3 and 4, we examined whether depth perception could affect color ensembles by indirectly influencing attention allocation in 3D space. The items were separated in two depth planes, and no explicit cues were applied. The results showed that color ensemble was biased to closer items when depth information was task relevant. This suggests that ensemble perception is naturally biased in 3D space, probably through the mechanism of attention. Computational modeling consistently showed that attention exerted a direct shift on the ensemble statistics rather than averaging the feature values over the cued and noncued items, providing evidence against an averaging process of individual perception.

A preliminary exploration of quantitative models for evaluating binocular visual perception impairments among patients diagnosed with stroke.

The objective of this study is to formulate and implement graded biological models pertaining to binocular visual perception function with the use of computer algorithms. We aim to quantitatively assess the location, severity, and degree of impairment in binocular visual perception among patients who have suffered stroke, thereby providing valuable insights into the repercussions of cerebral tissue damage on the visual system. To overcome the shortcomings of previous instruments used to assess binocular function in terms of stereoscopic effects and the challenges posed by physiological and psychological interference during examinations, this study optimized its approach by integrating polarized stereovision technology with computer graphic modeling techniques. This study employed computer models to assess binocular visual perception function in stroke patients. Computer models refer to psychophysical testing methods used to measure binocular visual perception function, including various assessment tasks such as recognizing inverted letters and assessing stereopsis during high-speed movements. The cross-into-circle test was used as a means to quantify perceptual eye position. Subsequently, the collected data was analyzed to assess the magnitude of impairment in binocular visual perception. The results of the study revealed a spectrum of binocular visual perception impairment among patients diagnosed with stroke, demonstrating discernible variations in the recognition of inverted letters and stereopsis across different movement speeds. Importantly, perceptual eye position measurements offered valuable insights into ocular misalignment. The computational models effectively quantified both the spatial distribution and severity of these identified impairments. Damage to brain tissue resulting from a stroke can give rise to notable impairments in binocular visual perception function. Graded biological models, formulated through computer algorithms, provide a systematic framework for the comprehensive evaluation and quantification of these impairments. The comprehension of the nature and extent of visual impairments observed in patients with stroke establishes a basis for the development of personalized visual perception learning methodologies. Based on such tailored approaches, we aim to facilitate the recovery of impaired visual function, thereby contributing to the broader objective of neural system rehabilitation.

When sounds come alive: animacy in the auditory sense

Despite the interest in animacy perception, few studies have considered sensory modalities other than vision. However, even everyday experience suggests that the auditory sense can also contribute to the recognition of animate beings, for example through the identification of voice-like sounds or through the perception of sounds that are the by-products of locomotion. Here we review the studies that have investigated the responses of humans and other animals to different acoustic features that may indicate the presence of a living entity, with particular attention to the neurophysiological mechanisms underlying such perception. Specifically, we have identified three different auditory animacy cues in the existing literature, namely voicelikeness, consonance, and acoustic motion. While the first two characteristics are clearly exclusive to the auditory sense and indicate the presence of an animate being capable of producing vocalizations or harmonic sounds—with the adaptive value of consonance also being exploited in musical compositions in which the musician wants to convey certain meanings—acoustic movement is, on the other hand, closely linked to the perception of animacy in the visual sense, in particular to self-propelled and biological motion stimuli. The results presented here support the existence of a multifaceted auditory sense of animacy that is shared by different distantly related species and probably represents an innate predisposition, and also suggest that the mechanisms underlying the perception of living things may all be part of an integrated network involving different sensory modalities.

ViCo: Human visual perception-based contour extraction of Chinese Tang dynasty textile patterns

The study of ancient textile patterns not only enriches historical material but also inspires modern costume art design. Recent developments in computer vision technology provide novel analytical methods for the study of ancient textile patterns. As contours are one of the most significant features in computer vision tasks, contour extraction serves as a prerequisite to accomplishing intelligent understanding and automatic design of textile patterns. In this paper, a perception-consistent contour extraction method (ViCo) that simulates the human visual system from the retina, the lateral geniculate nucleus (LGN), to the V1 area in the cortex is proposed, along with an ancient textile pattern dataset represented by TuanKe in the Chinese Tang dynasty. Experiments were conducted both qualitatively and quantitatively to verify the effectiveness of the proposed method on the TuanKe dataset. Analysis of the results implies the effectiveness of ViCo because the retinal and LGN filter enhances edge information and restrains tiny pixel changes, and the cortex simulation is direction selective. This study is among the initial efforts to employ computer vision technology for ancient textile pattern studies. The study presents a TuanKe dataset and a novel bio-inspired contour extraction method, which has significant historical research importance, fabric art design inspiration, and computer application value.

A Review of Abnormal Crowd Behavior Recognition Technology Based on Computer Vision

Abnormal crowd behavior recognition is one of the research hotspots in computer vision. Its goal is to use computer vision technology and abnormal behavior detection models to accurately perceive, predict, and intervene in potential abnormal behaviors of the crowd and monitor the status of the crowd system in public places in real time, to effectively prevent and deal with public security risks and ensure public life safety and social order. To this end, focusing on the abnormal crowd behavior recognition technology in the computer vision system, a systematic review study of its theory and cutting-edge technology is conducted. First, the crowd level and abnormal behaviors in public places are defined, and the challenges faced by abnormal crowd behavior recognition are expounded. Then, from the dimensions based on traditional methods and based on deep learning, the mainstream technologies of abnormal behavior recognition are discussed, and the design ideas, advantages, and limitations of various methods are analyzed. Next, the mainstream software tools are introduced to provide a comprehensive reference for the technical framework. Secondly, typical abnormal behavior datasets at home and abroad are sorted out, and the characteristics of these datasets are compared in detail from multiple perspectives such as scale, characteristics, and uses, and the performance indicators of different algorithms on the datasets are compared and analyzed. Finally, the full text is summarized and the future development direction of abnormal crowd behavior recognition technology is prospected.

Monitoring of visual-evoked potentials during fat packing in endoscopic resection of a giant pituitary adenoma

Background: Endoscopic transsphenoidal surgery has become a mainstay surgical approach for sellar pathologies and can effectively decompress mass effects on the optic nerves. Visual-evoked potentials (VEPs) have been utilized as an intraoperative adjunct during endoscopic transsphenoidal surgery to monitor the integrity of the optic pathways, but the data surrounding its reliability and efficacy remain heterogeneous. Case Description: An 80-year-old male underwent endoscopic transsphenoidal resection of a pituitary macroadenoma with preoperative visual deficits related to optic nerve compression. During fat packing of the resection cavity, a decrease in VEPs was noted, which seemingly improved after partial fat removal, although with paradoxically reduced VEP latencies. Despite this, the patient developed a visual field deficit postoperatively, requiring re-operation for further removal of the fat packing. Conclusion: This was a case of initially poorly formed VEPs that deteriorated and apparently improved following surgical intervention. The finding of shortened latencies of the VEPs was likely from noise contamination, creating the illusion of improved signal amplitudes. We recommend careful assessment of VEP data for baseline reproducibility, particularly in patients with pre-existing visual field deficits. Appropriate anesthetic selection is also important to reduce noise interference from the electroencephalogram.