Sight Research Articles

The role of motor effort on the sensorimotor number system.

The integration of numerical information with motor processes has emerged as a fascinating area of investigation in both animal and human cognition. The interest in a sensorimotor number system has recently generated neurophysiological and psychophysical evidence which combine to highlight the importance of motor functions in the encoding of numerical information. Nevertheless, several key questions remain, such as the influence of non-numerical motor parameters over numerical perception. Here we tested the role of physical effort, a parameter positively correlated with the number of actions, in modulating the link between hand-actions and visual numerosity perception. Effort was manipulated during sensorimotor adaptation as well as during a new actions-estimation paradigm. The results of Experiment 1 shows that physical effort in the absence of actions (passive effort) is not sufficient to activate the sensorimotor number system, indicating that self-produced actions are instead necessary. Further experiments demonstrated that effort is marginally integrated during motor adaptation (Experiment 2) but discarded when estimating the number of self-produced hand actions (Experiment 3). Overall, the results indicate that the sensorimotor number system is largely fed by the number of discrete actions rather than the amount of effort but also indicates that effort (under specific circumstances) might be integrated. These findings provide novel insights into the sensorimotor numerical integration, paving the way for future investigations, such as on its functional role.

DeCoGAN: MVCT image denoising via coupled generative adversarial network.

Objective.In helical tomotherapy, image-guided radiotherapy employs megavoltage computed tomography (MVCT) for precise targeting. However, the high voltage of megavoltage radiation introduces substantial noise, significantly compromising MVCT image clarity. This study aims to enhance MVCT image quality using a deep learning-based denoising method.Approach.We propose an unpaired MVCT denoising network using a coupled generative adversarial network framework (DeCoGAN). Our approach assumes that a universal latent code within a shared latent space can reconstruct any given pair of images. By employing an encoder, we enforce this shared-latent space constraint, facilitating the conversion of low-quality (noisy) MVCT images into high-quality (denoised) counterparts. The network learns the joint distribution of images from both domains by leveraging samples from their respective marginal distributions, enhanced by adversarial training for effective denoising.Main Results.Compared to an analytical algorithm (BM3D) and three deep learning-based methods (RED-CNN, WGAN-VGG and CycleGAN), the proposed method excels in preserving image details and enhancing human visual perception by removing most noise and retaining structural features. Quantitative analysis demonstrates that our method achieves the highest peak signal-to-noise ratio and Structural Similarity Index Measurement values, indicating superior denoising performance.Significance.The proposed DeCoGAN method shows remarkable MVCT denoising performance, making it a promising tool in the field of radiation therapy.

Network mechanisms of ongoing brain activity's influence on conscious visual perception.

Sensory inputs enter a constantly active brain, whose state is always changing from one moment to the next. Currently, little is known about how ongoing, spontaneous brain activity participates in online task processing. We employed 7 Tesla fMRI and a threshold-level visual perception task to probe the effects of prestimulus ongoing brain activity on perceptual decision-making and conscious recognition. Prestimulus activity originating from distributed brain regions, including visual cortices and regions of the default-mode and cingulo-opercular networks, exerted a diverse set of effects on the sensitivity and criterion of conscious recognition, and categorization performance. We further elucidate the mechanisms underlying these behavioral effects, revealing how prestimulus activity modulates multiple aspects of stimulus processing in highly specific and network-dependent manners. These findings reveal heretofore unknown network mechanisms underlying ongoing brain activity's influence on conscious perception, and may hold implications for understanding the precise roles of spontaneous activity in other brain functions.

Highly Efficient Back-End-of-Line Compatible Flexible Si-Based Optical Memristive Crossbar Array for Edge Neuromorphic Physiological Signal Processing and Bionic Machine Vision.

The emergence of the Internet-of-Things is anticipated to create a vast market for what are known as smart edge devices, opening numerous opportunities across countless domains, including personalized healthcare and advanced robotics. Leveraging 3D integration, edge devices can achieve unprecedented miniaturization while simultaneously boosting processing power and minimizing energy consumption. Here, we demonstrate a back-end-of-line compatible optoelectronic synapse with a transfer learning method on health care applications, including electroencephalogram (EEG)-based seizure prediction, electromyography (EMG)-based gesture recognition, and electrocardiogram (ECG)-based arrhythmia detection. With experiments on three biomedical datasets, we observe the classification accuracy improvement for the pretrained model with 2.93% on EEG, 4.90% on ECG, and 7.92% on EMG, respectively. The optical programming property of the device enables an ultra-low power (2.8 × 10-13J) fine-tuning process and offers solutions for patient-specific issues in edge computing scenarios. Moreover, the device exhibits impressive light-sensitive characteristics that enable a range of light-triggered synaptic functions, making it promising for neuromorphic vision application. To display the benefits of these intricate synaptic properties, a 5 × 5 optoelectronic synapse array is developed, effectively simulating human visual perception and memory functions. The proposed flexible optoelectronic synapse holds immense potential for advancing the fields of neuromorphic physiological signal processing and artificial visual systems in wearable applications.

Assessment of Body Image Perception and Quality of Life among Oncology Patients

Assessment of Body Image Perception and Quality of Life among Oncology Patients

Image stitching and target perception for Autonomous Underwater Vehicle-collected side-scan sonar images

Image stitching and target perception for Autonomous Underwater Vehicle-collected side-scan sonar images

DIPA: Adversarial Attack on DNNs by Dropping Information and Pixel-Level Attack on Attention

Deep neural networks (DNNs) have shown remarkable performance across a wide range of fields, including image recognition, natural language processing, and speech processing. However, recent studies indicate that DNNs are highly vulnerable to well-crafted adversarial samples, which can cause incorrect classifications and predictions. These samples are so similar to the original ones that they are nearly undetectable by human vision, posing a significant security risk to DNNs in the real world due to the impact of adversarial attacks. Currently, the most common adversarial attack methods explicitly add adversarial perturbations to image samples, often resulting in adversarial samples that are easier to distinguish by humans. To address this issue, we are motivated to develop more effective methods for generating adversarial samples that remain undetectable to human vision. This paper proposes a pixel-level adversarial attack method based on attention mechanism and high-frequency information separation, named DIPA. Specifically, our approach involves constructing an attention suppression loss function and utilizing gradient information to identify and perturb sensitive pixels. By suppressing the model’s attention to the correct classes, the neural network is misled to focus on irrelevant classes, leading to incorrect judgments. Unlike previous studies, DIPA enhances the attack of adversarial samples by separating the imperceptible details in image samples to more effectively hide the adversarial perturbation while ensuring a higher attack success rate. Our experimental results demonstrate that under the extreme single-pixel attack scenario, DIPA achieves higher attack success rates for neural network models with various architectures. Furthermore, the visualization results and quantitative metrics illustrate that the DIPA can generate more imperceptible adversarial perturbation.

The modelling of flower colour: spectral purity or colour contrast as biologically relevant descriptors of flower colour signals for bees depending upon the perceptual task.

Flower colour is an important mediator of plant-pollinator interactions. While the reflectance of light from the flower surface and background are governed by physical properties, the perceptual interpretation of such information is generated by complex multilayered visual processing. Should quantitative modelling of flower signals strive for repeatable consistency enabled by parameter simplification, or should modelling reflect the dynamic way in which bees are known to process signals? We discuss why colour is an interpretation of spectral information by the brain of an animal. Different species, or individuals within a species, may respond differently to colour signals depending on sensory apparatus and/or individual experience. Humans and bees have different spectral ranges, but colour theory is strongly rooted in human colour perception and many principles of colour vision appear to be common. We discuss bee colour perception based on physiological, neuroanatomical and behavioural evidence to provide a pathway for modelling flower colours. We examine whether flower petals and floral guides as viewed against spectrally different backgrounds should be considered as a simple colour contrast problem or require a more dynamic consideration of how bees make perceptual decisions. We discuss that plants such as deceptive orchids may present signals to exploit bee perception, whilst many plants do provide honest signalling where perceived saturation indicates the probability of collecting nutritional rewards towards the centre of a flower that then facilitates effective pollination.

OccTr: A Two-Stage BEV Fusion Network for Temporal Object Detection

Temporal fusion approaches are critical for 3D visual perception tasks in IOV (Internet of Vehicles), but they often rely on intermediate representations without fully utilizing position information from the previous frame’s detection results, which cannot compensate for the lack of depth information in visual data. In this work, we propose a novel framework called OccTr (Occupancy Transformer) that combines two temporal cues, intermediate representation and back-end representation, via occupancy map to enhance temporal fusion in object detection task. OccTr leverages attention mechanisms to perform both intermediate and back-end temporal fusion by incorporating intermediate BEV (bird’s-eye view) features and back-end prediction results of the detector. Our two-stage framework includes occupancy map generation and cross-attention feature fusion. In stage one, the prediction results are converted into occupancy grid map format to generate back-end representation. In stage two, the high-resolution occupancy maps are fused with BEV features using cross-attention layers. This fused temporal cue provides a strong prior for the temporal detection process. Experimental results demonstrate the effectiveness of our method in improving detection performance, achieving an NDS (nuScenes Detection Score) metric score of 37.35% on the nuScenes test set, which is 1.94 points higher than the baseline.

O-260 Identifying seminiferous tubules for sperm retrieval during microdissection testicular sperm extraction (M-TESE) using deep learning

Abstract Study question Is it possible to utilize artificial intelligence (AI) in the selection of seminiferous tubules for sperm retrieval during microdissection testicular sperm extraction (M-TESE)? Summary answer AI algorithm was able to segment seminiferous tubules deemed to be good candidates for sperm retrieval with IoU of 0.746 and F1 score of 72.5%. What is known already M-TESE is an advanced surgical procedure reserved for sperm retrieval in men with non-obstructive azoospermia (NOA). Meta-analysis have shown M-TESE might be superior in relation to sperm retrieval rates and postsurgical complications when compared to conventional techniques. Limitations of the technique are the relatively long learning curve for the surgeons and the longer intraoperative time required to identify the tubules with active spermatogenesis. Convolutional neural networks (CNNs) have been shown to successfully identify areas of interest in medical images such as CT and MRI scans. The possibility of utilizing trained CNNs in M-TESE procedures has so far not been studied. Study design, size, duration This is a cross sectional study analyzing image data gathered during M-TESE procedures including the first five men enrolled in a prospective study designed specifically to test the utilization of AI in the treatment of NOA. All participants signed written informed consent. Images were collected intraoperatively with low and high resolution cameras according to predefined protocol. Participants/materials, setting, methods CNN model with down and upsampling architecture was trained using patch-based technique with corresponding labelled truth image pairs based on combination of semi-automated hue saturation value thresholding and manual segmentation of seminiferous tubules. Tubules were allocated into two different classes by the operating surgeon according to the visual perception of their morphology as type 1 (i.e. larger, opaque, good candidates for micro dissection) and type 2 (i.e. sclerotic or atrophic; unlikely to harbor active spermatogenesis). Main results and the role of chance CNN models were trained on 1809 patches with corresponding ground truth segmented images. Two algorithms were tested: the first was trained solely to detect and semantically distinguish type 1 seminiferous tubules. The second was trained to semantically segment the major anatomical structures: tunica albuginea, testicular parenchyma, blood vessels; lobules and the two types of seminiferous tubules describe above. Validation was performed on 988 image patches not previously shown to the algorithm. After grid search for optimal learning rate and augmentation the first algorithm was able to segment type 1 tubules with F1 score of 72.5% and Jaccard Index (IoU) of 0.746. Training took 856 minutes on 48 core high performance computer (HPC) equipped with single graphics processing unit (GPU). Same computational environment was used in the training of the second algorithm. This model was able to segment anatomical structures such as parenchyma (IoU=0.732) and testicular lobules (IoU=0.683) but performed sub optimally on the clinically relevant type 1 tubules (IoU=0.432). The first trained algorithm was able to segment a single high resolution surgical image in 3.4 seconds on an HPC. However, the same procedure took 5.5 minutes on standard desktop computer with 8 core Intel processor without GPU. Limitations, reasons for caution The number of patients is relatively small and therefore more training data would be needed to achieve robust performance. Training data were labelled by single surgeon limiting the generalizability and possibly biasing the selection process. The need of HPC environment for real time segmentation can limit the clinical use. Wider implications of the findings This pilot study shows the potential of AI to help in choosing suitable tubules for micro dissection, shortening the learning curve for young andrological surgeons. The access to cluster based computation is getting easier by the day and can soon allow instant intraoperative AI segmentation, thus shortening the surgical procedure. Trial registration number NA

Different Categories of Social Media Use and Their Association With Body Image Among Adolescents in 42 Countries

ObjectivesSocial media has become integrated into adolescents’ lives and influences body image perceptions. Our study examined four patterns of social media use (SMU): non-active, active, intensive, and problematic. We hypothesised that intensive SMU and problematic SMU would be associated with negative body image (negative subjective body weight) and over/underestimated body weight congruence, compared to non-active and active SMU. In addition, we expect these associations to be stronger for girls.MethodsData from 190,892 respondents aged 11, 13, and 15 from 42 countries involved in the Health Behaviour in School-aged Children study were analysed.ResultsFindings revealed higher rates of intensive or problematic SMU among adolescents who perceived themselves as too fat or too thin. Two-level regression analyses showed intensive and problematic SMU as more likely to perceive themselves as too fat or too thin than active users. The association was significant among intensive and problematic girl social media users, whereas, among boys, the relationship was only significant for problematic users.ConclusionOur findings highlight the importance of assessing SMU patterns to evaluate associations with body image.

Assessing the Relationship between Body Image Satisfaction and Physical Activity in Italian Adolescents: A Cross-Sectional Investigation

Background: Body image perception can significantly influence various aspects of adolescent lives. The study analyzed the relationship between body image satisfaction and sports participation in adolescents, examining various factors that contribute to body image concerns and their implications for sports engagement. Methods: A total of 237 schoolchildren were recruited from lower secondary Italian schools. Anthropometric characteristics were measured directly. Assessment of body image perception was performed using Body Silhouette Charts for preadolescent children. The Italian version of the International Physical Activity Questionnaire for Adolescents questionnaire was administered to assess physical activity (PA) levels at school and during leisure time in the last 7 days. Analysis of variance was used to test differences in PA levels, while multiple regression models were carried out to assess possible predictors of body dissatisfaction. Results: In total, 42.6% of children were not satisfied with their figure and 23.2% were very dissatisfied; among them, the vast majority would have liked to be thinner. The dissatisfaction and satisfaction were similar in boys and girls. The frequency of satisfaction with one’s body image was higher in children who practiced extracurricular sports compared to those who did not. The percentage of dissatisfaction with one’s body image was similar in the two groups, but the frequency of children being very dissatisfied was double in the group that does not practice extracurricular sport (31.2% vs. 17.7%). Body dissatisfaction increases with increasing body mass index in both genders but decreases in children involved in extracurricular sports. Conclusions: Encouraging teenagers to engage in regular physical activity should be a key component of therapies supporting positive body image. This study found a relationship between extracurricular sports and body image satisfaction, suggesting that physical activity protects teenagers’ body image satisfaction.

All-photonic artificial synapses based on photochromic perovskites for noncontact neuromorphic visual perception

Recently optoelectronic synapses generating light-driven electrical memories have played a vital role in the neuromorphic computing of visual perception. However, all the optoelectronic synapses demonstrate photoelectric conversion. Peripheral circuits are used for contact photocurrent measurement, leading to significant energy consumption and impeding the evolution of optical wireless communication. It is crucial to develop noncontact neuromorphic visual perception based on light-driven photonic memories. Herein, we report all-photonic artificial synapses based on photochromic perovskites. Triggered by ultraviolet and visible light pulses, cesium lead iodide bromine induces a structural disorder. Optical transmittance changes induced by the disorder last after the pulses are gone. Next, the photonic memories are propagated in the air and processed by a recurrent neural network. The accuracy of binary image recognition is instantly stabilized at 1.0, and accuracy above 0.8 after 7 epochs is achieved in the recognition of digitals from 0 to 9. The all-photonic synapses realize remote perception with zero in-situ energy consumption and enable artificial sensory systems with low-power computation, remote control, and ultrahigh propagation speed.

Alcohol Toxicity in the Developing Cerebellum

The impact of ethanol on the fetus is a significant concern as an estimated 2–5% of live births may be affected by prenatal alcohol exposure. This exposure can lead to various functional and structural abnormalities in the cerebral cortex, basal ganglia, diencephalon, and cerebellum, resulting in region-specific symptoms. The deficits relate to the motor and cognitive domains, affecting, in particular, general intelligence, attention, executive functions, language, memory, visual perception, and social skills—collectively called the fetal alcohol spectrum disorder (FASD). Recent studies suggest that damage to the developing cerebellum (in form of alcohol exposure) can impair the cortical targets of the cerebello-thalamo-cortical tract. This malfunction in the cerebello-cerebral loop optimization may be due to disruptions in the formation of the foundational elements of the internal model within the developing cerebellum. Alcohol exposure targets multiple nodes in the reciprocal loops between the cerebellum and cerebral cortex. Here, we examine the possibility that prenatal alcohol exposure damages the developing cerebellum and disrupts the connectivity within the cerebello-cerebral neuronal circuits, exacerbating FASD-related cortical dysfunctions. We propose that malfunctions between cerebellar internal model (critically involved in predictions) and cerebral regions contribute to the deficits observed in FASD. Given the major role of the cerebellum in motor, cognitive, and affective functions, we suggest that therapies should target these malfunctions to mitigate the burden of FASD. We discuss the concept of therapies oriented towards malfunctioning cerebello-cerebral loops (TOMCCLs), emphasizing anti-inflammatory strategies and treatments aimed at modulating cerebellar myelination to restore optimal and predictive cerebello-cerebral functions.

Capturing Transient Events in Series: A Review of Framing Photography

AbstractObserving transient events is of great importance for understanding fundamental principles and further controlling the related processes. To surmount the limitations of human vision, special tools are required to detect and record these transient events. Among existing approaches, framing photography stands out by its high spatiotemporal resolution with a 2D field of view and low crosstalk between adjacent frames. This review aims to summarize the technical routes of framing photography and provide a guide for choosing suitable tools for the observation of transient phenomena. The basic principles of framing photography are introduced and then an overview of the main categories by analyzing the system configurations and working principles are presented. Then, the existing devices are classified into mechanical, electrical, and optical framing photography. For each category, representative techniques and applications are discussed. Finally, a prospect for framing photography is provided.

Neuropsychological profile in tuberous sclerosis complex: a study of clinical and cognitive variables in a cohort from Brazil.

Tuberous sclerosis complex (TSC) is an autosomal dominant genetic disorder with a wide clinical, cognitive, and behavioral expressivity. To assess the neuropsychological profile of individuals clinically diagnosed with TSC and the factors that could significantly impact their cognitive development. A total of 62 individuals with ages ranging from 3 to 38 years were followed up in a tertiary attention hospital in Southern Brazil, and they were assessed using a standard battery and the Vineland Adaptive Behavior Scales, when intellectual disability was observed. History of epilepsy was found in 56 participants (90.3%), and 31 (50%) presented an intellectual disability. Among the other half of TSC individuals without intellectual disability, 8 (12.9%) presented borderline classification, 20 (32.2%) presented average scores, and 3 (4.8%) were above average. In total, 17 participants (27.4%) fulfilled the diagnostic criteria for autism spectrum disorder. The results of the multiple linear regression analysis suggested that seizures, age at diagnosis, visual perception, and general attention significantly impact cognitive performance indexes. The present study suggests that the occurrence of epileptic seizures and older age at diagnosis contribute to higher impairment in the domains of cognitive development, underlining the importance of early diagnosis and the prevention of epileptic seizures or their rapid control. The development of attentional skills, visual perception, and executive functions must be followed up.

Neuronal Stability, Volumetric Changes, and Decrease in GFAP Expression of Marmoset (Callithrix jacchus) Subcortical Visual Nuclei During Aging.

The physiological aging process is well known for functional decline in visual abilities. Among the components of the visual system, the dorsal lateral geniculate nucleus (DLG) and superior colliculus (SC) provide a good model for aging investigations, as these structures constitute the main visual pathways for retinal inputs reaching the visual cortex. However, there are limited data available on quantitative morphological and neurochemical aspects in DLG and SC across lifespan. Here, we used optical density to determine immunoexpression of glial fibrillary acidic protein (GFAP) and design-based stereological probes to estimate the neuronal number, total volume, and layer volume of the DLG and SC in marmosets (Callithrix jacchus), ranging from 36 to 143 months of age. Our results revealed an age-related increase in total volume and layer volume of the DLG, with an overall stability in SC volume. Furthermore, a stable neuronal number was demonstrated in DLG and superficial layers of SC (SCv). A decrease in GFAP immunoexpression was observed in both visual centers. The results indicate region-specific variability in volumetric parameter, possibly attributed to structural plastic events in response to inflammation and compensatory mechanisms at the cellular and subcellular level. Additionally, the DLG and SCv seem to be less vulnerable to aging effects in terms of neuronal number. The neuropeptidergic data suggest that reduced GFAP expression may reflect morphological atrophy in the astroglial cells. This study contributes to updating the current understanding of aging effects in the visual system and stablishes a crucial foundation for future research on visual perception throughout the aging process.

The Domains of Aesthetics and Perception Theories: A Review Relevant to Practice-based Doctoral Theses in the Visual Arts

Abstract Every doctoral thesis requires contextualization within its specific discipline's theoretical bases. For a visual arts practice-based thesis, the relevant bases include those of aesthetics and visual perception. This article reviews a Western history of the domain of visual aesthetic theory, addressing both the analytical philosophical efforts to define art and the continental approaches, which construe art as social construction. It then reviews a third, normative stance that foregrounds cognitive value before definition or sociological context—an aesthetic cognitivist position, art practice as a means of acquiring and sharing experiential knowledge and understanding. Contemporary concerns about environmental aesthetics and somaesthetics are also addressed in the Introduction. Aesthetic theories are related to visual perception theories as an aid to fulfilling the scholarly requirements for a practice-based thesis.

Video Coding for Machines: Compact Visual Representation Compression for Intelligent Collaborative Analytics.

As an emerging research practice leveraging recent advanced AI techniques, e.g. deep models based prediction and generation, Video Coding for Machines (VCM) is committed to bridging to an extent separate research tracks of video/image compression and feature compression, and attempts to optimize compactness and efficiency jointly from a unified perspective of high accuracy machine vision and full fidelity human vision. With the rapid advances of deep feature representation and visual data compression in mind, in this paper, we summarize VCM methodology and philosophy based on existing academia and industrial efforts. The development of VCM follows a general rate-distortion optimization, and the categorization of key modules or techniques is established including feature-assisted coding, scalable coding, intermediate feature compression/optimization, and machine vision targeted codec, from broader perspectives of vision tasks, analytics resources, etc. From previous works, it is demonstrated that, although existing works attempt to reveal the nature of scalable representation in bits when dealing with machine and human vision tasks, there remains a rare study in the generality of low bit rate representation, and accordingly how to support a variety of visual analytic tasks. Therefore, we investigate a novel visual information compression for the analytics taxonomy problem to strengthen the capability of compact visual representations extracted from multiple tasks for visual analytics. A new perspective of task relationships versus compression is revisited. By keeping in mind the transferability among different machine vision tasks (e.g. high-level semantic and mid-level geometry-related), we aim to support multiple tasks jointly at low bit rates. In particular, to narrow the dimensionality gap between neural network generated features extracted from pixels and a variety of machine vision features/labels (e.g. scene class, segmentation labels), a codebook hyperprior is designed to compress the neural network-generated features. As demonstrated in our experiments, this new hyperprior model is expected to improve feature compression efficiency by estimating the signal entropy more accurately, which enables further investigation of the granularity of abstracting compact features among different tasks.

VNVC: A Versatile Neural Video Coding Framework for Efficient Human-Machine Vision.

Almost all digital videos are coded into compact representations before being transmitted. Such compact representations need to be decoded back to pixels before being displayed to humans and - as usual - before being enhanced/analyzed by machine vision algorithms. Intuitively, it is more efficient to enhance/analyze the coded representations directly without decoding them into pixels. Therefore, we propose a versatile neural video coding (VNVC) framework, which targets learning compact representations to support both reconstruction and direct enhancement/analysis, thereby being versatile for both human and machine vision. Our VNVC framework has a feature-based compression loop. In the loop, one frame is encoded into compact representations and decoded to an intermediate feature that is obtained before performing reconstruction. The intermediate feature can be used as reference in motion compensation and motion estimation through feature-based temporal context mining and cross-domain motion encoder-decoder to compress the following frames. The intermediate feature is directly fed into video reconstruction, video enhancement, and video analysis networks to evaluate its effectiveness. The evaluation shows that our framework with the intermediate feature achieves high compression efficiency for video reconstruction and satisfactory task performances with lower complexities.