Mechanisms Of Visual Perception Research Articles

Inspiration from Visual Ecology for Advancing Multifunctional Robotic Vision Systems: Bio-inspired Electronic Eyes and Neuromorphic Image Sensors.

In robotics, particularly for autonomous navigation and human-robot collaboration, the significance of unconventional imaging techniques and efficient data processing capabilities is paramount. The unstructured environments encountered by robots, coupled with complex missions assigned to them, present numerous challenges necessitating diverse visual functionalities, and consequently, the development of multifunctional robotic vision systems has become indispensable. Meanwhile, rich diversity inherent in animal vision systems, honed over evolutionary epochs to meet their survival demands across varied habitats, serves as a profound source of inspirations. Here, recent advancements in multifunctional robotic vision systems drawing inspiration from natural ocular structures and their visual perception mechanisms are delineated. First, unique imaging functionalities of natural eyes across terrestrial, aerial, and aquatic habitats and visual signal processing mechanism of humans are explored. Then, designs and functionalities of bio-inspired electronic eyes are explored, engineered to mimic key components and underlying optical principles of natural eyes. Furthermore, neuromorphic image sensors are discussed, emulating functional properties of synapses, neurons, and retinas and thereby enhancing accuracy and efficiency of robotic vision tasks. Next, integration examples of electronic eyes with mobile robotic/biological systems are introduced. Finally, a forward-looking outlook on the development of bio-inspired electronic eyes and neuromorphic image sensors is provided.

Content adaptive JND profile by leveraging HVS inspired channel modeling and perception oriented energy allocation optimization

The existing just noticeable difference (JND) models consider the effects of various covariates, however, they rarely account for the fusion relationship between the covariates, i.e., they lack a holistic understanding of the mechanisms of visual perception and disregarding the significant impact of energy consumption on visual perception. In fact, visual perception is no exception to the rule that nerve activities and energy supply are inextricably linked. Based on this insight, this paper proposes a novel JND estimation model employing content-adaptive energy allocation. Primarily, the information theory is applied to the visual perception system by conceptualizing human visual system (HVS) as an information communication framework. Then, leveraging the relationship between energy consumption and information perception, this paper quantitatively measures the HVS energy consumption as uniform metric to describe the complicated and heterogeneous HVS perception process, and then construct JND model by fusing low-level and Semantic-level features. Numerous simulation results verify that the proposed JND model is significantly competitive with other frontier models and highly compatible with HVS.

A Light-Weight Self-Supervised Infrared Image Perception Enhancement Method

Convolutional Neural Networks (CNNs) have achieved remarkable results in the field of infrared image enhancement. However, the research on the visual perception mechanism and the objective evaluation indicators for enhanced infrared images is still not in-depth enough. To make the subjective and objective evaluation more consistent, this paper uses a perceptual metric to evaluate the enhancement effect of infrared images. The perceptual metric mimics the early conversion process of the human visual system and uses the normalized Laplacian pyramid distance (NLPD) between the enhanced image and the original scene radiance to evaluate the image enhancement effect. Based on this, this paper designs an infrared image-enhancement algorithm that is more conducive to human visual perception. The algorithm uses a lightweight Fully Convolutional Network (FCN), with NLPD as the similarity measure, and trains the network in a self-supervised manner by minimizing the NLPD between the enhanced image and the original scene radiance to achieve infrared image enhancement. The experimental results show that the infrared image enhancement method in this paper outperforms existing methods in terms of visual perception quality, and due to the use of a lightweight network, it is also the fastest enhancement method currently.

Исследование линейности цветовосприятия человека в графическом интерфейсе систем управления

The work is aimed at studying the process of human perception of visual information in conditions of interaction with a certain interface. The authors describe experimental research of the linearity of human colour perception in the graphical interface of control systems when observing colour pictograms in a conditional interface. To analyze the mechanism operation of the visual information perception, an information model of visual perception is used. Features of human perception of visual information can be of both an individual nature associated with a specific individual, and an external nature associated with the systemic interference of the operator’s direct interaction with the interface. The paper defines the factors of the study and describes the methodology for preparing stimulus material using the Processing programming language. As a tool for conducting the experiment, a software and hardware complex recording oculomotor activity Eye-tracker SMI RED 250 is used. The importance of creating effective user interfaces for control systems of complex objects and insufficient study of adapting interfaces to the features of human perception of visual information at the perceptual level of awareness determine the relevance of the study. The means of mathematical statistics process the obtained parametric data of the template for examining the stimulus material. The results got in the experiment allow concluding that the mechanism of human visual information perception, when the colour spot brightness decreases, remains sensitive to yellow colour the longest.

Intelligent Advanced Attack Detection Technology based on Multi-modal Data Fusion

This paper proposes an adaptive Wedman intrusion detection algorithm (AID-DFS) for data fusion. Firstly, feature extraction of abnormal text detection is carried out using a BI-gated loop (Bi-GRU). Multi-branch convolutional recurrent neural network (CNN-RNN) extracts hierarchical features from abnormal images. The multi-mode dynamic fusion uses the intermodal and intramodal attention mechanisms. In this way, a joint representation of multiple modes is obtained. The visual perception mechanism is used to realize multichannel integration and strengthen the function of original information in multichannel. The experimental results show that the proposed method has 99.6% accuracy and 94.9% accuracy. Compared with other algorithms, the proposed method can improve the performance of the intrusion system by about 10.2%.

Detection Method of Molten Iron Flow Velocity at Blast Furnace Taphole Combining Visual Perception and Jet Mechanism

Detection Method of Molten Iron Flow Velocity at Blast Furnace Taphole Combining Visual Perception and Jet Mechanism

EEGVision: Reconstructing vision from human brain signals

Abstract The intricate mechanisms elucidating the interplay between human visual perceptions and cognitive processes remain elusive. Exploring and reconstructing visual stimuli from cerebral signals could help us better understand the processes by which the human brain generates visual imagery. However, the inherent complexity and significant noise in brain signals limit current efforts to reconstruct visual stimuli, resulting in low-granularity images that miss details. To address these challenges, this paper proposes EEGVision, a comprehensive framework for generating high-quality images directly from brain signals. Leveraging the recent strides in multi-modal models within the realm of deep learning, it is now feasible to bridge the gap between EEG data and visual representation. This process starts with a time-frequency fusion encoder in EEGVision, which quickly pulls out cross-domain and robust features from EEG signals. We then design two parallel pipelines to align EEG embeddings with image features at both perceptual and semantic levels. The process uses a stable diffusion-trained image-to-image pipeline that combines coarse and fine-grained data to get high-quality images back from EEG data. Both quantitative and qualitative assessments affirm that EEGVision surpasses contemporary benchmarks. This network architecture holds promise for further applications in the domain of neuroscience, aiming to unravel the genesis of human visual perception mechanisms. All code is accessible via https://github.com/AvancierGuo/EEGVision.

Underwater Image Enhancement based on Retinex Decomposition and Unsupervised Generative Adversarial Networks

Background:: Due to the difficulty of obtaining the real dataset of paired underwater images, it is urgent to build an unsupervised underwater image enhancement network. Objective:: To address the problem, a novel underwater image enhancement based on Retinex decomposition and Unsupervised Generative Adversarial Network (RUGAN) is proposed. Method:: A color correction module is proposed considering the different color distortions of underwater images. Further, considering the human visual perception mechanism, the RUGAN network, which is similar to U-Net, is constructed using the characteristics of underwater imaging and Retinex decomposition. Based on Retinex decomposition and the characteristics of underwater imaging, the RUGAN network similar to U-Net is constructed. The reflectance image and illumination image are obtained. The reflectance image with a better effect is taken as the enhancement result. Unlike the previous supervised methods, RUGAN adopts clear air images and distorted underwater images as training. RUGAN adopts the underwater image of the color correction module as pseudo-ground truth to achieve an unsupervised effect. Results:: The superiority of RUGAN network is further supported by extensive experiments that compared it with more methods. conclusion: The proposed RUGAN achieves better results both subjectively and objectively.

Brain Decoding over the MEG Signals Using Riemannian Approach and Machine Learning

Brain decoding is an emerging approach for understanding the face perception mechanism in the human brain. Face visual stimuli and perception mechanism are considered as a challenging ongoing research of the neuroscience field. In this study, face/scrambled face visual stimulations were implemented over the sixteen participants to be decoded the face or scrambled face classification using machine learning (ML) algorithms via magnetoencephalography (MEG) signals. This noninvasive and high spatial/temporal resolution signal is a neurophysiological technique which measures the magnetic fields generated by the neuronal activity of the brain. The Riemannian approach was used as a highly promising feature extraction technique. Then Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), Convolutional Neural Network (CNN) were employed as deep learning algorithms, Linear Discriminant Analysis (LDA) and Quadratic Discriminant Analysis (QDA) were implemented as shallow algorithms. The improved classification performances are very encouraging, especially for deep learning algorithms. The LSTM and GRU have achieved 92.99% and 91.66% accuracy and 0.977 and 0.973 of the area under the curve (AUC) scores, respectively. Moreover, CNN has yielded 90.62% accuracy. As our best knowledge, the improved outcomes and the usage of the deep learning on the MEG dataset signals from 16 participants are critical to expand the literature of brain decoding after visual stimuli. And this study is the first attempt with these methods in systematic comparison. Moreover, MEG-based Brain-Computer Interface (BCI) approaches may also be implemented for Internet of Things (IoT) applications, including biometric authentication, thanks to the specific stimuli of individual’s brainwaves.

Image2Brain: a cross-modality model for blind stereoscopic image quality ranking

Objective. Human beings perceive stereoscopic image quality through the cerebral visual cortex, which is a complex brain activity. As a solution, the quality of stereoscopic images can be evaluated more accurately by attempting to replicate the human perception from electroencephalogram (EEG) signals on image quality in a machine, which is different from previous stereoscopic image quality assessment methods focused only on the extraction of image features. Approach. Our proposed method is based on a novel image-to-brain (I2B) cross-modality model including a spatial-temporal EEG encoder (STEE) and an I2B deep convolutional generative adversarial network (I2B-DCGAN). Specifically, the EEG representations are first learned by STEE as real samples of I2B-DCGAN, which is designed to extract both quality and semantic features from the stereoscopic images by a semantic-guided image encoder, and utilize a generator to conditionally create the corresponding EEG features for images. Finally, the generated EEG features are classified to predict the image perceptual quality level. Main results. Extensive experimental results on the collected brain-visual multimodal stereoscopic image quality ranking database, demonstrate that the proposed I2B cross-modality model can better emulate the visual perception mechanism of the human brain and outperform the other methods by achieving an average accuracy of 95.95. Significance. The proposed method can convert the learned stereoscopic image features into brain representations without EEG signals during testing. Further experiments verify that the proposed method has good generalization ability on new datasets and the potential for practical applications.

Wafer defect recognition method based on multi-scale feature fusion.

Wafer defect recognition is an important process of chip manufacturing. As different process flows can lead to different defect types, the correct identification of defect patterns is important for recognizing manufacturing problems and fixing them in good time. To achieve high precision identification of wafer defects and improve the quality and production yield of wafers, this paper proposes a Multi-Feature Fusion Perceptual Network (MFFP-Net) inspired by human visual perception mechanisms. The MFFP-Net can process information at various scales and then aggregate it so that the next stage can abstract features from the different scales simultaneously. The proposed feature fusion module can obtain higher fine-grained and richer features to capture key texture details and avoid important information loss. The final experiments show that MFFP-Net achieves good generalized ability and state-of-the-art results on real-world dataset WM-811K, with an accuracy of 96.71%, this provides an effective way for the chip manufacturing industry to improve the yield rate.

Different aspects of visual perception are important for 12-year social functioning depending on gestational age.

Perceptual mechanisms in social functioning might promote interventions. We investigated relations between visual perception and social functioning, in preterm children. A prospective preterm cohort born in Uppsala County, Sweden, in 2004-2007 and 49 full-term controls were examined at 12 years. Aspects of visual perception, including static shapes, emotions, and time to detect biological motion, were related to social functioning and visual acuity. The preterm group comprised 25 extremely preterm children, EPT, born below 28 gestational weeks and 53 children born between 28-31 weeks. Preterm children had difficulties in perception of static shapes (p=0.004) and biological motion, p<0.001), but not in emotion perception, compared to controls. In the EPT children, poorer shape perception and lower scores on emotion perception were associated with more social problems (p=0.008) and lower visual acuity (p=0.004). Shape perception explained more variance in social functioning than emotion perception. In controls, fewer social problems were linked to faster biological motion perception (p=0.04). Static shape and biological motion perception was affected in the preterm groups. Biological motion perception was relevant for social functioning in full-term children. In EPT children, only shape perception was linked to social functioning, suggesting differential visual perception mechanisms for social deficits.

Flipping the world upside down: Using eye tracking in virtual reality to study visual search in inverted scenes.

Image inversion is a powerful tool for investigating cognitive mechanisms of visual perception. However, studies have mainly used inversion in paradigms presented on twodimensional computer screens. It remains open whether disruptive effects of inversion also hold true in more naturalistic scenarios. In our study, we used scene inversion in virtual reality in combination with eye tracking to investigate the mechanisms of repeated visual search through three-dimensional immersive indoor scenes. Scene inversion affected all gaze and head measures except fixation durations and saccade amplitudes. Our behavioral results, surprisingly, did not entirely follow as hypothesized: While search efficiency dropped significantly in inverted scenes, participants did not utilize more memory as measured by search time slopes. This indicates that despite the disruption, participants did not try to compensate the increased difficulty by using more memory. Our study highlights the importance of investigating classical experimental paradigms in more naturalistic scenarios to advance research on daily human behavior.

Models of Visual Perception in the Turkish Language Worldview

This article is devoted to the study of the mechanism of modeling visual perception in the Turkish language picture of the world. It should be noted that when modeling visual perception, a number of factors play a role, under the influence of which different models are formed. This explains the structure of a typical visual perception situation, which is presented as the basis of modeling.

A Mathematical Model for the Action Spectrum of Steady-State Pupil Size in Photopic Vision with Insight into Healthful Lighting

The pupillary light reflex, which has been seen as an important noninvasive and objective indicator of autonomic nervous system function, can be used for evaluating the impact of different lighting conditions in buildings on circadian behaviors, assessing ipRGC function in healthy and diseased retinas, and explaining luminance adaptation. However, the mechanism by which the intrinsic and extrinsic signals of ipRGCs regulate the steady-state pupil size under continuous lighting stimuli is still not clearly understood after decades of exploration. This paper presents a new experimental protocol with a large hemisphere LED screen as the stimulation device, allowing for a more realistic and comprehensive study in architectural spaces, which can potentially inform the design of lighting systems in buildings that promote healthy vision and comfort. Results reveal that both intrinsic and extrinsic signals participated in the process of regulating pupil size under continuous lighting conditions. Based on the findings, a new mathematical model was further proposed to calculate the contribution of these two signal sources to the changing intensity of melanopic radiance. The research outcomes also provide new insight into the mechanism of visual perception and adaptation and the nonvisual effect of eyes under different light conditions. Results suggest that the contribution of extrinsic signals may have been underestimated in previous studies since the extrinsic signal increases with reducing intensity in photopic conditions with lower melanopic radiance.

Leibniz and Kant

<i>Leibniz and Kant</i>

Efficient Human Vision Inspired Action Recognition using Adaptive Spatiotemporal Sampling.

Adaptive sampling that exploits the spatiotemporal redundancy in videos is critical for always-on action recognition on wearable devices with limited computing and battery resources. The commonly used fixed sampling strategy is not context-aware and may under-sample the visual content, and thus adversely impacts both computation efficiency and accuracy. Inspired by the concepts of foveal vision and pre-attentive processing from the human visual perception mechanism, we introduce a novel adaptive spatiotemporal sampling scheme for efficient action recognition. Our system pre-scans the global scene context at low-resolution and decides to skip or request high-resolution features at salient regions for further processing. We validate the system on EPIC-KITCHENS and UCF-101 (split-1) datasets for action recognition, and show that our proposed approach can greatly speed up inference with a tolerable loss of accuracy compared with those from state-of-the-art baselines. Source code is available in https://github.com/knmac/adaptive_spatiotemporal.

Mechanisms of human dynamic visual perception revealed by sequential deep neural networks

Mechanisms of human dynamic visual perception revealed by sequential deep neural networks

Attentional capture mediates the emergence and suppression of intrusive memories.

Intrusive memories hijack consciousness and their control may lead to forgetting. However, the contribution of reflexive attention to qualifying a memory signal as interfering is unknown. We used machine learning to decode the brain's electrical activity and pinpoint the otherwise hidden emergence of intrusive memories reported during a memory suppression task. Importantly, the algorithm was trained on an independent attentional model of visual activity, mimicking either the abrupt and interfering appearance of visual scenes into conscious awareness or their deliberate exploration. Intrusion of memories into conscious awareness were decoded above chance. The decoding accuracy increased when the algorithm was trained using a model of reflexive attention. Conscious detection of intrusive activity decoded from the brain signal was central to the future silencing of suppressed memories and laterforgetting. Unwanted memories require the reflexive orienting of attention and access to consciousness to be suppressed effectively by inhibitory control.

Restructuring of the Optokinetic Reflex during Metamorphosis in Pelobates fuscus Laur.

The restructuring of the gaze stabilization system in Pelobates fuscus was investigated by quantitative analysis of the optomotor response using video imaging. Gaze stabilization is an important component in the system of neural mechanisms of visual depth perception. It was shown that the optomotor response in aquatic tadpoles of P. fuscus is similar to that of fish (movement of the animal in the direction of the visual background movement and eye nystagmus consisting of a fast and a slow phase). During metamorphosis (transition from the aquatic to the terrestrial lifestyle), froglets of P. fuscus responded to the movement of the visual background by eyes and head movements. One year after metamorphosis, P. fuscus responded to movement of the visual background as adult Anura: only by head movements (a slow and a fast phase), while eye movements were absent. Possible causes of the loss of active eye movements by Anura amphibians in the process of evolution are discussed.