Human Perception Mechanism Research Articles

Quantification of Visual Attention by Using Eye-Tracking Technology for Soundscape Assessment Through Physiological Response.

Because soundscapes affect human health and comfort, methodologies for evaluating them through physiological responses have attracted considerable attention. In this study, we proposed a novel method for evaluating visual attention by using eye-tracking technology to objectively assess soundscape perception. The study incorporated questionnaire surveys and physiological measurements focusing on visual attention responses. Results from the questionnaire indicated that perceptions of vehicles and the sky were 6% and 26% more sensitive, respectively, whereas perceptions of vegetation, based on physiological responses, were approximately 3% to 50% more sensitive. The soundscape quality prediction model indicates that the proposed methodology can complement conventional questionnaire-based models and provide a nuanced interpretation of eventfulness relationships. Additionally, the visual attention quantification technique enhanced the restoration responses of questionnaire-based methods by 1-2%. The results of this study are significant because the study introduces a novel methodology for quantifying visual attention, which can be used as a supplementary tool for physiological responses in soundscape research. The proposed method can uncover novel mechanisms of human perception of soundscapes that may not be captured by questionnaires, providing insights for future research in soundscape evaluation through physiological measurements.

Sense of embodiment with synchronized avatar during walking in mixed reality.

Gait guidance systems that synchronize the gait rhythm with an avatar in a mixed reality (MR) environment are attracting attention owing to their rehabilitation applications. More effective gait guidance can be achieved by changing body sensations for the sense of embodiment (SoE), which refers to the feeling of owning, controlling, and being inside a body in MR. This study investigated full-body synchronous motion between a human and a virtual avatar to enhance the SoE in walking with actual position changes in the real world. The full-body motion and gait rhythm were measured using body-worn inertial measurement units and a visual avatar was provided through a transparent head-mounted display. The results showed that the SoE of the participants was enhanced under higher synchronization conditions. In addition, questionnaire results showed that the SoE in the synchronous condition was significantly higher than that in the asynchronous condition, and the SoE in the self-avatar condition was significantly higher than that in the other-avatar condition. This indicates that a higher synchronization level with the appearance of an avatar leads to a stronger SoE in the human perception mechanism, which is important for potential application in medical or other fields.

Исследование линейности цветовосприятия человека в графическом интерфейсе систем управления

The work is aimed at studying the process of human perception of visual information in conditions of interaction with a certain interface. The authors describe experimental research of the linearity of human colour perception in the graphical interface of control systems when observing colour pictograms in a conditional interface. To analyze the mechanism operation of the visual information perception, an information model of visual perception is used. Features of human perception of visual information can be of both an individual nature associated with a specific individual, and an external nature associated with the systemic interference of the operator’s direct interaction with the interface. The paper defines the factors of the study and describes the methodology for preparing stimulus material using the Processing programming language. As a tool for conducting the experiment, a software and hardware complex recording oculomotor activity Eye-tracker SMI RED 250 is used. The importance of creating effective user interfaces for control systems of complex objects and insufficient study of adapting interfaces to the features of human perception of visual information at the perceptual level of awareness determine the relevance of the study. The means of mathematical statistics process the obtained parametric data of the template for examining the stimulus material. The results got in the experiment allow concluding that the mechanism of human visual information perception, when the colour spot brightness decreases, remains sensitive to yellow colour the longest.

Depth perception optimization of mixed reality simulation systems based on multiple‐cue fusion

AbstractMixed reality technology can be applied to simulation training, improve surgical performance, enhance 3D game experience, and so on, attracting extensive attention of researchers. The perception of the user about head‐mounted display MR using HoloLens is critical, especially for precision applications such as virtual hoisting. Designing and adding appropriate depth cues in MR scenes is an effective way to improve users' depth perception. In this study, taking virtual hoisting training system as an example, the depth perception strategy of multi‐cue fusion is proposed to improve the perception effect. Based on the mechanism of human depth perception, five kinds of depth cues are designed. The depth perception effect of adding single cue is studied by perceptual matching experiment. Based on the principle of fuzzy clustering, a multiple‐cue comprehensive depth optimization strategy on viewing distance scale is proposed. Finally, the perceptual matching results demonstrate the effectiveness of the multi‐cue fusion strategy, and the average error is reduced by 20.68% compared with the single‐cue strategy, which can significantly improve the spatial depth perception. This research can provide a reference for improving users' depth perception in interactive MR simulation systems.

Memristor-based adaptive neuromorphic perception in unstructured environments

Efficient operation of control systems in robotics or autonomous driving targeting real-world navigation scenarios requires perception methods that allow them to understand and adapt to unstructured environments with good accuracy, adaptation, and generality, similar to humans. To address this need, we present a memristor-based differential neuromorphic computing, perceptual signal processing, and online adaptation method providing neuromorphic style adaptation to external sensory stimuli. The adaptation ability and generality of this method are confirmed in two application scenarios: object grasping and autonomous driving. In the former, a robot hand realizes safe and stable grasping through fast ( ~ 1 ms) adaptation based on the tactile object features with a single memristor. In the latter, decision-making information of 10 unstructured environments in autonomous driving is extracted with an accuracy of 94% with a 40×25 memristor array. By mimicking human low-level perception mechanisms, the electronic neuromorphic circuit-based method achieves real-time adaptation and high-level reactions to unstructured environments.

Thresholds and mechanisms of human magnetophosphene perception induced by low frequency sinusoidal magnetic fields

BackgroundVirtually everyone is exposed to power-frequency MF (50/60 Hz), inducing in our body electric fields and currents, potentially modulating brain function. MF-induced electric fields within the central nervous system can generate flickering visual perceptions (magnetophosphenes), which form the basis of international MF exposure guidelines and recommendations protecting workers and the general public. However, magnetophosphene perception thresholds were estimated 40 years ago in a small, unreplicated study with significant uncertainties and leaving open the question of the involved interaction site. MethodsWe used a stimulation modality termed transcranial alternating magnetic stimulation (tAMS), delivering in situ sinusoidal electric fields comparable to transcranial alternating current stimulation (tACS). Magnetophosphene perception was quantified in 81 volunteers exposed to MF (eye or occipital exposure) between 0 and 50 mT at frequencies of 20, 50, 60 and 100 Hz. ResultsReliable magnetophosphene perception was induced with tAMS without any scalp sensation, a major advantage as compared to tACS. Frequency-dependent thresholds were quantified using binary logistic regressions hence allowing to establish condition dependent probabilities of perception. Results support an interaction between induced current density and retinal rod cells. ConclusionBeyond fundamental and immediate implications for international safety guidelines, and for identifying the interaction site underlying phosphene perception (ubiquitous in tACS experiments), our results support exploring the potential of tAMS for the differential diagnosis of retinal disorders and neuromodulation therapy.

“Reading”, “listening” and sound, light environment

Sound is always an important medium for passing on knowledge and skills as well as for communicating information. Reading accompanies the use of the brain in the silent reading of words, an act that transforms textual information into verbal information. People can “listen” and “read” in a dark or low light environment. Learning and thinking in a quiet, dark environment is not only conducive to the formation of innovative thinking, but also the inheritance of ancient ideas of health and wisdom. Modern disciplines follow the human perception mechanism and advocate concepts of “Quiet Area” (in soundscape discipline) and “Dark Night Sky” (in lightscape discipline). It is in accordance with the ancients preference of thinking and learning in a “quiet, dark environment”, which is consistent with the law of the matter and the origin of things. This also reflects the integration of human life wisdom and modern science.

EEGVision: Reconstructing vision from human brain signals

Abstract The intricate mechanisms elucidating the interplay between human visual perceptions and cognitive processes remain elusive. Exploring and reconstructing visual stimuli from cerebral signals could help us better understand the processes by which the human brain generates visual imagery. However, the inherent complexity and significant noise in brain signals limit current efforts to reconstruct visual stimuli, resulting in low-granularity images that miss details. To address these challenges, this paper proposes EEGVision, a comprehensive framework for generating high-quality images directly from brain signals. Leveraging the recent strides in multi-modal models within the realm of deep learning, it is now feasible to bridge the gap between EEG data and visual representation. This process starts with a time-frequency fusion encoder in EEGVision, which quickly pulls out cross-domain and robust features from EEG signals. We then design two parallel pipelines to align EEG embeddings with image features at both perceptual and semantic levels. The process uses a stable diffusion-trained image-to-image pipeline that combines coarse and fine-grained data to get high-quality images back from EEG data. Both quantitative and qualitative assessments affirm that EEGVision surpasses contemporary benchmarks. This network architecture holds promise for further applications in the domain of neuroscience, aiming to unravel the genesis of human visual perception mechanisms. All code is accessible via https://github.com/AvancierGuo/EEGVision.

Perception-Aware Texture Similarity Prediction.

Texture similarity plays important roles in texture analysis and material recognition. However, perceptually-consistent fine-grained texture similarity prediction is still challenging. The discrepancy between the texture similarity data obtained using algorithms and human visual perception has been demonstrated. This dilemma is normally attributed to the texture representation and similarity metric utilised by the algorithms, which are inconsistent with human perception. To address this challenge, we introduce a Perception-Aware Texture Similarity Prediction Network (PATSP-Net). This network comprises a Bilinear Lateral Attention Transformer network (BiLAViT) and a novel loss function, namely, RSLoss. The BiLAViT contains a Siamese Feature Extraction Subnetwork (SFEN) and a Metric Learning Subnetwork (MLN), designed on top of the mechanisms of human perception. On the other hand, the RSLoss measures both the ranking and the scaling differences. To our knowledge, either the BiLAViT or the RSLoss has not been explored for texture similarity tasks. The PATSP-Net performs better than, or at least comparably to, its counterparts on three data sets for different fine-grained texture similarity prediction tasks. We believe that this promising result should be due to the joint utilization of the BiLAViT and RSLoss, which is able to learn the perception-aware texture representation and similarity metric.

Pupil responses to colorfulness are selectively reduced in healthy older adults

The alignment between visual pathway signaling and pupil dynamics offers a promising non-invasive method to further illuminate the mechanisms of human color perception. However, only limited research has been done in this area and the effects of healthy aging on pupil responses to the different color components have not been studied yet. Here we aim to address this by modelling the effects of color lightness and chroma (colorfulness) on pupil responses in young and older adults, in a closely controlled passive viewing experiment with 26 broad-spectrum digital color fields. We show that pupil responses to color lightness and chroma are independent from each other in both young and older adults. Pupil responses to color lightness levels are unaffected by healthy aging, when correcting for smaller baseline pupil sizes in older adults. Older adults exhibit weaker pupil responses to chroma increases, predominantly along the Green–Magenta axis, while relatively sparing the Blue–Yellow axis. Our findings complement behavioral studies in providing physiological evidence that colors fade with age, with implications for color-based applications and interventions both in healthy aging and later-life neurodegenerative disorders.

Deepfake detection via inter-frame inconsistency recomposition and enhancement

Due to the remarkable progress in face manipulation technology, malicious applications of these technologies may pose a great threat to the social stability. Therefore, it is essential to carry out the research of deepfake detection. In this paper, we assumed that the illumination on frames that skip a certain space is basically consistent in real videos, but tends to be inconsistent in fake videos. From this point, a network which contains a learnable Image Decomposition Module (IDM) and multi-level feature enhancement is proposed. IDM decomposes frames into illumination and reflection, and frame recomposition is followed to highlight the frame-level illumination inconsistency. Multi-level feature enhancement is proposed to enhance the illumination inconsistency at feature level. In addition, considering the computational complexity and human vision perception mechanism, we train the network in logarithm domain. Experimental results show that the proposed method is effective and superior compared with other state-of-the-art deepfake detection methods on mainstream deepfake datasets.

Modeling of Human Motion Perception Mechanism: A Simulation based on Deep Neural Network and Attention Transformer

Modeling of Human Motion Perception Mechanism: A Simulation based on Deep Neural Network and Attention Transformer

Wafer defect recognition method based on multi-scale feature fusion.

Wafer defect recognition is an important process of chip manufacturing. As different process flows can lead to different defect types, the correct identification of defect patterns is important for recognizing manufacturing problems and fixing them in good time. To achieve high precision identification of wafer defects and improve the quality and production yield of wafers, this paper proposes a Multi-Feature Fusion Perceptual Network (MFFP-Net) inspired by human visual perception mechanisms. The MFFP-Net can process information at various scales and then aggregate it so that the next stage can abstract features from the different scales simultaneously. The proposed feature fusion module can obtain higher fine-grained and richer features to capture key texture details and avoid important information loss. The final experiments show that MFFP-Net achieves good generalized ability and state-of-the-art results on real-world dataset WM-811K, with an accuracy of 96.71%, this provides an effective way for the chip manufacturing industry to improve the yield rate.

A Comprehensive Survey on Video Saliency Detection With Auditory Information: The Audio-Visual Consistency Perceptual is the Key!

Video saliency detection (VSD) aims at fast locating the most attractive objects/things/patterns in a given video clip. Existing VSD-related works have mainly relied on the visual system but paid less attention to the audio aspect. In contrast, our audio system is the most vital complementary part of our visual system. Also, audio-visual saliency detection (AVSD), one of the most representative research topics for mimicking human perceptual mechanisms, is currently in its infancy, and none of the existing survey papers have touched on it, especially from the perspective of saliency detection. Thus, the ultimate goal of this paper is to provide an extensive review to bridge the gap between audio-visual fusion and saliency detection. In addition, as another highlight of this review, we have provided a deep insight into key factors that could directly determine AVSD deep models’ performances. We claim that the audio-visual consistency degree (AVC) — a long-overlooked issue, can directly influence the effectiveness of using audio to benefit its visual counterpart when performing saliency detection. Moreover, to make the AVC issue more practical and valuable for future followers, we have newly equipped almost all existing publicly available AVSD datasets with additional frame-wise AVC labels. Based on these upgraded datasets, we have conducted extensive quantitative evaluations to ground our claim on the importance of AVC in the AVSD task. In a word, our ideas and new sets serve as a convenient platform with preliminaries and guidelines, all of which can potentially facilitate future works in further promoting state-of-the-art (SOTA) performance.

ВОСПРИЯТИЕ СИММЕТРИЧНОСТИ ПРАВИЛЬНО ОРИЕНТИРОВАННОГО И ИНВЕРТИРОВАННОГО ЛИЦА

Background. X e uniqueness of human face perception mechanisms and their variability has become a subject for extensive investigation and wide discussion in the recent decades. While now the concept that human face is perceived as a whole is taken as a fact, the same cannot be said about the structure and methods to measure holistic processing. In the present study the factor of symmetry is investigated as one of the possible aspects of conW gural processing of facial perception. Objective. X e aim of the study was to apply the inversion method to the investigation of how distortion of facial conW guration perception impacts the rating of its symmetry/asymmetry Sample. 54 participants (24 female and 30 male) aged 18 to 68 (mean age 26) took part in the study. Method. An intra-individual experimental design was used. Participants rated asymmetry of normally oriented and inverted face-stimuli on a W ve-point scale. To set the symmetry level, leY eye and/or mouth regions were rotated around their W ducial points, there were three rotation angles for each feature region. Zero asymmetry faces were created by blending halves of normal and mirror images of the sample face-stimulus. X e mean asymmetry subjective ratings were submitted to repeated measures analysis of variance and paired samples t-test. Results. X e study revealed signiW cant diZ erences in the asymmetry ratings of upright and inverted faces: normal upright faces with rotated features were rated as more asymmetric compared to inverted. X us, we may conclude that symmetry perception is aZ ected by inversion. It was also revealed, that the eZ ect of the eye region and the lips region rotation were not equivalent: the higher the level rotation of the eye area was the more signiW cant the diZ erence between the rates of upright and inverted faces was, while rotation of the lips area did not result in higher diZ erence between the rates. Conclusion.X e observed face-inversion eZ ect in the perception of symmetry in facial stimuli suggests that (a)symmetry is a component of holistic (conW gurative) processing of human face perception. At the same time, the contributions of individual features to the overall facial conW guration may diZ er.

Loudspeaker Abnormal Sound Classification Using Auditory Perception Weighted by Energy Entropy

In order to improve the classification accuracy of loudspeaker abnormal sounds, this paper proposes a method based on time-varying specific loudness weighted by energy entropy and principal component analysis. This method simulates human auditory perception mechanism to process loudspeaker sound response signal to obtain more effective features. The human hearing system can be divided into many parallel and functionally independent conduction pathways according to frequency. The energy of the acoustic signal in each pathway can be discerned. Therefore, the time-varying specific loudness is calculated firstly to build the quantitative correlation of loudspeaker’s acoustic response signals and human hearing sensations. Each sub band loudness is weighted by energy entropy to highlight the acoustic strength variation with time and frequency. Then, important features are extracted by two dimensional-principal component analysis (2D-PCA). Finally, the whale optimization algorithm-least squares support vector machine (WOA-LSSVM) is adopted for classification. Visual analysis of the extracted features shows that this method can extract loudspeaker response signal features with better discriminability. Classification experimental results show that the average accuracy of this method reached 98.4%, which is higher than the classification method based on traditional time-frequency domain statistical features. The loudspeaker abnormal sound classification method in this paper simulates human auditory perception to extract features and is able to improve classification accuracy and automation effectively.

Mechanisms of human dynamic visual perception revealed by sequential deep neural networks

Mechanisms of human dynamic visual perception revealed by sequential deep neural networks

Machine-Readable Law, a Statute and a Human-Readable (Russian) Language

The paper defines the role played by the human-readable, namely Russian, language in the technologies of machine reading of law and statutes. The author analyzes the complexity factor of interpretation according to the rules of formal logic of conceptual units and terminology of law and statutes, which may be an obstacle to the processes of machine-reading of law, as well as the formation of machine-readable law. The author draws a number of conclusions regarding the methodology of machine-reading technologies and machine-drafting technologies, in particular that these processes should be formed and implemented in a single legal-ideological, legal-aesthetic, ethical, stylistic, methodological, technological, etc. paradigm and in the context of processes common to law and statutes: «the formation of law — lawmaking — implementation of law.» If, in the presence of all critical assessments of the «quantum theory of consciousness», we assume that there are features of quantum laws in the human psyche, then it should be stated that human thinking, world perception, and therefore legal perception are not limited to formal logical, computational frameworks, but include some «non-computational,» irrational mechanisms. Anyway, it seems obvious that the activity of human consciousness goes beyond purely algorithmic actions. Trying to «fit» the mechanisms of human perception of law and statutes into some logically consistent schemes means a priori incorrect reflection of law and statutes in such «schemes.» Among other things, the author comes to the conclusion that machine-reading processes should be carried out on the basis of the widest possible semiotic paradigm and based on the super-task of constructing a holographic, «quantum-like» model of law. At the same time, the Russian language, as the language of the synthetic system, is quite acceptable for use in machine-reading technologies and, perhaps, even more preferable than the analytical languages.

Tactile Perception for Surgical Status Recognition in Robot-Assisted Laminectomy

In laminectomy, the laminas above the spinal nerves need to be removed by milling. This process is very dangerous because of the special lesion, and surgeons usually control the milling process by sensing the surgical status. However, in robot-assisted laminectomy, safety monitoring will be difficult because the surgeon is not directly involved. Based on the human tactile perception mechanism, this article proposes a surgical status perception method using the acceleration sensor and force sensor installed on the robot. The Sinc-convolutional layer is introduced to process high-frequency vibration signals, and the obtained features are processed together with the averaged force signals using a 1-D convolutional network. The proposed method classifies the surgical status into five categories, and outputs the burr clogging probability. Experiments on animal bones show the effectiveness of the proposed model. In addition, it has also been proved that the fusion of the two tactile signals can significantly improve the status recognition accuracy under changing milling parameters.

Nonlinear Tactile Estimation Model Based on Perceptibility of Mechanoreceptors Improves Quantitative Tactile Sensing.

Tactile sensing has attracted significant attention as a tactile quantitative evaluation method because the tactile sensation is an important factor while evaluating consumer products. Although the human tactile perception mechanism has nonlinearity, previous studies have often developed linear regression models. In contrast, this study proposes a nonlinear tactile estimation model that can estimate sensory evaluation scores from physical measurements. We extracted features from the vibration data obtained by a tactile sensor based on the perceptibility of mechanoreceptors. In parallel, a sensory evaluation test was conducted using 10 evaluation words. Then, the relationship between the extracted features and the tactile evaluation results was modeled using linear/nonlinear regressions. The best model was concluded by comparing the mean squared error between the model predictions and the actual values. The results imply that there are multiple evaluation words suitable for adopting nonlinear regression models, and the average error was 43.8% smaller than that of building only linear regression models.