Human Visual System Processes Research Articles

PS-Net: Human perception-guided segmentation network for EM cell membrane.

Cell membrane segmentation in electron microscopy (EM) images is a crucial step in EM image processing. However, while popular approaches have achieved performance comparable to that of humans on low-resolution EM datasets, they have shown limited success when applied to high-resolution EM datasets. The human visual system, on the other hand, displays consistently excellent performance on both low and high resolutions. To better understand this limitation, we conducted eye movement and perceptual consistency experiments. Our data showed that human observers are more sensitive to the structure of the membrane while tolerating misalignment, contrary to commonly used evaluation criteria. Additionally, our results indicated that the human visual system processes images in both global-local and coarse-to-fine manners. Based on these observations, we propose a computational framework for membrane segmentation that incorporates these characteristics of human perception. This framework includes a novel evaluation metric, the perceptual Hausdorff distance (PHD), and an end-to-end network called the PHD-guided segmentation network (PS-Net) that is trained using adaptively tuned PHD loss functions and a multiscale architecture. Our subjective experiments showed that the PHD metric is more consistent with human perception than other criteria, and our proposed PS-Net outperformed state-of-the-art methods on both low and high resolution EM image datasets as well as other natural image datasets. The code and dataset can be found at https://github.com/EmmaSRH/PS-Net. Supplementary Information for this article is available online.

Flexplot: Graphically-based data analysis.

The human visual processing system has enormous bandwidth, able to interpret vast amounts of data in fractions of a second (Otten et al., 2015). Despite this amazing ability, there is a troubling lack of graphics in scientific literature (Healy & Moody, 2014), and the graphics most traditionally used tend to bias perception in unintentional ways (Weissgerber et al., 2015). I suspect the reason for the underuse and misuse of graphics is because sound graphs are difficult to produce with existing software (Wainer, 2010). While ggplot2 allows immense flexibility in creating graphics, its learning curve is quite steep, and even basic graphics require multiple lines of code. flexplot is an R package that aims to address these issues by providing a formula-based suite of tools that simplifies and automates much of the graphical decision-making. Additionally, flexplot pairs well with statistical modeling, making it easy for researchers to produce graphs that map onto statistical procedures. With one-line functions, users can visualize bivariate statistical models (e.g., scatterplots for regression, beeswarm plots for ANOVA/t-tests), multivariate statistical models (e.g., ANCOVA and multiple regression), and even more sophisticated models like multilevel models and logistic regressions. Further, this package utilizes old tools (e.g., added variable plots and coplots) as well as introduces new tools for complex visualizations, including ghost lines and point sampling. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Temporal characteristics of global form perception in translational and circular Glass patterns

The human visual system is continuously exposed to a natural environment with static and moving objects that the visual system needs to continuously integrate and process. Glass patterns (GPs) are a class of visual stimuli widely used to study how the human visual system processes and integrates form and motion signals. GPs are made of pairs of dots that elicit a strong percept of global form. A rapid succession of unique frames originates dynamic GPs. Previous psychophysical studies showed that dynamic translational GPs are easier to detect than the static version because of the spatial summation across the unique frames composing the pattern. However, it is not clear whether the same mechanism is involved in dynamic circular GPs. In the present study, we psychophysically investigated the role of the temporal and spatial summation in the perception of both translational and circular GPs. We manipulated the number of unique frames in dynamic GPs and the update rate of the frames presentation. The results suggest that spatial and temporal summation across unique frames takes place for both translational and circular GPs. Moreover, the number of unique frames and the pattern update rate equally influence the discrimination thresholds of translational and circular GPs. These results show that form and motion integration is likely to be processed similarly for translational and circular GPs.

The effect of flanker category on crowding is modulated by processing time

When the human visual system processes complex scenes, crowding is considered a bottleneck in interpreting visual information. Several studies have reported that visual crowding seems to occur in the early stage of visual possessing. After that, extensive evidence suggested that high-level visual information has a major effect on crowding, which showed substantial interest in the impact of high-level visual information on crowding. However, the temporal property of high-level crowding is not well understood. Here, we performed a series of behavioural experiments to show how flanker category information and exposure time affected crowding's intensity. We found that as the category's effect becomes stronger, the intensity of crowding will be reduced in longer exposure time. The peak of this effect occurs at the time window of 100 ms-150 ms after the stimulus appears. We propose that exposure time may modulate high-level information processing depth, which can adjust multi-level crowding.

The Spatiotemporal Power Spectrum of Natural Human Vision

When engaging in natural tasks, the human visual system processes a highly dynamic visual data stream. The retina, performing the very first steps in this processing, is thought to be adapted to take advantage of low-level signal regularities, such as the autocorrelation function or power spectrum, to produce a more efficient encoding of the data (Atick & Redlich, 1992). Previous work examined the joint spatio-temporal power spectrum of handheld camera videos and Hollywood movies, showing that power falls as an inverse power-law function of spatial and temporal frequency, with an inseparable relationship (Dong & Attick, 1995). However these data are far from a true characterization of a day in the life of the retina due to body, head and eye motion. In addition, the distribution of natural tasks will influence the statistics of this signal. Here, we aim to characterize these statistics of natural vision using a custom device that consists of a head-mounted eye tracker coupled with high frame-rate world cameras and orientation sensors. Using video data captured from this setup, we analyze the joint spatiotemporal power spectrum for three conditions: 1) a static camera viewing a natural task being performed, 2) a head mounted camera worn by a subject engaged in a natural task, and 3) videos simulating the dynamic retinal image, created by overlaying the subject's eye motions on the head-mounted camera video stream. Results suggest that compared to a static camera, body and head motion have the effect of boosting high temporal frequencies. Eye motion enhances this effect, particularly for mid to high spatial frequencies, causing this portion to deviate from the power law and become nearly flat. These data will be important for developing efficient coding models relevant to natural vision.

Variational Models for Color Image Correction Inspired by Visual Perception and Neuroscience

Reproducing the perception of a real-world scene on a display device is a very challenging task which requires the understanding of the camera processing pipeline, the display process, and the way the human visual system processes the light it captures. Mathematical models based on psychophysical and physiological laws on color vision, named Retinex, provide efficient tools to handle degradations produced during the camera processing pipeline like the reduction of the contrast. In particular, Batard and Bertalmío (in J Math Imaging Vis 60(6):849–881, 2018) described some psychophysical laws on brightness perception as covariant derivatives, included them into a variational model, and observed that the quality of the color image correction is correlated with the accuracy of the vision model it includes. Based on this observation, we postulate that this model can be improved by including more accurate data on vision with a special attention on visual neuroscience here. Then, inspired by the presence of neurons responding to different visual attributes in the area V1 of the visual cortex as orientation, color or movement, to name a few, and horizontal connections modeling the interactions between those neurons, we construct two variational models to process both local (edges, textures) and global (contrast) features. This is an improvement with respect to the model of Batard and Bertalmío as the latter cannot process local and global features independently and simultaneously. Finally, we conduct experiments on color images which corroborate the improvement provided by the new models.

Unraveling nonlinear electrophysiologic processes in the human visual system with full dimension spectral analysis

Natural sensory signals have nonlinear structures dynamically composed of the carrier frequencies and the variation of the amplitude (i.e., envelope). How the human brain processes the envelope information is still poorly understood, largely due to the conventional analysis failing to quantify it directly. Here, we used a recently developed method, Holo-Hilbert spectral analysis, and steady-state visually evoked potential collected using electroencephalography (EEG) recordings to investigate how the human visual system processes the envelope of amplitude-modulated signals, in this case with a 14 Hz carrier and a 2 Hz envelope. The EEG results demonstrated that in addition to the fundamental stimulus frequencies, 4 Hz amplitude modulation residing in 14 Hz carrier and a broad range of carrier frequencies covering from 8 to 32 Hz modulated by 2 Hz amplitude modulation are also found in the two-dimensional frequency spectrum, which have not yet been recognized before. The envelope of the stimulus is also found to dominantly modulate the response to the incoming signal. The findings thus reveal that the electrophysiological response to amplitude-modulated stimuli is more complex than could be revealed by, for example, Fourier analysis. This highlights the dynamics of neural processes in the visual system.

The Study of Spatial Frequency Channels for Human Visual System

Similar to auditory perception of sound system, color perception of the human visual system also presents a multi-frequency channel property. In order to study the multi-frequency channel mechanism of how the human visual system processes color information, the paper proposed a psychophysical experiment to measure the contrast sensitivities based on 17 color samples of 16 spatial frequencies on CIELAB opponent color space. Correlation analysis was carried out on the psychophysical experiment data, and the results show obvious linear correlations of observations for different spatial frequencies of different observers, which indicates that a linear model can be used to model how human visual system processes spatial frequency information. The results of solving the model based on the experiment data of color samples show that 9 spatial frequency tuning curves can exist in human visual system with each lightness, R–G and Y–B color channel and each channel can be represented by 3 tuning curves, which reflect the “center-around” form of the human visual receptive field. It is concluded that there are 9 spatial frequency channels in human vision system. The low frequency tuning curve of a narrow-frequency bandwidth shows the characteristics of lower level receptive field for human vision system, the medium frequency tuning curve shows a low pass property of the change of medium frequent colors and the high frequency tuning curve of a width-frequency bandwidth, which has a feedback effect on the low and medium frequency channels and shows the characteristics of higher level receptive field for human vision system, which represents the discrimination of details.

Early sensitivity of evoked potentials to surface and volumetric structure during the visual perception of three-dimensional object shape.

This study used event-related potentials (ERPs) to elucidate how the human visual system processes three-dimensional (3-D) object shape structure. In particular, we examined whether the perceptual mechanisms that support the analysis of 3-D shape are differentially sensitive to higher order surface and volumetric part structure. Observers performed a whole-part novel object matching task in which part stimuli comprised sub-regions of closed edge contour, surfaces or volumetric parts. Behavioural response latency data showed an advantage in matching surfaces and volumetric parts to whole objects over contours, but no difference between surfaces and volumes. ERPs were analysed using a convergence of approaches based on stimulus dependent amplitude modulations of evoked potentials, topographic segmentation, and spatial frequency oscillations. The results showed early differential perceptual processing of contours, surfaces, and volumetric part stimuli. This was first reliably observed over occipitoparietal electrodes during the N1 (140-200ms) with a mean peak latency of 170ms, and continued on subsequent P2 (220-260ms) and N2 (260-320ms) components. The differential sensitivity in perceptual processing during the N1 was accompanied by distinct microstate patterns that distinguished among contours, surfaces and volumes, and predominant theta band activity around 4-7Hz over right occipitoparietal and orbitofrontal sites. These results provide the first evidence of early differential perceptual processing of higher order surface and volumetric shape structure within the first 200ms of stimulus processing. The findings challenge theoretical models of object recognition that do not attribute functional significance to surface and volumetric object structure during visual perception.

The human visual system estimates angle features in an internal reference frame: A computational and psychophysical study.

Angle perception is an important middle-level visual process, combining line features to generate an integrated shape percept. Previous studies have proposed two theories of angle perception-a combination of lines and a holistic feature following Weber's law. However, both theories failed to explain the dual-peak fluctuations of the just-noticeable difference (JND) across angle sizes. In this study, we found that the human visual system processes the angle feature in two stages: first, by encoding the orientation of the bounding lines and combining them into an angle feature; and second, by estimating the angle in an orthogonal internal reference frame (IRF). The IRF model fits well with the dual-peak fluctuations of the JND that neither the theory of line combinations nor Weber's law can explain. A statistical image analysis of natural images revealed that the IRF was in alignment with the distribution of the angle features in the natural environment, suggesting that the IRF reflects human prior knowledge of angles in the real world. This study provides a new computational framework for angle discrimination, thereby resolving a long-standing debate on angle perception.

A machine learning framework for full-reference 3D shape quality assessment

To decide whether the perceived quality of a mesh is influenced by a certain modification such as compression or simplification, a metric for estimating the visual quality of 3D meshes is required. Today, machine learning and deep learning techniques are getting increasingly popular since they present efficient solutions to many complex problems. However, these techniques are not much utilized in the field of 3D shape perception. We propose a novel machine learning-based approach for evaluating the visual quality of 3D static meshes. The novelty of our study lies in incorporating crowdsourcing in a machine learning framework for visual quality evaluation. We deliberate that this is an elegant way since modeling human visual system processes is a tedious task and requires tuning many parameters. We employ crowdsourcing methodology for collecting data of quality evaluations and metric learning for drawing the best parameters that well correlate with the human perception. Experimental validation of the proposed metric reveals a promising correlation between the metric output and human perception. Results of our crowdsourcing experiments are publicly available for the community.

Use and Usefulness of Dynamic Face Stimuli for Face Perception Studies-a Review of Behavioral Findings and Methodology.

Faces that move contain rich information about facial form, such as facial features and their configuration, alongside the motion of those features. During social interactions, humans constantly decode and integrate these cues. To fully understand human face perception, it is important to investigate what information dynamic faces convey and how the human visual system extracts and processes information from this visual input. However, partly due to the difficulty of designing well-controlled dynamic face stimuli, many face perception studies still rely on static faces as stimuli. Here, we focus on evidence demonstrating the usefulness of dynamic faces as stimuli, and evaluate different types of dynamic face stimuli to study face perception. Studies based on dynamic face stimuli revealed a high sensitivity of the human visual system to natural facial motion and consistently reported dynamic advantages when static face information is insufficient for the task. These findings support the hypothesis that the human perceptual system integrates sensory cues for robust perception. In the present paper, we review the different types of dynamic face stimuli used in these studies, and assess their usefulness for several research questions. Natural videos of faces are ecological stimuli but provide limited control of facial form and motion. Point-light faces allow for good control of facial motion but are highly unnatural. Image-based morphing is a way to achieve control over facial motion while preserving the natural facial form. Synthetic facial animations allow separation of facial form and motion to study aspects such as identity-from-motion. While synthetic faces are less natural than videos of faces, recent advances in photo-realistic rendering may close this gap and provide naturalistic stimuli with full control over facial motion. We believe that many open questions, such as what dynamic advantages exist beyond emotion and identity recognition and which dynamic aspects drive these advantages, can be addressed adequately with different types of stimuli and will improve our understanding of face perception in more ecological settings.

No-reference stereoscopic image quality assessment guided by visual hierarchical structure and binocular effects.

Stereoscopic image quality assessment (SIQA) is an essential technique for modern 3D image and video processing systems serving as performance evaluators and monitors. However, the study on SIQA remains immature due to the complexity of the human visual system (HVS) and binocular effects that binocular vision brings about. To overcome the difficulties, a novel method is proposed that extracts and quantifies image quality-aware features related to cortex areas in charge of visual quality perception, rather than attempting to rigorously simulate the biological processing in HVS, so that the predicting accuracy is preserved while the computational complexity remains moderate. Meanwhile, binocular effects including binocular rivalry and visual discomfort are taken into consideration. Moreover, the proposed method can be operated completely without the assistance of reference images, indicating its wide practical usages. Compared to state-of-the-art works, our method shows evident superiority in terms of effectiveness and robustness.

The electrophysiological correlates of integrated face and body-part perception.

Previous studies have suggested that the human visual system processes faces and bodies holistically-that is, the different body parts are integrated into a unified representation. However, the time course of this integrative process is less known. In the present study, we investigated this issue by recording event-related potentials evoked by a face and two hands presented simultaneously and in different configurations. When the hands were rotated to obtain a biologically implausible configuration, a reduction of the P2 amplitude was observed relative to the condition in which the face and hands were retained in their veridical configuration and were supplemented with visual cues to highlight further the overall body posture. Our results show that the P2 component is sensitive to manipulations affecting the configuration of face and hand stimuli and suggest that the P2 reflects the operation of perceptual mechanisms responsible for the integrated processing of visually presented body parts.

Pattern Recognition: Invariance Learning in Convolutional Auto Encoder Network

The ability of the human visual processing system to accommodate and retain clear understanding or identification of patterns irrespective of their orientations is quite remarkable. Conversely, pattern invariance, a common problem in intelligent recognition systems is not one that can be overemphasized; obviously, one’s definition of an intelligent system broadens considering the large variability with which the same patterns can occur. This research investigates and reviews the performance of convolutional networks, and its variant, convolutional auto encoder networks when tasked with recognition problems considering invariances such as translation, rotation, and scale. While, various patterns can be used to validate this query, handwritten Yoruba vowel characters have been used in this research. Databases of images containing patterns with constraints of interest are collected, processed, and used to train and simulate the designed networks. We provide extensive architectural and learning paradigms review of the considered networks, in view of how built-in invariance is learned. Lastly, we provide a comparative analysis of achieved error rates against back propagation neural networks, denoising auto encoder, stacked denoising auto encoder, and deep belief network.

Facilitation of contrast detection by flankers without perceived orientation.

To extract meaningful structure from noisy input signals, the human visual-processing system uses elementary structures, such as contours, to extract more complex informative structures. The first step in contour processing involves identifying local orientation. The phenomenon of collinear facilitation is important for understanding how orientation detection is implemented; at the fovea, a stripe near the contrast threshold (target) is easier to detect when it is collinearly flanked by stripes with the same orientation (flankers). This facilitation requires collinear alignment and presumably reflects cortical mechanisms in the early visual cortex. Strong collinear orientation signals are said to help in detecting a feeble signal and in establishing a smooth conscious linkage of orientations. However, contrary to this notion, we show here that relatively small but significant facilitation occurs even when the flankers have no perceived orientation. One such case involves concentric flankers that have unbiased luminance energies in all orientations. When collinearly surrounding an oriented target, these flankers facilitated detection of the target. In another case, oriented flankers that were made invisible through interocular suppression and that were monocularly surrounding an oriented target yielded collinear facilitation even though the flankers themselves were completely masked by random patterns presented to the other eye. These findings indicate that automatic, preconscious processing of orientation information at some early stage can improve the visibility of local linear elements. They also indicate the usefulness of latent visual information in detecting orientation and constructing our visual world.

A unified account of perceptual layering and surface appearance in terms of gamut relativity.

When we look at the world—or a graphical depiction of the world—we perceive surface materials (e.g. a ceramic black and white checkerboard) independently of variations in illumination (e.g. shading or shadow) and atmospheric media (e.g. clouds or smoke). Such percepts are partly based on the way physical surfaces and media reflect and transmit light and partly on the way the human visual system processes the complex patterns of light reaching the eye. One way to understand how these percepts arise is to assume that the visual system parses patterns of light into layered perceptual representations of surfaces, illumination and atmospheric media, one seen through another. Despite a great deal of previous experimental and modelling work on layered representation, however, a unified computational model of key perceptual demonstrations is still lacking. Here we present the first general computational model of perceptual layering and surface appearance—based on a boarder theoretical framework called gamut relativity—that is consistent with these demonstrations. The model (a) qualitatively explains striking effects of perceptual transparency, figure-ground separation and lightness, (b) quantitatively accounts for the role of stimulus- and task-driven constraints on perceptual matching performance, and (c) unifies two prominent theoretical frameworks for understanding surface appearance. The model thereby provides novel insights into the remarkable capacity of the human visual system to represent and identify surface materials, illumination and atmospheric media, which can be exploited in computer graphics applications.

Template changes with perceptual learning are driven by feature informativeness.

Perceptual learning changes the way the human visual system processes stimulus information. Previous studies have shown that the human brain's weightings of visual information (the perceptual template) become better matched to the optimal weightings. However, the dynamics of the template changes are not well understood. We used the classification image method to investigate whether visual field or stimulus properties govern the dynamics of the changes in the perceptual template. A line orientation discrimination task where highly informative parts were placed in the peripheral visual field was used to test three hypotheses: (1) The template changes are determined by the visual field structure, initially covering stimulus parts closer to the fovea and expanding toward the periphery with learning; (2) the template changes are object centered, starting from the center and expanding toward edges; and (3) the template changes are determined by stimulus information, starting from the most informative parts and expanding to less informative parts. Results show that, initially, the perceptual template contained only the more peripheral, highly informative parts. Learning expanded the template to include less informative parts, resulting in an increase in sampling efficiency. A second experiment interleaved parts with high and low signal-to-noise ratios and showed that template reweighting through learning was restricted to stimulus elements that are spatially contiguous to parts with initial high template weights. The results suggest that the informativeness of features determines how the perceptual template changes with learning. Further, the template expansion is constrained by spatial proximity.

Functional Role of Colour in Preattentive Cognitive Processing from Thematic Map Images

The growing use of colour in thematic map design and production, particularly in GIS based mapping environments, brings to notice a state of affairs where at least certain basic rules are required to be made to help in better, effective and efficient use of colour. One approach towards this end appears in efforts towards making the map image and input from it more compatible with the human visual processing system. This entails selection and structuring of the visual variables and the ensuing sign system of map graphics into a map image in a way in which such structuring results in an increase in the efficiency of information processing system. Many aspects of use of these visual variables from the commonly used term colour i.e. hue, value and chroma in symbolisation of thematic maps have not been investigated individually in the areas as segregation of map image’s focal realm form its peripheral realm or in other words the figure from its ground, organising perceptual groups, creating visual hierarchies by depth cues. The present paper attempts to fill certain gaps in studies related to the effects of the use of visual variables hue, value and chroma individually or in combination in map symbolisation on the preattentive stage of visual processing of information. These effects will be studied in case of area symbolised and line symbolised maps including those graduated point symbolised maps where colour dimensions are used as area fills in the blank outline symbol forms. Evidence in support of these effects will be sought from the experimental studies in design research in thematic cartography and psychology.

Repetitive Paired-pulse Transcranial Magnetic Stimulation Over the Visual Cortex Selectively Inhibits Focal Flash VEPs

BackgroundThe human visual system processes different aspects of visual information such as luminance and contrast via multiple channels. We previously used repetitive paired-pulse stimulation (rPPS) over the visual cortex to elicit a disinhibitory effect on the visual recovery function of paired pattern-reversal (PR) visual evoked potentials (VEPs). ObjectiveWe tested the hypothesis that different visual channels exhibit diverse response patterns after rPPS over the visual cortex. Thus, we examined how rPPS influenced each channel of the visual cortex. MethodsWe employed rPPS with a 1.5-ms interstimulus interval over the visual cortex at the stimulus intensity of the visual masking effect. Focal flash (FF) and PR-VEPs (check size, 15 min) were recorded to evaluate the response properties of visual channels in 10 healthy subjects. Visual stimuli were presented for the lower half-field that subtended 6° in radius. The amplitudes and latencies of each VEP were compared before and after rPPS for up to 30 min. ResultsFollowing rPPS, N1-P1 and P1-N2 amplitudes of FF-VEPs decreased linearly up to 30 min. In contrast, the amplitudes of PR-VEPs were not significantly changed. Latencies of FF- and PR-VEPs were also unchanged. ConclusionsModulation of FF-VEPs by rPPS suggests that rPPS selectively induced an inhibitory effect on the luminance channel. Therefore, rPPS may be a promising tool for exploring plastic changes in the visual cortex.