Photographs Of Natural Scenes Research Articles

Binocular Information Improves the Reliability and Consistency of Pictorial Relief

Binocular disparity is an important cue to three-dimensional shape. We assessed the contribution of this cue to the reliability and consistency of depth in stereoscopic photographs of natural scenes. Observers viewed photographs of cluttered scenes while adjusting a gauge figure to indicate the apparent three-dimensional orientation of the surfaces of objects. The gauge figure was positioned on the surfaces of objects at multiple points in the scene, and settings were made under monocular and binocular, stereoscopic viewing. Settings were used to create a depth relief map, indicating the apparent three-dimensional structure of the scene. We found that binocular cues increased the magnitude of apparent depth, the reliability of settings across repeated measures, and the consistency of perceived depth across participants. These results show that binocular cues make an important contribution to the precise and accurate perception of depth in natural scenes that contain multiple pictorial cues.

Reduced peripheral vision in glaucoma and boundary extension

ABSTRACT Clinical relevance Peripheral vision is known to be critical for spatial navigation. However, visual cognition, which impacts peripheral vision, has not been studied extensively in glaucoma. Background Spatial memory was assessed with a known to induce a robust memory distortion called “boundary extension” in which participants erroneously remember seeing more of a scene than was present in the sensory input. Methods Fifteen patients with glaucoma and 15 age-matched normally sighted controls participated in the experiment. Participants were shown 10 photographs of natural scenes randomly displayed for 0.5 s or 10 s. Following each scene, the participant was asked to draw it from memory. Results On average, boundary extension was larger, by 12%, for patients than for controls, but the difference was significant for 4 photographs. Patients tended to add more space between the object and the edges than there was between the objects and the border of the photograph. A control experiment in which participants were asked to draw isolated objects without scene context resulted in a significant reduction of the memory distortion in both groups, but patients still drew the objects smaller than controls. Conclusion The reduced field of view in glaucoma has an impact on spatial memory for scenes and on perception of size.

Steady-state visual evoked potential responses predict visual discomfort judgements.

It has been suggested that aesthetically pleasing stimuli are processed efficiently by the visual system, whereas uncomfortable stimuli are processed inefficiently. This study consists of a series of three experiments investigating this idea using a range of images of abstract artworks, photographs of natural scenes, and computer-generated stimuli previously shown to be uncomfortable. Subjective judgements and neural correlates were measured using electroencephalogram (EEG) (steady-state visual evoked potentials, SSVEPs). In addition, global image statistics (contrast, Fourier amplitude spectral slope and fractal dimension) were taken into account. When effects of physical image contrast were controlled, fractal dimension predicted discomfort judgements, suggesting the SSVEP response is more likely to be influenced by distribution of edges than the spectral slope. Importantly, when effects of physical contrast and fractal dimension were accounted for using linear mixed effects modelling, SSVEP responses predicted subjective judgements of images. Specifically, when stimuli were not matched for perceived contrast, there was a positive relationship between SSVEP responses and how pleasing a stimulus was judged to be, and conversely a negative relationship between discomfort and SSVEP response. This is significant as it shows that the neural responses in early visual areas contribute to the subjective (un)pleasantness of images, although the results of this study do not provide clear support for the theory of efficient coding as the cause of perceived pleasantness or discomfort of images, and so other explanations need to be considered.

Salient objects dominate the central fixation bias when orienting toward images.

Short-latency saccades are often biased toward salient objects or toward the center of images, for example, when inspecting photographs of natural scenes. Here, we measured the contribution of salient objects and central fixation bias to visual selection over time. Participants made saccades to images containing one salient object on a structured background and were instructed to either look at (i) the image center, (ii) the salient object, or (iii) at a cued position halfway in between the two. Results revealed, first, an early involuntary bias toward the image center irrespective of strategic behavior or the location of objects in the image. Second, the salient object bias was stronger than the center bias and prevailed over the latter when they directly competed for visual selection. In a second experiment, we tested whether the center bias depends on how well the image can be segregated from the monitor background. We asked participants to explore images that either did or did not contain a salient object while we manipulated the contrast between image background and monitor background to make the image borders more or less visible. The initial orienting toward the image was not affected by the image-monitor contrast, but only by the presence of objects—with a strong bias toward the center of images containing no object. Yet, a low image-monitor contrast reduced this center bias during the subsequent image exploration.

Guided Facial Skin Color Correction

This paper proposes an automatic image correction method for portrait photographs, which promotes consistency of facial skin color by suppressing skin color changes due to background colors. In portrait photographs, skin color is often distorted due to the lighting environment (e.g., light reflected from a colored background wall and over-exposure by a camera strobe). This color distortion is emphasized when artificially synthesized with another background color, and the appearance becomes unnatural. In our framework, we, first, roughly extract the face region and rectify the skin color distribution in a color space. Then, we perform color and brightness correction around the face in the original image to achieve a proper color balance of the facial image, which is not affected by luminance and background colors. Our color correction process attains natural results by using a guide image, unlike conventional algorithms. In particular, our guided image filtering for the color correction does not require a perfectly-aligned guide image required in the original guide image filtering method proposed by He et al. Experimental results show that our method generates more natural results than conventional methods on not only headshot photographs but also natural scene photographs. We also show automatic yearbook style photo generation as another application.

An Application of Super-Resolution Generative Adversary Networks for Quasi-Static Ultrasound Strain Elastography: A Feasibility Study

In this work, a super-resolution approach based on generative adversary network (GAN) was used to interpolate (up-sample) ultrasound radio-frequency (RF) echo data along the lateral (perpendicular to the acoustic beam direction) direction before motion estimation. Our primary objective was to investigate the feasibility of using a GAN-based super-solution approach to improve lateral resolution in the RF data as a means of improving strain image quality in quasi-static ultrasound strain elastography (QUSE). Unlike natural scene photographs, axial (parallel to the acoustic beam direction) resolution is significantly higher than that of lateral resolution in ultrasound RF data. To better handle RF data, we first modified a super-resolution generative adversary network (SRGAN) model developed by the computer vision community. We named the modified SRGAN model as super-resolution radio-frequency neural network (SRRFNN) model. Our preliminary experiments showed that, compared with axial strain elastograms obtained using the original ultrasound RF data, axial strain elastograms using ultrasound RF data up-sampled by the proposed SRRFNN model were improved. Based on the Wilcoxon rank-sum tests, such improvements were statistically significant ( $p ) for large deformation (3-5%). Also, the proposed SRRFNN model outperformed a commonly-used method ( i.e. bi-cubic interpolation used in MATLAB [Mathworks Inc., MA, USA]) in terms of improving axial strain elastograms. We concluded that applying the proposed (SRRFNN) model was feasible and good-quality strain elastography data could be obtained in in vivo tumor-bearing breast ultrasound data.

Sustained conscious access to incidental memories in RSVP.

In visual search of natural scenes, differentiation of briefly fixated but task-irrelevant distractor items from incidental memory is often comparable to explicit memorization. However, many characteristics of incidental memory remain unclear, including the capacity for its conscious retrieval. Here, we examined incidental memory for faces in either upright or inverted orientation using Rapid Serial Visual Presentation (RSVP). Subjects were instructed to detect a target face in a sequence of 8-15 faces cropped from natural scene photographs (Experiment 1). If the target face was identified within a brief time window, the subject proceeded to an incidental memory task. Here, subjects used incidental memory to discriminate between a probe face (a distractor in the RSVP stream) and a novel, foil face. In Experiment 2 we reduced scene-related semantic coherency by intermixing faces from multiple scenes and contrasted incidental memory with explicit memory, a condition where subjects actively memorized each face from the sequence without searching for a target. In both experiments, we measured objective performance (Type 1 AUC) and metacognitive accuracy (Type 2 AUC), revealing sustained and consciously accessible incidental memory for upright and inverted faces. In novel analyses of face categories, we examined whether accuracy or metacognitive judgments are affected by shared semantic features (i.e., similarity in gender, race, age). Similarity enhanced the accuracy of incidental memory discriminations but did not influence metacognition. We conclude that incidental memory is sustained and consciously accessible, is not reliant on scene contexts, and is not enhanced by explicit memorization.

ERP responses to images of abstract artworks, photographs of natural scenes, and artificially created uncomfortable images

ABSTRACTThe idea of efficient coding in the visual brain allows for predictions for the processing of various types of images, including certain artworks, natural images and uncomfortable images. Efficient processing is thought to result in lower responses compared to less efficient processing. The efficiency of the processing is suggested to depend on the architecture of the visual system and the properties of the input image. In this study, neural responses were estimated using EEG across the categories of a set of five images of abstract artworks, a set of five photographs of natural images and a set of five computer-generated uncomfortable images. EEG responses to contrast-matched images were found to be lower for the set of five abstract artworks used in the study compared to the set of photographs of natural images, lending preliminary support for the idea that certain abstract artworks, for example the work of Pollock, may be processed efficiently.

Representational differences between line drawings and photographs of natural scenes: A dissociation between multi-voxel pattern analysis and repetition suppression

Distributed representations of scene categories are consistent between color photographs (CPs) and line drawings (LDs) in the parahippocampal place area (PPA) and the retrosplenial cortex (RSC), as shown using multi-voxel pattern analysis (MVPA). Here, we used repetition suppression (RS) to further investigate the degree of representational convergence between CPs and LDs of natural scenes. MVPA and RS can capture different aspects of visual representations, and RS may prove useful in elucidating important differences in the representations of CPs and LDs of natural scenes. We performed an event-related fMRI experiment, including image-repetitions either within-type (i.e., CP to CP or LD to LD) or between-types (CP to LD, LD to CP). We found significant RS for within-type repetitions in PPA, RSC and the occipital place area (OPA), but did not observe RS for between-types repetitions. By contrast, scene categories were decodable from activity patterns evoked by both CPs and LDs using SVM classification for both within-type decoding and between-types cross-decoding. We conclude that there are representational differences between CPs and LDs in scene-selective cortex despite a category-level correspondence.

Large-Scale Traveling Waves in EEG Activity Following Eye Movement.

In spontaneous, stimulus-evoked, and eye-movement evoked EEG, the oscillatory signal shows large scale, dynamically organized patterns of phase. We investigated eye-movement evoked patterns in free-viewing conditions. Participants viewed photographs of natural scenes in anticipation of a memory test. From 200ms intervals following saccades, we estimated the EEG phase gradient over the entire scalp, and the wave activity, i.e. the goodness of fit of a wave model involving a phase gradient assumed to be smooth over the scalp. In frequencies centered at 6.5Hz, large-scale phase organization occurred, peaking around 70ms after fixation onset and taking the form of a traveling wave. According to the wave gradient, most of the times the wave spreads from the posterior-inferior to anterior-superior direction. In these directions, the gradients depended on the size and direction of the saccade. Wave propagation velocity decreased in the course of the fixation, particularly in the interval from 50 to 150ms after fixation onset. This interval corresponds to the fixation-related lambda activity, which reflects early perceptual processes following fixation onset. We conclude that lambda activity has a prominent traveling wave component. This component consists of a short-term whole-head phase pattern of specific direction and velocity, which may reflect feedforward propagation of visual information at fixation.

Orientation-dependent biases in length judgments of isolated stimuli

Vertical line segments tend to be perceived as longer than horizontal ones of the same length, but this may in part be due to configuration effects. To minimize such effects, we used isolated line segments in a two-interval, forced choice paradigm, not limiting ourselves to horizontal and vertical. We fitted psychometric curves using a Bayesian method that assumes that, for a given subject, the lapse rate is the same across all conditions. The closer a line segment's orientation was to vertical, the longer it was perceived to be. Moreover, subjects tended to report the standard line (in the second interval) as longer. The data were well described by a model that contains both an orientation-dependent and an interval-dependent multiplicative bias. Using this model, we estimated that a vertical line was on average perceived as 9.2% ± 2.1% longer than a horizontal line, and a second-interval line was on average perceived as 2.4% ± 0.9% longer than a first-interval line. Moving from a descriptive to an explanatory model, we hypothesized that anisotropy in the polar angle of lines in three dimensions underlies the horizontal-vertical illusion, specifically, that line segments more often have a polar angle of 90° (corresponding to the ground plane) than any other polar angle. This model qualitatively accounts not only for the empirical relationship between projected length and projected orientation that predicts the horizontal-vertical illusion, but also for the empirical distribution of projected orientation in photographs of natural scenes and for paradoxical results reported earlier for slanted surfaces.

Finding faces, animals, and vehicles in far peripheral vision.

Neuroimaging studies have shown that faces exhibit a central visual field bias, as compared to buildings and scenes. With a saccadic choice task, Crouzet, Kirchner, and Thorpe (2010) demonstrated a speed advantage for the detection of faces with stimuli located 8° from fixation. We used the same paradigm to examine whether the face advantage, relative to other categories (animals and vehicles), extends across the whole visual field (from 10° to 80° eccentricity) or whether it is limited to the central visual field. Pairs of photographs of natural scenes (a target and a distractor) were displayed simultaneously left and right of central fixation for 1s on a panoramic screen. Participants were asked to saccade to a target stimulus (faces, animals, or vehicles). The distractors were images corresponding to the two other categories. Eye movements were recorded with a head-mounted eye tracker. Only the first saccade was measured. Experiment 1 showed that (a) in terms of speed of categorization, faces maintain their advantage over animals and vehicles across the whole visual field, up to 80° and (b) even in crowded conditions (an object embedded in a scene), performance was above chance for the three categories of stimuli at 80° eccentricity. Experiment 2 showed that, when compared to another category with a high degree of within category structural similarity (cars), faces keep their advantage at all eccentricities. These results suggest that the bias for faces is not limited to the central visual field, at least in a categorization task.

World knowledge affects prediction as quickly as selectional restrictions: evidence from the visual world paradigm

ABSTRACTThere has been considerable debate regarding the question of whether linguistic knowledge and world knowledge are separable and used differently during processing or not (Hagoort et al, 2004). Integration of word meaning and world knowledge in language comprehension (Matsuki et al, 2011). Event-based plausibility immediately influences on-line language comprehension (Paczynski et al, 2012). Multiple influences of semantic memory on sentence processing: Distinct effects of semantic relatedness on violations of real-world event/state knowledge and animacy selection restrictions (Warren et al, 2007). Investigating effects of selectional restriction violations and plausibility violation severity on eye movements in reading. Previous investigations into this question have provided mixed evidence as to whether violations of selectional restrictions are detected earlier than violations of world knowledge. We report a visual world eye-tracking study comparing the timing of facilitation contributed by selectional restrictions vs. world knowledge. College-aged adults (n = 36) viewed photographs of natural scenes while listening to sentences. Participants anticipated upcoming direct objects similarly regardless of whether facilitation was provided by only world knowledge or a combination of selectional restrictions and world knowledge. These results suggest that selectional restrictions are not available earlier in comprehension than world knowledge.

The (In)Effectiveness of Simulated Blur for Depth Perception in Naturalistic Images.

We examine depth perception in images of real scenes with naturalistic variation in pictorial depth cues, simulated dioptric blur and binocular disparity. Light field photographs of natural scenes were taken with a Lytro plenoptic camera that simultaneously captures images at up to 12 focal planes. When accommodation at any given plane was simulated, the corresponding defocus blur at other depth planes was extracted from the stack of focal plane images. Depth information from pictorial cues, relative blur and stereoscopic disparity was separately introduced into the images. In 2AFC tasks, observers were required to indicate which of two patches extracted from these images was farther. Depth discrimination sensitivity was highest when geometric and stereoscopic disparity cues were both present. Blur cues impaired sensitivity by reducing the contrast of geometric information at high spatial frequencies. While simulated generic blur may not assist depth perception, it remains possible that dioptric blur from the optics of an observer’s own eyes may be used to recover depth information on an individual basis. The implications of our findings for virtual reality rendering technology are discussed.

Perceptual expertise improves category detection in natural scenes.

There is much debate about how detection, categorization, and within-category identification relate to one another during object recognition. Whether these tasks rely on partially shared perceptual mechanisms may be determined by testing whether training on one of these tasks facilitates performance on another. In the present study we asked whether expertise in discriminating objects improves the detection of these objects in naturalistic scenes. Self-proclaimed car experts (N = 34) performed a car discrimination task to establish their level of expertise, followed by a visual search task where they were asked to detect cars and people in hundreds of photographs of natural scenes. Results revealed that expertise in discriminating cars was strongly correlated with car detection accuracy. This effect was specific to objects of expertise, as there was no influence of car expertise on person detection. These results indicate a close link between object discrimination and object detection performance, which we interpret as reflecting partially shared perceptual mechanisms and neural representations underlying these tasks: the increased sensitivity of the visual system for objects of expertise – as a result of extensive discrimination training – may benefit both the discrimination and the detection of these objects. Alternative interpretations are also discussed.

Linear and non-linear properties of feature selectivity in V4 neurons.

Extrastriate area V4 is a critical component of visual form processing in both humans and non-human primates. Previous studies have shown that the tuning properties of V4 neurons demonstrate an intermediate level of complexity that lies between the narrow band orientation and spatial frequency tuning of neurons in primary visual cortex and the highly complex object selectivity seen in inferotemporal neurons. However, the nature of feature selectivity within this cortical area is not well understood, especially in the context of natural stimuli. Specifically, little is known about how the tuning properties of V4 neurons, measured in isolation, translate to feature selectivity within natural scenes. In this study, we assessed the degree to which preferences for natural image components can readily be inferred from classical orientation and spatial frequency tuning functions. Using a psychophysically-inspired method we isolated and identified the specific visual “driving features” occurring in natural scene photographs that reliably elicited spiking activity from single V4 neurons. We then compared the measured driving features to those predicted based on the spectral receptive field (SRF), estimated from responses to narrowband sinusoidal grating stimuli. This approach provided a quantitative framework for assessing the degree to which linear feature selectivity was preserved during natural vision. First, we found evidence of both spectrally and spatially tuned suppression within the receptive field, neither of which were present in the linear SRF. Second, we found driving features that were stable during translation of the image across the receptive field (due to small fixational eye movements). The degree of translation invariance fell along a continuum, with some cells showing nearly complete invariance across the receptive field and others exhibiting little to no position invariance. This form of limited translation invariance could indicate that a subset of V4 neurons are insensitive to small fixational eye movements, supporting perceptual stability during natural vision.

Stimulus Type and the List Strength Paradigm

In recognition memory, increasing the strength of studied items does not reduce performance on other items, an effect dubbed the null list strength effect (LSE). While this finding has been replicated many times, it has rarely been tested using stimuli other than single words. Kinnell and Dennis ( 2012 ) recently tested for the presence of list length effects using non-word stimulus classes while controlling for the confounds that are present in list length designs. Small list length effects were found for fractal and face images. We adopted the same paradigm and stimuli used by Kinnell and Dennis to test whether these stimuli would be susceptible to list strength effects as well. We found significant LSEs for fractal images, but null LSEs for face images and natural scene photographs. Stimuli other than words do appear to be susceptible to list strength effects, but these effects are small and restricted to particular stimulus classes, as is the case in list length designs. Models of memory may be able to address differences between these stimulus classes by attributing differences in representational overlap between the stimulus classes.

The timing and directional connectivity of human frontoparietal and ventral visual attention networks in emotional scene perception

Electrocortical and hemodynamic measures reliably identify enhanced activity in the ventral and dorsal visual cortices during the perception of emotionally arousing versus neutral images, an effect that may reflect directive feedback from the subcortical amygdala. However, other brain regions strongly modulate visual attention, such as frontal eye fields (FEF) and intraparietal sulcus (IPS). Here we employ rapid sampling of BOLD signal (4Hz) in the amygdala, fusiform gyrus (FG), FEF and IPS in 42 human participants as they viewed a series of emotional and neutral natural scene photographs balanced for luminosity and complexity, to test whether emotional discrimination is evident in dorsal structures prior to such discrimination in the amygdala and FG. Granger causality analyses were used to assess directional connectivity within dorsal and ventral networks. Results demonstrate emotionally-enhanced peak BOLD signal in the amygdala, FG, FEF, and IPS, with the onset of BOLD signal discrimination occurring between 2 and 3s after stimulus onset in ventral structures, and between 4 and 5s in FEF and IPS. Granger causality estimates yield stronger directional connectivity from IPS to FEF than the reverse in this emotional picture paradigm. Consistent with a reentrant perspective of emotional scene perception, greater directional connectivity was found from the amygdala to FG compared to the reverse. These data support a perspective in which the registration of emotional scene content is orchestrated by the amygdala and rostral inferotemporal visual cortex.

The contribution of central and peripheral vision in scene categorization: A study on people with central vision loss

Studies in normally sighted people suggest that scene recognition is based on global physical properties and can be accomplished by the low resolution of peripheral vision. We examine the contribution of peripheral and central vision in scene gist recognition in patients with central vision loss and age-matched controls. Twenty-one patients with neovascular age related macular degeneration (AMD), with a visual acuity lower than 20/50, and 15 age-matched normally sighted controls participated in a natural/urban scene categorization task. The stimuli were colored photographs of natural scenes presented randomly at one of five spatial locations of a computer screen: centre, top left, top right, bottom left and bottom right at 12° eccentricity. Sensitivity (d') and response times were recorded. Normally sighted people exhibited higher sensitivity and shorter response times when the scene was presented centrally than for peripheral pictures. Sensitivity was lower and response times were longer for people with AMD than for controls at all spatial location. In contrast to controls patients were not better for central than for peripheral pictures. The results of normally sighted controls indicate that scene categorization can be accomplished by the low resolution of peripheral vision but central vision remains more efficient than peripheral vision for scene gist recognition. People with central vision loss likely categorized scenes on the basis of low frequency information both in normal peripheral vision and in low acuity central vision.

The scene and the unseen: Manipulating photographs for experiments on change blindness and scene memory

The change blindness paradigm, in which participants often fail to notice substantial changes in a scene, is a popular tool for studying scene perception, visual memory, and the link between awareness and attention. Some of the most striking and popular examples of change blindness have been demonstrated with digital photographs of natural scenes; in most studies, however, much simpler displays, such as abstract stimuli or "free-floating" objects, are typically used. Although simple displays have undeniable advantages, natural scenes remain a very useful and attractive stimulus for change blindness research. To assist researchers interested in using natural-scene stimuli in change blindness experiments, we provide here a step-by-step tutorial on how to produce changes in natural-scene images with a freely available image-processing tool (GIMP). We explain how changes in a scene can be made by deleting objects or relocating them within the scene or by changing the color of an object, in just a few simple steps. We also explain how the physical properties of such changes can be analyzed using GIMP and MATLAB (a high-level scientific programming tool). Finally, we present an experiment confirming that scenes manipulated according to our guidelines are effective in inducing change blindness and demonstrating the relationship between change blindness and the physical properties of the change and inter-individual differences in performance measures. We expect that this tutorial will be useful for researchers interested in studying the mechanisms of change blindness, attention, or visual memory using natural scenes.