Pairs Of Scenes Research Articles

Scene construction processes in the anterior hippocampus during temporal episodic memory retrieval.

Although the hippocampus has been implicated in both the temporal organization of memories and association of scene elements, some theoretical accountsposit that the role of the hippocampus in episodic memory is largely atemporal. In this study, we set out to explore this discrepancy by identifying hippocampal activity patterns related to scene construction while participants performed a temporal order memory task. Participants in the fMRI scanner were shown a sequence of photographs, each consisting of a central object and a contextual background scene. On each retrieval trial, participants were shown a pair of the original photographs (FULL), objects from the scenes without the background (OBJ), or background contexts without the main foreground object (BACK). In the temporal order judgment (TOJ) task, participants judged the temporal order of the pair of scenes; in the Viewing trials, two identical scenes were shown without any task. First, we found that the anterior hippocampus-particularly the CA1 and subiculum-showed similar patterns of activation between the BACK and OBJ conditions, suggesting that scene construction occurred spontaneously during both TOJ and Viewing. Furthermore, neural markers of scene construction in the anterior hippocampus did not apply to incorrect trials, showing that successful temporal memory retrieval was functionally linked to scene construction. In the cortex, time-processing areas, such as the supplementary motor area and the precuneus, and scene-processing areas, such as the parahippocampal cortex, were activated and functionally connected with the hippocampus. Together, these results support the view that the hippocampus is concurrently involved in scene construction and temporal organization of memory and propose a model of hippocampal episodic memory that takes both processes into account.

A scene with an invisible wall - navigational experience shapes visual scene representation.

Human navigation heavily relies on visual information. Although many previous studies have investigated how navigational information is inferred from visual features of scenes, little is understood about the impact of navigational experience on visual scene representation. In this study, we examined how navigational experience influences both the behavioral and neural responses to a visual scene. During training, participants navigated in the virtual reality (VR) environments which we manipulated navigational experience while holding the visual properties of scenes constant. Half of the environments allowed free navigation (navigable), while the other half featured an 'invisible wall' preventing the participants to continue forward even though the scene was visually navigable (non-navigable). During testing, participants viewed scene images from the VR environment while completing either a behavioral perceptual identification task (Experimentl) or an fMRI scan (Experiment2). Behaviorally, we found that participants judged a scene pair to be significantly more visually different if their prior navigational experience varied, even after accounting for visual similarities between the scene pairs. Neurally, multi-voxel pattern of the parahippocampal place area (PPA) distinguished visual scenes based on prior navigational experience alone. These results suggest that the human visual scene cortex represents information about navigability obtained through prior experience, beyond those computable from the visual properties of the scene. Taken together, these results suggest that scene representation is modulated by prior navigational experience to help us construct a functionally meaningful visual environment.

SynFAGnet: A Fully Automated Generative Network for Realistic Fire Image Generation

This paper proposes a fully automated generative network (“SynFAGnet”) for automatically creating a realistic-looking synthetic fire image. SynFAGnet is used as a data augmentation technique to create diverse data for training models, thereby solving problems related to real data acquisition and data imbalances. SynFAGnet comprises two main parts: an object-scene placement net (OSPNet) and a local–global context-based generative adversarial network (LGC-GAN). The OSPNet identifies suitable positions and scales for fires corresponding to the background scene. The LGC-GAN enhances the realistic appearance of synthetic fire images created by a given fire object-background scene pair by assembling effects such as halos and reflections in the surrounding area in the background scene. A comparative analysis shows that SynFAGnet achieves better outcomes than previous studies for both the Fréchet inception distance and learned perceptual image patch similarity evaluation metrics (values of 17.232 and 0.077, respectively). In addition, SynFAGnet is verified as a practically applicable data augmentation technique for training datasets, as it improves the detection and instance segmentation performance.

A finely tuned deep transfer learning algorithm to compare outsole images

AbstractIn forensic practice, evaluating shoeprint evidence is challenging because the differences between images of two different outsoles can be subtle. In this paper, we propose a deep transfer learning‐based matching algorithm called the Shoe‐MS algorithm that quantifies the similarity between two outsole images. The Shoe‐MS algorithm consists of a Siamese neural network for two input images followed by a transfer learning component to extract features from outsole impression images. The added layers are finely tuned using images of shoe soles. To test the performance of the method we propose, we use a study dataset that is both realistic and challenging. The pairs of images for which we know ground truth include (1) close non‐matches and (2) mock‐crime scene pairs. The Shoe‐MS algorithm performed well in terms of prediction accuracy and was able to determine the source of pairs of outsole images, even when comparisons were challenging. When using a score‐based likelihood ratio, the algorithm made the correct decision with high probability in a test of the hypothesis that images had a common source. An important advantage of the proposed approach is that pairs of images can be compared without alignment. In initial tests, Shoe‐MS exhibited better‐discriminating power than existing methods.

ObScene database: Semantic congruency norms for 898 pairs of object-scene pictures

Research on the interaction between object and scene processing has a long history in the fields of perception and visual memory. Most databases have established norms for pictures where the object is embedded in the scene. In this study, we provide a diverse and controlled stimulus set comprising real-world pictures of 375 objects (e.g., suitcase), 245 scenes (e.g., airport), and 898 object–scene pairs (e.g., suitcase–airport), with object and scene presented separately. Our goal was twofold. First, to create a database of object and scene pictures, normed for the same variables to have comparable measures for both types of pictures. Second, to acquire normative data for the semantic relationships between objects and scenes presented separately, which offers more flexibility in the use of the pictures and allows disentangling the processing of the object and its context (the scene). Along three experiments, participants evaluated each object or scene picture on name agreement, familiarity, and visual complexity, and rated object–scene pairs on semantic congruency. A total of 125 septuplets of one scene and six objects (three congruent, three incongruent), and 120 triplets of one object and two scenes (in congruent and incongruent pairings) were built. In future studies, these objects and scenes can be used separately or combined, while controlling for their key features. Additionally, as object–scene pairs received semantic congruency ratings along the entire scale, researchers may select among a wide range of congruency values. ObScene is a comprehensive and ecologically valid database, useful for psychology and neuroscience studies of visual object and scene processing.

ORCA: A picture database of object-scene arrangements for cross-cultural and aging research.

In recent years, cross-cultural research on the modulation of basic cognitive processes by culture has intensified - also from an aging perspective. Despite this increased research interest, only a few cross-culturally normed non-verbal stimulus sets are available to support cross-cultural cognitive research in younger and older adults. Here we present the ORCA (Official Rating of Complex Arrangements) picture database, which includes a total of 720 object-scene compositions sorted into 180 quadruples (e.g., two different helmets placed in two different deserts). Each quadruple contains visually and semantically matched pairs of objects and pairs of scenes with varying degrees of semantic fit between objects and scenes. A total of 95 younger and older German and Chinese adults rated every object-scene pair on object familiarity and semantic fit between object and scene. While the ratings were significantly correlated between cultures and age groups, small but significant culture and age differences emerged. Object familiarity was higher for older adults than younger adults and for German participants than for Chinese participants. Semantic fit was rated lower by German older adults and Chinese younger adults as compared to German younger adults and Chinese older adults. Due to the large number of stimuli, our database is particularly well suited for cognitive and neuroscientific research on cross-cultural and age-related differences in perception, attention, and memory.

A Meditation on Loss

A Meditation on Loss

Landsat 9 Cross Calibration Under-Fly of Landsat 8: Planning, and Execution

During the early post-launch phase of the Landsat 9 mission, the Landsat 8 and 9 mission teams conducted a successful under-fly of Landsat 8 by Landsat 9, allowing for the near-simultaneous data collection of common Earth targets by the on-board sensors for cross-calibration. This effort, coordinated by the Landsat Calibration and Validation team, required contributions from various entities across National Aeronautics and Space Administration and U.S. Geological Survey such as Flight Dynamics, Systems, Mission Planning, and Flight Operations teams, beginning about 18 months prior to launch. Plans existed to allow this under-fly for any possible launch date of Landsat 9. This included 16 ascent plans and 16 data acquisition plans, one for every day of the Landsat orbital repeat period, with a minimum of 5 days of useful coverage overlap between the sensors. After the Landsat 9 launch, the plan executed, and led to the acquisition of over 2000 partial to full overlapping scene pairs. Although containing less than the expected number of scenes, this dataset was larger than previous Landsat mission under-fly efforts and more than sufficient for performing cross-calibration of the Landsat 8 and Landsat 9 sensors. The details of the planning process and execution of this under-fly are presented.

Attentional bias toward negative stimuli in PTSD: an eye-tracking study.

Research on biased processing of aversive stimuli in posttraumatic stress disorder (PTSD) has produced inconsistent results between response time (RT) and eye-tracking studies. Recent RT-based results of dot-probe studies showed no attentional bias (AB) for threat while eye-tracking research suggested heightened sustained attention for this information. Here, we used both RT-based and eye-tracking measures to explore the dynamics of AB to negative stimuli in PTSD. Twenty-three individuals diagnosed with PTSD, 23 trauma-exposed healthy controls, and 23 healthy controls performed an emotional dot-probe task with pairs of negative and neutral scenes presented for either 1 or 2 s. Analyses included eye movements during the presentation of the scenes and RT associated with target localization. There was no evidence for an AB toward negative stimuli in PTSD from RT measures. However, the main eye-tracking results revealed that all three groups showed longer dwell times on negative pictures than neutral pictures at 1 s and that this AB was stronger for individuals with PTSD. Moreover, although AB disappeared for the two groups of healthy controls with prolonged exposure, it persisted for individuals with PTSD. PTSD is associated with an AB toward negative stimuli, characterized by heightened sustained attention toward negative scenes once detected. This study sheds light on the dynamics of AB to negative stimuli in PTSD and encourages us to consider optimized therapeutic interventions targeting abnormal AB patterns.

Stability and consistency of metamemory judgments within a session.

There has been much interest in assessing individual and group differences in metamemory skills. These endeavors require or would benefit from enhanced knowledge about the stability and consistency of metamemory judgments over successive trials. However, few studies have examined these aspects. Thus, the present study investigated and compared the within-session stability and consistency of three major types of metamemory judgment: judgment of learning (JOL), feeling-of-knowing judgment (FOK), and retrospective confidence judgment (RCJ), using a single-task paradigm. A total of 38 healthy young adults (17 males, 21 females) completed three trials of a face–scene associative learning task designed to assess metamemory. In each trial, participants rated their JOLs while studying a new set of face–scene pairs, and then rated their FOKs and RCJs while their memory was being tested. The stability and consistency of the mean confidence ratings and the relationships between confidence rating and memory performance, indexed by two gamma estimates, were analyzed and compared across types of metamemory judgments. Over trials, there was a significant decrease in the mean rating for JOL but not for FOK or RCJ. Also, the gamma scores of JOL, but not that of FOK or RCJ, significantly improved with practice. Furthermore, for each type of metamemory judgment, the mean confidence rating showed excellent consistency across trials. Depending on the judgment type and gamma estimation method, the consistency of gamma scores ranged from poor to excellent. Thus, the present study clarified the temporal dynamics of various types of metamemory judgments and the consistency of metamemory measures.

Talk-to-Resolve: Combining scene understanding and spatial dialogue to resolve granular task ambiguity for a collocated robot

The utility of collocating robots largely depends on the easy and intuitive interaction mechanism with the human. If a robot accepts task instruction in natural language, first, it has to understand the user’s intention by decoding the instruction. However, while executing the task, the robot may face unforeseeable circumstances due to the variations in the observed scene and therefore requires further user intervention. In this article, we present a system called Talk-to-Resolve (TTR) that enables a robot to initiate a coherent dialogue exchange with the instructor by observing the scene visually to resolve the impasse. Through dialogue, it either finds a cue to move forward in the original plan, an acceptable alternative to the original plan, or affirmation to abort the task altogether. To realize the possible stalemate, we utilize the dense captions of the observed scene and the given instruction jointly to compute the robot’s next action. We evaluate our system based on a data set of initial instruction and situational scene pairs. Our system can identify the stalemate and resolve them with appropriate dialogue exchange with 82% accuracy. Additionally, a user study reveals that the questions from our systems are more natural (4.02 on average on a scale of 1 to 5) as compared to a state-of-the-art (3.08 on average).

A Double-Edged Sword: The Role of Prior Knowledge in Memory Aging.

IntroductionPeople accumulate knowledge throughout their lifespan and the accumulated knowledge influences how we encode and retrieve information in memory processing. This study aims to investigate the role of knowledge in associative memory across the adult lifespan, and specifically examines the effects of two material properties that interact with prior knowledge: congruency – whether the material is congruent with people’s prior knowledge, and ambiguity – whether the material is ambiguous to interpret based on prior knowledge.Method273 participants (aged 22–70 years old) completed an incidental memory task online. Participants were shown pictures depicting an object in a scene and judged if the object was likely or unlikely to be in the particular scene. Later, in the recognition test, participants were asked to identify if the exact picture was presented earlier. The pictures were manipulated to have varying levels of congruency, meaning that some depicted likely object–scene pairs and some unlikely. We also measured how different the likely/unlikely judgment for each object–scene pair was across all participants to determine the ambiguity level of the object–scene pair: some were more likely to receive diverse responses across people, whereas others are unambiguously consistent (or inconsistent) with common knowledge shared by most people. We used mixed-effects logistic regressions to predict memory outcome for each trial as a function of age, age2, congruency/ambiguity, and their interactions.ResultsThe object–scene pairs perceived as congruent had higher hit rates than incongruent ones, as well as higher false alarm rates, especially in middle-aged and older people. Higher ambiguity was also related to both greater true and false memory, independent of age. Finally, the effect of ambiguity only emerged when the object–scene pair was perceived incongruent.DiscussionThe results suggest that people rely on prior knowledge to process new information and that this reliance improves hit responses, but also induces false memories particularly for middle-aged and older people, suggesting a double-edged role of knowledge in associative memory and its disproportionate influence on memory aging. Over-reliance on knowledge in older adults, which has been suspected in other cognitive processes, may be one of the mechanisms underlying associative memory decrease in aging.

Cross-Situational Word Learning With Multimodal Neural Networks.

In order to learn the mappings from words to referents, children must integrate co-occurrence information across individually ambiguous pairs of scenes and utterances, a challenge known as cross-situational word learning. In machine learning, recent multimodal neural networks have been shown to learn meaningful visual-linguistic mappings from cross-situational data, as needed to solve problems such as image captioning and visual question answering. These networks are potentially appealing as cognitive models because they can learn from raw visual and linguistic stimuli, something previous cognitive models have not addressed. In this paper, we examine whether recent machine learning approaches can help explain various behavioral phenomena from the psychological literature on cross-situational word learning. We consider two variants of a multimodal neural network architecture and look at seven different phenomena associated with cross-situational word learning and word learning more generally. Our results show that these networks can learn word-referent mappings from a single epoch of training, mimicking the amount of training commonly found in cross-situational word learning experiments. Additionally, these networks capture some, but not all of the phenomena we studied, with all of the failures related to reasoning via mutual exclusivity. These results provide insight into the kinds of phenomena that arise naturally from relatively generic neural network learning algorithms, and which word learning phenomena require additional inductivebiases.

Seasonal and Multiyear Flow Variability on the Prince of Wales Icefield, Ellesmere Island: 2009–2019

AbstractOver the last two decades, Trinity, Wykeham, Ekblaw, and Cadogan glaciers have retreated and been identified as the primary sources of iceberg flux from Prince of Wales (POW) Icefield, Ellesmere Island, accounting for ∼83% of total solid ice discharge. In this study, we used a total of 167 Radarsat‐2 Synthetic Aperture Radar scene pairs collected between 2009 and 2019 to derive winter surface velocities of these four major basins of the POW Icefield. Over this period both Cadogan and Ekblaw glaciers underwent multiyear acceleration and deceleration limited to their lower parts, consistent with characteristics of “pulse‐type” glaciers. Trinity and Wykeham glaciers are currently the fastest flowing glaciers in the CAA and are grounded below sea level for ∼40 km up‐glacier from their termini. Both glaciers underwent multiyear repeating periods of velocity acceleration between 2009 and 2019 which coincided with significant thinning at their termini. As of 2017, Trinity and Wykeham were each within ∼10 m of flotation over their lowermost 4 km. We also identified a shift in flow since 2014 on Trinity and Wykeham, after which winter flow rates began to propagate up‐glacier and were nearly identical to flow rates observed throughout the rest of the year, indicating that seasonal variability in flow has become less apparent in recent years. Our findings suggest that Trinity and Wykeham glaciers have transitioned to a flow type dominated by dynamic thinning, which is strongly influenced by subglacial topography and may be susceptible to instability of the glacier front and large‐scale collapse.

Extended Pseudo Invariant Calibration Site-Based Trend-to-Trend Cross-Calibration of Optical Satellite Sensors

Satellite sensors have been extremely useful and are in massive demand in the understanding of the Earth’s surface and monitoring of changes. For quantitative analysis and acquiring consistent measurements, absolute radiometric calibration is necessary. The most common vicarious approach of radiometric calibration is cross-calibration, which helps to tie all the sensors to a common radiometric scale for consistent measurement. One of the traditional methods of cross-calibration is performed using temporally and spectrally stable pseudo-invariant calibration sites (PICS). This technique is limited by adequate cloud-free acquisitions for cross-calibration which would require a longer time to study the differences in sensor measurements. To address the limitation of traditional PICS-based approaches and to increase the cross-calibration opportunity for quickly achieving high-quality results, the approach presented here is based on using extended pseudo invariant calibration sites (EPICS) over North Africa. With the EPICS-based approach, the area of extent of the cross-calibration site covers a large portion of the North African continent. With targets this large, many sensors should image some portion of EPICS nearlydaily, allowing for evaluation of performance with much greater frequency. By using these near-daily measurements, trends of the sensor’s performance are then used to evaluate sensor-to-sensor daily cross-calibration. With the use of the proposed methodology, the dataset for cross-calibration is increased by an order of magnitude compared to traditional approaches, resulting in the differences between any two sensors being detected within a much shorter time. Using this new trend in trend cross-calibration approaches, gains were evaluated for Landsat 7/8 and Sentinel 2A/B, with the results showing that the sensors are calibrated within 2.5% (within less than 8% uncertainty) or better for all sensor pairs, which is consistent with what the traditional PICS-based approach detects. The proposed cross-calibration technique is useful to cross-calibrate any two sensors without the requirement of any coincident or near-coincident scene pairs, while still achieving results similar to traditional approaches in a short time.

Testing the adequacy of luminous change descriptors to represent dynamic attributes in outdoor views

Outdoor views are critical for sustaining well-being in urbanized environments, as they increase opportunities for contact with natural and dynamic features. Yet, current view assessment procedures do not investigate the role of dynamic attributes in views. In this study, we evaluated a systematic procedure to test the adequacy of representing dynamic outdoor attributes as a function of lightness changes over time. Using image processing and monitoring techniques, we examined the presence of lightness changes in views in two seasons and as a function of three spatial parameters. In Part 1, we evaluated light variations in 16 distinct view scenes collected in summer (n = 320 intervals). In Part 2, we examined daylight changes in four pairs of identical view scenes collected in summer and fall (n = 80 intervals each). Continuous solar radiation measurements anchored lightness changes values as a function of sky conditions. The results showed the recurrence of lightness changes in views for Part 1 (77%) and Part 2 (75%), suggesting the prevalence of light changes in views across the year. In Part 1, lightness changes in views presented statistical significance in naturalistic conditions and three sky types, yet more data is needed for comparative analysis. In Part 1, the prevalence of Global variation in views was demonstrated. Likewise, two view types (i.e., Roof and Wall) showed a clear frequency of lightness changes under naturalistic conditions (76% and 74%). These outcomes help to sustain the adequacy of lightness change for describing dynamic attributes in views.

The Visual Polemic in Tolstoy's War and Peace: Icons and Oil Paintings

The phenomenon of Napoleon and Napoleonism that Tolstoy attacks in War and Peace is not only—arguably, not even primarily—a textual phenomenon. The cult of Napoleon was to a great extent a phenomenon created by the visual arts; portraits of Napoleon and of key moments in his career played a central role in promoting him as a “Great Man.” War and Peace contains numerous direct and indirect references to these images, and Tolstoy uses them to build his narrative. This paper analyzes two key pairs of scenes in which Tolstoy explicitly invokes Napoleonic visual images and undercuts them by juxtaposing them to Russian icons.

A Saccadic Choice Task for Target Face Detection at Large Visual Eccentricities in Patients with Glaucoma.

Little is known about the perception of glaucomatous patients at large visual eccentricities. We show that the patients' performance drops beyond 40° eccentricity even for large images of scenes, suggesting that clinical tests should assess the patients' vision at larger eccentricities than 24 or 30°. Daily activities such as visual search, spatial navigation, and hazard detection require rapid scene recognition on a wide field of view. We examined whether participants with visual field loss at standard automated perimetry 30-2 were able to detect target faces at large visual eccentricities. Twelve patients with glaucoma and 14 control subjects were asked to detect a face in a two-alternative saccadic forced choice task. Pairs of scenes, one containing a face, were randomly displayed at 10, 20, 40, 60, or 80° eccentricity on a panoramic screen covering 180° horizontally. Participants were asked to detect and to saccade toward the scene containing a face. Saccade latencies were significantly slower in patients (264 milliseconds; confidence interval [CI], 222 to 306 milliseconds) than in control subjects (207 milliseconds; CI, 190 to 226 milliseconds), and accuracy was significantly lower in patients (70% CI, 65 to 85%) than in control subjects (75.7% CI, 71.5 to 79.5%). Although still significantly above chance at 60°, the patients' performance dropped beyond 40° eccentricity. The control subjects' performance was still above chance at 80° eccentricity. In patients with various degrees of peripheral visual field defect, performance dropped beyond 40° eccentricity for large images at a high contrast. This result could reflect reduced spread of exploration in glaucoma.

Convolutional Neural Network Training for RGBN Camera Color Restoration Using Generated Image Pairs

RGBN cameras that can capture visible light and near-infrared (NIR) light simultaneously produce better color image quality in low-light-level conditions. However, these RGBN cameras introduce additional color bias caused by the mixing of visible information and NIR information. The color correction matrix model widely used in current commercial color digital cameras cannot handle the complicated mapping function between biased color and ground truth color. Convolutional neural networks (CNNs) are good at fitting such complicated relationships, but they require a large quantity of training image pairs of different scenes. In order to achieve satisfactory training results, large amounts of data must be captured manually, even when data augmentation techniques are applied, requiring significant time and effort. Hence, a data generation method for training pairs that are consistent with target RGBN camera parameters, based on an open access RGB-NIR dataset, is proposed. The proposed method is verified by training an RGBN camera color restoration CNN model with generated data. The results show that the CNN model trained with the generated data can achieve satisfactory RGBN color restoration performance with different RGBN sensors.

Attentional bias towards negative stimuli in healthy individuals and the effects of trait anxiety

This study aimed to investigate the time course of attentional bias for negative information in healthy individuals and to assess the associated influence of trait anxiety. Thirty-eight healthy volunteers performed an emotional dot-probe task with pairs of negative and neutral scenes, presented for either 1 or 2 s and followed by a target placed at the previous location of either negative or neutral stimulus. Analyses included eye movements during the presentation of the scenes and response times associated with target localization. In a second step, analyses focused on the influence of trait anxiety. While there was no significant difference at the behavioral level, the eye-tracking data revealed that negative information held longer attention than neutral stimuli once fixated. This initial maintenance bias towards negative pictures then increased with increasing trait anxiety. However, at later processing stages, only individuals with the highest trait anxiety appeared to fixate longer on negative pictures than neutral pictures, individuals with low trait anxiety showing the opposite pattern. This study provides novel evidence that healthy individuals display an attentional maintenance bias towards negative stimuli, which is associated with trait anxiety.