Spatial Relational Information Research Articles

Towards bias-aware visual question answering: Rectifying and mitigating comprehension biases

Transformers have become essential for capturing intra- and inter-dependencies in visual question answering (VQA). Yet, challenges remain in overcoming inherent comprehension biases and improving the relational dependency modeling and reasoning capabilities crucial for VQA tasks. This paper presents RMCB, a novel VQA model designed to mitigate these biases by integrating contextual information from both visual and linguistic sources and addressing potential comprehension limitations at each end. RMCB introduces enhanced relational modeling for language tokens by leveraging textual context, addressing comprehension biases arising from the isolated pairwise modeling of token relationships. For the visual component, RMCB systematically incorporates both absolute and relative spatial relational information as contextual cues for image tokens, refining dependency modeling and strengthening inferential reasoning to alleviate biases caused by limited contextual understanding. The model’s effectiveness was evaluated on benchmark datasets VQA-v2 and CLEVR, achieving state-of-the-art results with accuracies of 71.78% and 99.27%, respectively. These results underscore RMCB’s capability to effectively address comprehension biases while advancing the relational reasoning needed for VQA.

SPRITE: improving spatial gene expression imputation with gene and cell networks.

Spatially resolved single-cell transcriptomics have provided unprecedented insights into gene expression in situ, particularly in the context of cell interactions or organization of tissues. However, current technologies for profiling spatial gene expression at single-cell resolution are generally limited to the measurement of a small number of genes. To address this limitation, several algorithms have been developed to impute or predict the expression of additional genes that were not present in the measured gene panel. Current algorithms do not leverage the rich spatial and gene relational information in spatial transcriptomics. To improve spatial gene expression predictions, we introduce Spatial Propagation and Reinforcement of Imputed Transcript Expression (SPRITE) as a meta-algorithm that processes predictions obtained from existing methods by propagating information across gene correlation networks and spatial neighborhood graphs. SPRITE improves spatial gene expression predictions across multiple spatial transcriptomics datasets. Furthermore, SPRITE predicted spatial gene expression leads to improved clustering, visualization, and classification of cells. SPRITE can be used in spatial transcriptomics data analysis to improve inferences based on predicted gene expression. The SPRITE software package is available at https://github.com/sunericd/SPRITE. Code for generating experiments and analyses in the manuscript is available at https://github.com/sunericd/sprite-figures-and-analyses.

Hierarchical processing of feature, egocentric and relational information for spatial orientation in domestic chicks.

Animals can use different types of information for navigation. Domestic chicks (Gallus gallus) prefer to use local features as a beacon over spatial relational information. However, the role of egocentric navigation strategies is less understood. Here, we tested domestic chicks' egocentric and allocentric orientation abilities in a large circular arena. In experiment 1, we investigated whether domestic chicks possess a side bias during viewpoint-dependent egocentric orientation, revealing facilitation for targets on the chicks' left side. Experiment 2 showed that local features are preferred over viewpoint-dependent egocentric information when the two conflict. Lastly, in experiment 3, we found that in a situation where there is a choice between egocentric and allocentric spatial relational information provided by free-standing objects, chicks preferentially rely on egocentric information. We conclude that chicks orient according to a hierarchy of cues, in which the use of the visual appearance of an object is the dominant strategy, followed by viewpoint-dependent egocentric information and finally by spatial relational information.

What is moving where? Infants' visual attention to dynamic objects may assist with processing of spatial relations.

A central question in infant spatial cognition concerns how infants form abstract categories of spatial relations such as support (on) and containment (in). Prior work suggests two different possibilities regarding the role of attention to objects in infants' formation of abstract categories of spatial relations: Attention to objects may compete with (and thus hamper) attention to the spatial relations between them, or assist with encoding of the spatial relation information. Using eye-tracking, we examined how infants' visual attention to objects related to their successful formation of an abstract category of support relations (i.e., an object on another). Thirty-eight 8-month-old infants' eye movements were recorded during a support categorization task, where infants were habituated to four dynamic events depicting support relations (e.g., resting a block on a box) and then presented with test events that depicted either a support or containment relation with objects that they had seen or not seen in the habituation phase. Based on their looking time to the familiar versus novel spatial relation in the test, infants were classified into two groups: categorizers, who formed an abstract category of a support relation, and non-categorizers, who did not do so. During their initial phase of learning (i.e., the first habituation trial), categorizers paid greater attention to the object moved by a hand (i.e., the dynamic object) in comparison to non-categorizers, whereas their attention to the static object or their gaze shifts between the two objects did not differ. In addition, when presented with novel objects in a novel spatial relation after habituation, only categorizers displayed asymmetric attention between the objects, attending to the dynamic object more than the static object. Gaze shifts and attention to the concave area (i.e., hole) of the container did not differ between categorizers and non-categorizers. These findings suggest that infants' focused attention to an object in motion may play a key role in young infants' spatial category learning, and support the idea that attention to objects can assist with encoding of the spatial relational information.

Aerobic Fitness Unrelated to Acquisition of Spatial Relational Memory in College-Aged Adults.

While compelling evidence indicates that poorer aerobic fitness relates to impairments in retrieving information from hippocampal-dependent memory, there is a paucity of research on how aerobic fitness relates to the acquisition of such relational information. Accordingly, the present investigation examined the association between aerobic fitness and the rate of encoding spatial relational memory-assessed using a maximal oxygen consumption test and a spatial configuration task-in a sample of 152 college-aged adults. The findings from this investigation revealed no association between aerobic fitness and the acquisition of spatial relational memory. These findings have implications for how aerobic fitness is characterized with regard to memory, such that aerobic fitness does not appear to relate to the rate of learning spatial-relational information; however, given previously reported evidence, aerobic fitness may be associated with a greater ability to recall relational information from memory.

Neural Representation of Spatial Layout and Relational Information among Multiple Objects

Neural Representation of Spatial Layout and Relational Information among Multiple Objects

Tolerance to spatial-relational transformations in unfamiliar faces: A further challenge to a configural processing account of identity recognition

One of the most important questions in face perception research is to understand what information is extracted from a face in order to recognize its identity. Recognition of facial identity has been attributed to a special sensitivity to “configural” information. However, recent studies have challenged the configural account by showing that participants are poor in discriminating variations of metric distances among facial features, especially for familiar as opposed to unfamiliar faces, whereas a configural account predicts the opposite. We aimed to extend these previous results by examining classes of unfamiliar faces with which we have different levels of expertise. We hypothesized an inverse relation between sensitivity to configural information and expertise with a given class of faces, but only for neutral expressions. By first matching perceptual discriminability, we measured tolerance to subtle configural transformations with same-race (SR) versus other-race (OR) faces, and with upright versus upside-down faces. Consistently with our predictions, we found a lower sensitivity to at-threshold configural changes for SR compared to OR faces. We also found that, for our stimuli, the face inversion effect disappeared for neutral but not for emotional faces – a result that can also be attributed to a lower sensitivity to configural transformations for faces presented in a more familiar orientation. The present findings question a purely configural account of face processing and suggest that the role of spatial-relational information in face processing varies according to the functional demands of the task and to the characteristics of the stimuli.

Reconstructing relational information.

Hippocampal involvement in learning and remembering relational information has an extensive history, often focusing specifically on spatial information. In humans, spatial reconstruction (SR) paradigms are a powerful tool for evaluating an individuals' spatial-relational memory. In SR tasks, participants study locations of items in space and subsequently reconstruct the studied display after a short delay. Previous work has revealed that patients with hippocampal damage are impaired both in overall placement accuracy as well as on a specific measure of relational memory efficacy, "swaps" (i.e., when the relative location of two items is reversed). However, the necessity of the hippocampus for other types of spatial-relational information involved in reconstruction behaviors (e.g., where in the environment and relative to which other items an item was located) have not yet been investigated systematically. In this work, three patients with hippocampal damage and nine healthy matched comparison participants performed an SR task. An analysis framework was developed to independently assess three first-order types of relations: (1) memory for the binding of specific item identities to locations, (2) memory for arrangement of items in relation to each other or the environment bounds, regardless of memory for the item identity, and (3) higher-order, compound relational errors (i.e., errors involving multiple pieces of relational information). Reconstruction errors were evaluated to determine the degree to which patients and comparisons differed (or not) on each type of spatial-relational information. Data revealed that the primary group difference in performance was for identity-location information. However, when the locations of items were evaluated without regarding the identities, no group difference was found in the number of item placements to studied locations. The present work provides a principled approach to analysis of SR data and clarifies our understanding of the types of spatial relations impaired in hippocampal damaged.

Contextual object categorisation with energy‐based model

Object categorization is a hot issue of an image mining. Contextual information between objects is one of the important semantic knowledge of an image. However, the previous researches for an object categorization have not made full use of the contextual information, especially the spatial relations between objects. In addition, the object categorization methods, which generally use the probabilistic graphical models to implement the incorporation of contextual information with appearance of objects, are almost inevitable to evaluate the intractable partition function for normalization. In this work, we introduced fully-connected fuzzy spatial relations including directional, distance and topological relations between object regions, so the spatial relational information could be fully utilized. Then, the spatial relations were considered as well as co-occurrence and appearance of objects by using energy-based model, where the energy function was defined as the region-object association potential and the configuration potential of objects. Minimizing the energy function of whole image arrangement, we obtained the optimal label set about the image regions and addressed the evaluation of intractable partition function in conditional random fields. Experimental results show the validity and reliability of this proposed method.

Static spatial descriptions in five Germanic languages

The paper presents qualitative and quantitative analyses of expressions describing static topological relations in Frisian, Icelandic, and Norwegian (Bokmål), Swiss German, and Standard German. According to the literature, speakers of Germanic languages typically express more than just the bare minimum of spatial relational information when describing spatial scenes. As an example, they can add postural or other manner information as an unmarked choice in their spatial descriptions (the cup stands on the table vs. the cup is on the table). The main focus lies on a detailed description of the modalities and proportions of this additional expression of information on the spatial scene in the five languages. Distributed expression of spatial relational semantics, posture verb usage and resultative constructions are analyzed. Descriptive and inferential methods are used to show the similarities and differences across the five languages, on the level of group tendencies (means), of individual speakers, and of individual stimulus items. The analyses show considerable differences across the languages. Speakers of Standard German and Frisian are most prone to integrate additional manner information into the descriptions, whereas speakers of Icelandic, Norwegian or Swiss German only rarely integrate this kind of information into their spatial descriptions.

Maintenance of relational information in working memory leads to suppression of the sensory cortex.

Working memory (WM) for sensory-based information about individual objects and their locations appears to involve interactions between lateral prefrontal and sensory cortexes. The mechanisms and representations for maintenance of more abstract, nonsensory information in WM are unknown, particularly whether such actively maintained information can become independent of the sensory information from which it was derived. Previous studies of WM for individual visual items found increased electroencephalogram (EEG) alpha (8-13 Hz) power over posterior electrode sites, which appears to correspond to the suppression of cortical areas that represent irrelevant sensory information. Here, we recorded EEG while participants performed a visual WM task that involved maintaining either concrete spatial coordinates or abstract relational information. Maintenance of relational information resulted in higher alpha power in posterior electrodes. Furthermore, lateralization of alpha power due to a covert shift of attention to one visual hemifield was marginally weaker during storage of relational information than during storage of concrete information. These results suggest that abstract relational information is maintained in WM differently from concrete, sensory representations and that during maintenance of abstract information, posterior sensory regions become task irrelevant and are thus suppressed.

Bayesian network based semantic image classification with attributed relational graph

Semantic image classification is a hot issue of image mining. Information of spatial relations between objects in an image is one of the important semantic information of an image. However, the previous researches have not made full use of the spatial relations for image modeling and classification. In addition, to classify the images with Bayesian network, the accuracy of conditional probability estimation may be insufficient, because the learning methods of spatial contextual models have usually used a limited number of training samples. In this work, the semantic image modeling based on attributed relational graph has been proposed, in which the distance measure method between images was presented, therefore the object information and spatial relational information could be fully utilized. Then, the semantic distance between images based on attributed relational graph could be calculated for the support vector machine to obtain the joint conditional probability distribution of Bayesian network. Therefore the probabilistic estimation problem under the sparse training samples could be solved, and the accuracy of semantic image classification with Bayesian network was improved. Experimental results show the validity and reliability of this proposed method.

Think spatial: The representation in mental rotation is nonvisual.

For mental rotation, introspection, theories, and interpretations of experimental results imply a certain type of mental representation, namely, visual mental images. Characteristics of the rotated representation can be examined by measuring the influence of stimulus characteristics on rotational speed. If the amount of a given type of information influences rotational speed, one can infer that it was contained in the rotated representation. In Experiment 1, rotational speed of university students (10 men, 11 women) was found to be influenced exclusively by the amount of represented orientation-dependent spatial-relational information but not by orientation-independent spatial-relational information, visual complexity, or the number of stimulus parts. As information in mental-rotation tasks is initially presented visually, this finding implies that at some point during each trial, orientation-dependent information is extracted from visual information. Searching for more direct evidence for this extraction, we recorded the EEG of another sample of university students (12 men, 12 women) during mental rotation of the same stimuli. In an early time window, the observed working memory load-dependent slow potentials were sensitive to the stimuli's visual complexity. Later, in contrast, slow potentials were sensitive to the amount of orientation-dependent information only. We conclude that only orientation-dependent information is contained in the rotated representation. (PsycINFO Database Record (c) 2013 APA, all rights reserved).

Spatial Cognition and the Hippocampus: The Anterior–Posterior Axis

We discuss the question of differentiation along the anterior-posterior longitudinal axis of the hippocampus. Data from a recent fMRI study are reanalyzed to determine whether activations in these hippocampal regions are affected by the nature of the information being accessed during a scanning session in which participants thought about episodes from their lives. Retrieving detailed spatial relational information preferentially activated the posterior hippocampus, whereas retrieving information about locales (or contexts) preferentially activated the anterior hippocampus. These data support the view that there is functional differentiation along the longitudinal axis in humans that matches what has been seen in rats, namely, that the posterior (dorsal) hippocampus is crucial for precise spatial behavior, and the anterior (ventral) hippocampus is crucial for context coding.

Spatial relations and spatial locations are dissociated within prefrontal and parietal cortex.

Item-specific spatial information is essential for interacting with objects and for binding multiple features of an object together. Spatial relational information is necessary for implicit tasks such as recognizing objects or scenes from different views but also for explicit reasoning about space such as planning a route with a map and for other distinctively human traits such as tool construction. To better understand how the brain supports these two different kinds of information, we used functional MRI to directly contrast the neural encoding and maintenance of spatial relations with that for item locations in equivalent visual scenes. We found a double dissociation between the two: whereas item-specific processing implicates a frontoparietal attention network, including the superior frontal sulcus and intraparietal sulcus, relational processing preferentially recruits a cognitive control network, particularly lateral prefrontal cortex (PFC) and inferior parietal lobule. Moreover, pattern classification revealed that the actual meaning of the relation can be decoded within these same regions, most clearly in rostrolateral PFC, supporting a hierarchical, representational account of prefrontal organization.

Spatial relational learning persists following neonatal hippocampal lesions in macaque monkeys

The hippocampus is important for the acquisition of spatial representations of the environment and consequently in contextual memory. This suggests that the neural substrates underlying spatial cognition might be essential for remembering specific life episodes. Indeed, hippocampal lesions prevent spatial relational learning in adult rodents and monkeys, and result in profound amnesia in adult humans. In contrast, we show here that monkeys with neonatal hippocampal lesions learned new spatial relational information. Our experiments suggest that early hippocampal damage leads to functional brain reorganization that enables spatial information to be acquired through the use of brain regions that normally do not subserve this function.

Different spatial-relational information is used to recognise faces and emotional expressions.

In a face photo in which the two eyes have been moved up into the forehead region, configural spatial relations are altered more than categorical relations; in a photo in which only one eye is moved up, categorical relations are altered more. Matching the identities of two faces was slower when an unaltered photo was paired with a two-eyes-moved photo than when paired with a one-eye-moved photo, implicating configural relations in face identification. But matching the emotional expressions of the same faces was slower when an unaltered photo was paired with a one-eye-moved photo than when paired with a two-eyes-moved photo, showing that expression recognition uses categorically coded relations. The findings also indicate that changing spatial-relational information affects the perceptual encoding of identities and expressions rather than their memory representations.

Revisiting the perception of upside-down faces.

In two experiments, the effect of orientation on face perception was assessed. Using a scale from 1 (normal) to 7 (bizarre), participants rated normal, unaltered faces and faces in which changes had been made to spatial-relational properties (eyes and mouth inverted or relative position of the eyes and mouth altered) or to component properties (eyes whitened and teeth blackened). For unaltered and component-distortion faces, bizarreness ratings increased linearly as orientation increased from 0 degree to 180 degrees. For spatial-distortion faces, a discontinuity in the function relating orientation and bizarreness was in evidence between 90 degrees and 120 degrees. The results provide support for the view that there is a qualitative difference in the processing of upright and inverted faces due to the disproportionate effect of inversion on the encoding of spatial-relational information.

Metaphoric structuring: understanding time through spatial metaphors

The present paper evaluates the claim that abstract conceptual domains are structured through metaphorical mappings from domains grounded directly in experience. In particular, the paper asks whether the abstract domain of time gets its relational structure from the more concrete domain of space. Relational similarities between space and time are outlined along with several explanations of how these similarities may have arisen. Three experiments designed to distinguish between these explanations are described. The results indicate that (1) the domains of space and time do share conceptual structure, (2) spatial relational information is just as useful for thinking about time as temporal information, and (3) with frequent use, mappings between space and time come to be stored in the domain of time and so thinking about time does not necessarily require access to spatial schemas. These findings provide some of the first empirical evidence for Metaphoric Structuring. It appears that abstract domains such as time are indeed shaped by metaphorical mappings from more concrete and experiential domains such as space.

Inversion and processing of component and spatial–relational information in faces.

Inversion and processing of component and spatial–relational information in faces.