Regions In Scene Research Articles

Forensic analysis of microtraces using image recognition through machine learning

Traces found at a crime scene can be interpreted as vectors of information that help describe the possible dynamics of the crime. However, some analyses show that pattern recognition, especially in materials, is subjective, as it depends on the analyst’s references. Based on this problem, the work aimed to use different statistical methods to establish pattern recognition in silver tapes. For this, two approaches were used, one based on deep learning (convolutional neural networks) and the other using a different method (Pearson’s correlation, distance metrics, and Principal Component Analysis). The dataset comprised four brands of silver-tape available in the retail market in the crime scene region, and fragments of tape originating after the detonation of a handmade explosive device. These materials were analyzed using a Leica DVM6 microscope. In both approaches, it was possible to recognize patterns. In deep learning, it was possible to establish that the fragments came from a common origin. The best model demonstrated that 92.1 % of the real materials questioned were the same silver-tape, with a confidence level of 0.94. By combining the methods, it was possible to observe a trend among the results. These responses demonstrated that using images in a computer vision context could remove the subjectivity of forensic analysis and correlate the microtraces found at a crime scene. In this way, these techniques can open new perspectives for the forensic area, making the interpretation more objective and transparent in its responses to society.

MRAM: Multi-scale Regional Attribute-weighting via Meta-learning for Personalized Image Aesthetics Assessment

Image aesthetics assessment (IAA) is inherently subjective, with generic image aesthetics assessment (GIAA) often overlooking individual user preferences. Personalized aesthetic assessment (PIAA) addresses this by accounting for user-specific inconsistencies. However, the limited annotated data from individual users and the scarcity of user-jointly annotated image data pose challenges. To cope with these problems, we propose a multi-scale regional attribute-weighting via meta-learning for PIAA, namely MRAM, which aims to simulate how the brain computes aesthetic values from visual stimuli. The human brain is to predict the subjective aesthetic value of the individual to the image by combining the features of the visual scene segmentation through the weighting method. MRAM introduces a multi-scale attention module to capture visual scene regions segmented at different scales, adjusting feature selection based on user attention. Additionally, a regional attribute-weighting module (RAM) is proposed to weigh these features using aesthetic attribute representations. In the meta-training phase, MRAM learns aesthetic prior knowledge from a diverse set of PIAA tasks. Transitioning to the personalization phase, MRAM is fine-tuned with only a small number of specific user samples, enabling it to rapidly adapt to the unique aesthetic preferences of individuals. Experimental results demonstrate the superior performance of the proposed MRAM compared to state-of-the-art PIAA methods.

An improved you only look once algorithm for pronuclei and blastomeres localization

Pronuclei and blastomeres are key structures in early embryonic development, and by localizing these structures simultaneously, the developmental state of the embryo can be assessed more comprehensively. However, there are several unavoidable problems in localization tasks due to the biological characteristics of embryos, including blastomeres overlap, pronuclei overlap, and similarity between pronuclei and background. In this study, we propose a novel localization network for pronuclei and blastomeres, which can solve these localization problems. Firstly, to address the issues related to the overlap of pronuclei and blastomeres, as well as the pronuclei and background similarity problem, we put the vision transformer with bi-level routing attention module (BiFormer) in the backbone. The BiFormer finds attention regions in the embryo image scene to obtain more edge and texture information of both pronuclei and blastomeres, which allows for a better feature realization of the neck region fusion interaction. Subsequently, to enhance model performance and mitigate redundant computation. The localization network uses partial convolution (PConv) in the backbone. The backbone network allows more efficient extraction of features by simultaneously reducing redundancy in computation with the effect of PConv. In addition, to mitigate the impact of low-quality samples in embryo images on localization, as well as to pay more attention to ordinary quality samples, we use wise intersection over union version 3 (WIoUv3) loss function with a dynamic non-monotonic focusing mechanism in the localization network, thus improving the overall performance of the algorithm. The experimental results show our model mAP@0.5 is 92.4% in localizing pronuclei and blastomeres. In practical terms, the ability to accurately localize pronuclei and blastomeres allows for better assessment of embryo quality and selects the best embryos.

SFD-SLAM: a novel dynamic RGB-D SLAM based on saliency region detection

Abstract In dynamic environments, several simultaneous localization and mapping (SLAM) systems effectively utilize optical flow fields to distinguish dynamic from static feature points. Commonly, these systems leverage the amplitude information within the optical flow field to develop adaptive thresholding segmentation models for identifying dynamic scene regions. Nevertheless, designing adaptive thresholding models typically necessitates meticulous planning and extensive experimentation. This study introduces a dynamic RGBD SLAM system, SFD-SLAM, which innovates by employing a saliency detection network for the direct extraction of dynamic regions via scene flow. This approach notably streamlines the design process associated with conventional adaptive thresholding models. Furthermore, SFD-SLAM incorporates a geometric module that merges depth residuals with hyperpixel segmentation to enhance the refinement of the dynamic mask. This is followed by integration with FCM clustering for the precise identification of moving objects. The efficacy of SFD-SLAM is assessed using the widely recognized TUM dynamic dataset. Experimental results demonstrate that the proposed system surpasses DGFlow-SLAM, which relies on an adaptive thresholding model for dynamic object segmentation, in terms of trajectory accuracy. It also achieves comparable localization accuracy to DynaSLAM. Moreover, SFD-SLAM maintains robust tracking capabilities, even in scenarios where DynaSLAM experiences tracking loss, thereby augmenting the robustness of RGBD-SLAM in dynamic settings.

Familiarity enhances functional connectivity between visual and nonvisual regions of the brain during natural viewing.

We explored the neural correlates of familiarity with people and places using a naturalistic viewing paradigm. Neural responses were measured using functional magnetic resonance imaging, while participants viewed a movie taken from Game of Thrones. We compared inter-subject correlations and functional connectivity in participants who were either familiar or unfamiliar with the TV series. Higher inter-subject correlations were found between familiar participants in regions, beyond the visual brain, that are typically associated with the processing of semantic, episodic, and affective information. However, familiarity also increased functional connectivity between face and scene regions in the visual brain and the nonvisual regions of the familiarity network. To determine whether these regions play an important role in face recognition, we measured responses in participants with developmental prosopagnosia (DP). Consistent with a deficit in face recognition, the effect of familiarity was significantly attenuated across the familiarity network in DP. The effect of familiarity on functional connectivity between face regions and the familiarity network was also attenuated in DP. These results show that the neural response to familiarity involves an extended network of brain regions and that functional connectivity between visual and nonvisual regions of the brain plays an important role in the recognition of people and places during natural viewing.

Immersive scene representation in human visual cortex with ultra-wide-angle neuroimaging

While human vision spans 220°, traditional functional MRI setups display images only up to central 10-15°. Thus, it remains unknown how the brain represents a scene perceived across the full visual field. Here, we introduce a method for ultra-wide angle display and probe signatures of immersive scene representation. An unobstructed view of 175° is achieved by bouncing the projected image off angled-mirrors onto a custom-built curved screen. To avoid perceptual distortion, scenes are created with wide field-of-view from custom virtual environments. We find that immersive scene representation drives medial cortex with far-peripheral preferences, but shows minimal modulation in classic scene regions. Further, scene and face-selective regions maintain their content preferences even with extreme far-periphery stimulation, highlighting that not all far-peripheral information is automatically integrated into scene regions computations. This work provides clarifying evidence on content vs. peripheral preferences in scene representation and opens new avenues to research immersive vision.

Foreground bias: Semantic consistency effects modulated when searching across depth.

When processing visual scenes, we tend to prioritize information in the foreground, often at the expense of background information. The foreground bias has been supported by data demonstrating that there are more fixations to foreground, and faster and more accurate detection of targets embedded in foreground. However, it is also known that semantic consistency is associated with more efficient search. Here, we examined whether semantic context interacts with foreground prioritization, either amplifying or mitigating the effect of target semantic consistency. For each scene, targets were placed in the foreground or background and were either semantically consistent or inconsistent with the context of immediately surrounding depth region. Results indicated faster response times (RTs) for foreground and semantically consistent targets, replicating established effects. More importantly, we found the magnitude of the semantic consistency effect was significantly smaller in the foreground than background region. To examine the robustness of this effect, in Experiment 2, we strengthened the reliability of semantics by increasing the proportion of targets consistent with the scene region to 80%. We found the overall results pattern to replicate the incongruous effect of semantic consistency across depth observed in Experiment 1. This suggests foreground bias modulates the effects of semantics so that performance is less impacted in near space.

ScanERU: Interactive 3D Visual Grounding Based on Embodied Reference Understanding

Aiming to link natural language descriptions to specific regions in a 3D scene represented as 3D point clouds, 3D visual grounding is a very fundamental task for human-robot interaction. The recognition errors can significantly impact the overall accuracy and then degrade the operation of AI systems. Despite their effectiveness, existing methods suffer from the difficulty of low recognition accuracy in cases of multiple adjacent objects with similar appearance. To address this issue, this work intuitively introduces the human-robot interaction as a cue to facilitate the development of 3D visual grounding. Specifically, a new task termed Embodied Reference Understanding (ERU) is first designed for this concern. Then a new dataset called ScanERU is constructed to evaluate the effectiveness of this idea. Different from existing datasets, our ScanERU dataset is the first to cover semi-synthetic scene integration with textual, real-world visual, and synthetic gestural information. Additionally, this paper formulates a heuristic framework based on attention mechanisms and human body movements to enlighten the research of ERU. Experimental results demonstrate the superiority of the proposed method, especially in the recognition of multiple identical objects. Our codes and dataset are available in the ScanERU repository.

NeRF-In: Free-Form Inpainting for Pretrained NeRF With RGB-D Priors.

Neural radiance field (NeRF) has emerged as a versatile scene representation. However, it is still unintuitive to edit a pretrained NeRF because the network parameters and the scene appearance are often not explicitly associated. In this article, we introduce the first framework that enables users to retouch undesired regions in a pretrained NeRF scene without accessing any training data and category-specific data prior. The user first draws a free-form mask to specify a region containing the unwanted objects over an arbitrary rendered view from the pretrained NeRF. Our framework transfers the user-drawn mask to other rendered views and estimates guiding color and depth images within transferred masked regions. Next, we formulate an optimization problem that jointly inpaints the image content in all masked regions by updating NeRF's parameters. We demonstrate our framework on diverse scenes and show it obtained visually plausible and structurally consistent results using less user manual efforts.

Mapping the functional and structural connectivity of the scene network.

The recognition and perception of places has been linked to a network of scene-selective regions in the human brain. While previous studies have focussed on functional connectivity between scene-selective regions themselves, less is known about their connectivity with other cortical and subcortical regions in the brain. Here, we determine the functional and structural connectivity profile of the scene network. We used fMRI to examine functional connectivity between scene regions and across the whole brain during rest and movie-watching. Connectivity within the scene network revealed a bias between posterior and anterior scene regions implicated in perceptual and mnemonic aspects of scene perception respectively. Differences between posterior and anterior scene regions were also evident in the connectivity with cortical and subcortical regions across the brain. For example, the Occipital Place Area (OPA) and posterior Parahippocampal Place Area (PPA) showed greater connectivity with visual and dorsal attention networks, whileanterior PPA and Retrosplenial Complex showed preferential connectivity with default mode and frontoparietal control networks and the hippocampus. We further measured the structural connectivity of the scene network using diffusion tractography. This indicated both similarities and differences with the functional connectivity, highlighting biases between posterior and anterior regions, but also between ventral and dorsal scene regions. Finally, we quantified the structural connectivity between the scene network and major white matter tracts throughout the brain. These findings provide a map of the functional and structural connectivity of scene-selective regions to each other and the rest of the brain.

The role of local meaning in infants' fixations of natural scenes.

As infants view visual scenes every day, they must shift their eye gaze and visual attention from location to location, sampling information to process and learn. Like adults, infants' gaze when viewing natural scenes (i.e., photographs of everyday scenes) is influenced by the physical features of the scene image and a general bias to look more centrally in a scene. However, it is unknown how infants' gaze while viewing such scenes is influenced by the semantic content of the scenes. Here, we tested the relative influence of local meaning, controlling for physical salience and center bias, on the eye gaze of 4- to 12-month-old infants (N=92) as they viewed natural scenes. Overall, infants were more likely to fixate scene regions rated as higher in meaning, indicating that, like adults, the semantic content, or local meaning, of scenes influences where they look. More importantly, the effect of meaning on infant attention increased with age, providing the first evidence for an age-related increase in the impact of local meaning on infants' eye movements while viewing natural scenes.

From vision to memory: How scene-sensitive regions support episodic memory formation during child development

Previous brain imaging studies have identified three brain regions that selectively respond to visual scenes, the parahippocampal place area (PPA), the occipital place area (OPA), and the retrosplenial cortex (RSC). There is growing evidence that these visual scene-sensitive regions process different types of scene information and may have different developmental timelines in supporting scene perception. How these scene-sensitive regions support memory functions during child development is largely unknown. We investigated PPA, OPA and RSC activations associated with episodic memory formation in childhood (5–7 years of age) and young adulthood, using a subsequent scene memory paradigm and a functional localizer for scenes. PPA, OPA, and RSC subsequent memory activation and functional connectivity differed between children and adults. Subsequent memory effects were found in activations of all three scene regions in adults. In children, however, robust subsequent memory effects were only found in the PPA. Functional connectivity during successful encoding was significant among the three regions in adults, but not in children. PPA subsequently memory activations and PPA-RSC subsequent memory functional connectivity correlated with accuracy in adults, but not children. These age-related differences add new evidence linking protracted development of the scene-sensitive regions to the protracted development of episodic memory.

Scene-aware Foveated Neural Radiance Fields.

Foveated rendering provides an idea for improving the image synthesis performance of neural radiance fields (NeRF) methods. In this paper, we propose a scene-aware foveated neural radiance fields method to synthesize high-quality foveated images in complex VR scenes at high frame rates. Firstly, we construct a multi-ellipsoidal neural representation to enhance the neural radiance field's representation capability in salient regions of complex VR scenes based on the scene content. Then, we introduce a uniform sampling based foveated neural radiance field framework to improve the foveated image synthesis performance with one-pass color inference, and improve the synthesis quality by leveraging the foveated scene-aware objective function. Our method synthesizes high-quality binocular foveated images at the average frame rate of 66 frames per second (FPS) in complex scenes with high occlusion, intricate textures, and sophisticated geometries. Compared with the state-of-the-art foveated NeRF method, our method achieves significantly higher synthesis quality in both the foveal and peripheral regions with 1.41-1.46× speedup. We also conduct a user study to prove that the perceived quality of our method has a high visual similarity with the ground truth.

A Study of Folk Culture in Francophone Central Africa Incorporating Deep Learning

Abstract Through the theoretical analysis of folk culture in French-speaking countries in Central Africa, six-dimensional features of folk culture in non-French-speaking countries are summarized, and the folk culture database of French-speaking countries in Central Africa is constructed based on a web crawler. Fusing scene region features with image color and texture features, an Extreme Learning Machine algorithm carries out the classification to realize the construction of a folk culture classification model based on deep learning, and the cross-entropy loss function and Adam optimizer are used respectively to optimize the above-constructed model. The relevant parameters, training set, and test set are determined, and an example analysis of non-Chinese folk culture from the perspective of deep learning is carried out. It is found that the accuracy of non-folk culture classification of the model after Adam and cross-entropy optimization is significantly higher than that of the CNN model, with an average of 0.03~0.04, indicating that after optimization, the model can more comprehensively reflect non-French-speaking countries’ folk culture information and features in images. In addition, there are fewer symbols of non-French-speaking folk culture in the image modality, such as religious beliefs (17, 4.57%) and music and dance (32, 8.60%), which is because it is difficult to represent this type of folk culture with images. This paper fully illustrates the prospects for the application of deep learning models in the study of folk culture in French-speaking Central Africa, which can contribute to the development of digitization and informatization of folk culture in French-speaking Central Africa.

Real Aperture Radar Super-Resolution Imaging for Sea Surface Monitoring Based on a Hybrid Model

In recent years, super-resolution imaging techniques have been intensely introduced to enhance the azimuth resolution of real aperture scanning radar (RASR). However, there is a paucity of research on the subject of sea surface imaging with small incident angles for complex scenarios. This research endeavors to explore super-resolution imaging for sea surface monitoring, with a specific emphasis on grounded or shipborne platforms. To tackle the inescapable interference of sea clutter, it was segregated from the imaging objects and was modeled alongside I/Q channel noise within the maximum likelihood framework, thus mitigating clutter’s impact. Simultaneously, for characterizing the non-stationary regions of the monitoring scene, we harnessed the Markov random field (MRF) model for its two-dimensional (2D) spatial representational capacity, augmented by a quadratic term to bolster outlier resilience. Subsequently, the maximum a posteriori (MAP) criterion was employed to unite the ML function with the statistical model regarding imaging scene. This hybrid model forms the core of our super-resolution methodology. Finally, a fast iterative threshold shrinkage method was applied to solve this objective function, yielding stable estimates of the monitored scene. Through the validation of simulation and real data experiments, the superiority of the proposed approach in recovering the monitoring scenes and clutter suppression has been verified.

CSFFNet: Lightweight cross‐scale feature fusion network for salient object detection in remote sensing images

AbstractSalient object detection (SOD), one of the most important applications in the field of computer vision, aims to extract the most visually appealing regions of scenes. However, the improvement of the accuracy of existing salient object detection in optical remote sensing images (ORSI‐SOD) is usually accompanied by an increase of network complexity, which affects the application of these models. Motivated by this, a novel lightweight edge‐supervised neural network for ORSI‐SOD is proposed, named CSFFNet. Specifically, the backbone (ResNet34) is first lightened by feature encoding module (FEM), building a lightweight subnet for feature extraction. Then, in the transformer‐based feature pyramid enhancement module (FPEM), the convolutional features obtained in the FEM are enhanced by long‐distance dependence to obtain multi‐scale features containing rich saliency cues. Based on this, the feature fusion module (FFM) is designed to capture cross‐scale long‐range dependencies and effectively fuse high‐level semantic information with low‐level detail information. Thus, the increase in network complexity due to multi‐level decoding is avoided. Finally, the segmentation results are optimized by using salient edges as auxiliary information, which effectively improves the contrast and completeness of the results. Experimental results on two public datasets demonstrate that the lightweight CSFFNet achieves competitive or even better performance compared with state‐of‐the‐art methods.

The Development of Human Cortical Scene Processing

Decades of research have uncovered the neural basis of place (or “scene”) processing in adulthood, revealing a set of three regions that respond selectively to visual scene information, each hypothesized to support distinct functions within scene processing (e.g., recognizing a particular kind of place versus navigating through it). Despite this considerable progress, surprisingly little is known about how these cortical regions develop. Here we review the limited evidence to date, highlighting the first few studies exploring the origins of cortical scene processing in infancy and the several studies addressing when the scene regions reach full maturity, unfortunately with inconsistent findings. This inconsistency likely stems from common pitfalls in pediatric functional magnetic resonance imaging, and accordingly, we discuss how these pitfalls may be avoided. Furthermore, we point out that almost all studies to date have focused only on general scene selectivity and argue that greater insight could be gleaned by instead exploring the more distinct functions of each region as well as their connectivity. Finally, with this last point in mind, we offer a novel hypothesis that scene regions supporting navigation (including the occipital place area and retrosplenial complex) mature later than those supporting scene categorization (including the parahippocampal place area).

Relational memory weakness in autism despite the use of a controlled encoding task.

Recent work challenged past findings that documented relational memory impairments in autism. Previous studies often relied solely on explicit behavioral responses to assess relational memory integrity, but successful performance on behavioral tasks may rely on other cognitive abilities (e.g., executive functioning) that are impacted in some autistic individuals. Eye-tracking tasks do not require explicit behavioral responses, and, further, eye movements provide an indirect measure of memory. The current study examined whether memory-specific viewing patterns toward scenes differ between autistic and non-autistic individuals. Using a long-term memory paradigm that equated for complexity between item and relational memory tasks, participants studied a series of scenes. Following the initial study phase, scenes were re-presented, accompanied by an orienting question that directed participants to attend to either features of an item (i.e., in the item condition) or spatial relationships between items (i.e., in the relational condition) that might be subsequently modified during test. At test, participants viewed scenes that were unchanged (i.e., repeated from study), scenes that underwent an "item" modification (an exemplar switch) or a "relational" modification (a location switch), and scenes that had not been presented before. Eye movements were recorded throughout. During study, there were no significant group differences in viewing directed to regions of scenes that might be manipulated at test, suggesting comparable processing of scene details during encoding. However, there was a group difference in explicit recognition accuracy for scenes that underwent a relational change. Marginal group differences in the expression of memory-based viewing effects during test for relational scenes were consistent with this behavioral outcome, particularly when analyses were limited to scenes recognized correctly with high confidence. Group differences were also evident in correlational analyses that examined the association between study phase viewing and recognition accuracy and between performance on the Picture Sequence Memory Test and recognition accuracy. Together, our findings suggest differences in the integrity of relational memory representations and/or in the relationships between subcomponents of memory in autism.

The Spatial Precision of Contextual Feedback Signals in Human V1.

Neurons in the primary visual cortex (V1) receive sensory inputs that describe small, local regions of the visual scene and cortical feedback inputs from higher visual areas processing the global scene context. Investigating the spatial precision of this visual contextual modulation will contribute to our understanding of the functional role of cortical feedback inputs in perceptual computations. We used human functional magnetic resonance imaging (fMRI) to test the spatial precision of contextual feedback inputs to V1 during natural scene processing. We measured brain activity patterns in the stimulated regions of V1 and in regions that we blocked from direct feedforward input, receiving information only from non-feedforward (i.e., feedback and lateral) inputs. We measured the spatial precision of contextual feedback signals by generalising brain activity patterns across parametrically spatially displaced versions of identical images using an MVPA cross-classification approach. We found that fMRI activity patterns in cortical feedback signals predicted our scene-specific features in V1 with a precision of approximately 4 degrees. The stimulated regions of V1 carried more precise scene information than non-stimulated regions; however, these regions also contained information patterns that generalised up to 4 degrees. This result shows that contextual signals relating to the global scene are similarly fed back to V1 when feedforward inputs are either present or absent. Our results are in line with contextual feedback signals from extrastriate areas to V1, describing global scene information and contributing to perceptual computations such as the hierarchical representation of feature boundaries within natural scenes.

PA-Net: Plane Attention Network for real-time urban scene reconstruction

Traditional urban reconstruction methods can only output incomplete 3D models, which depict the scene regions that are visible to the moving camera. While learning-based shape reconstruction techniques make single-view 3D reconstruction possible, they are designed to handle single objects that are well-presented in the training datasets. This paper presents a novel learning-based approach for reconstructing complete 3D meshes for large-scale urban scenes in real-time. The input video sequences are fed into a localization module, which segments different objects and determines their relative positions. Each object is then reconstructed under their local coordinates to better approximate models in the training datasets. The reconstruction module is adopted from BSP-Net (Chen et al., 2020), which is capable of producing compact polygon meshes. However, major changes have been made so that unoriented objects in large-scale scenes can be reconstructed efficiently using only a small number of planes. Experimental results demonstrate that our approach can reconstruct urban scenes with buildings and vehicles using 400∼800 convex parts in 0.1∼0.5 s.