Scene Images Research Articles

No evidence for a privileged role of global ensemble statistics in rapid scene perception: A registered replication attempt.

The nature of visual processes underlying scene perception remains a hotly debated topic. According to one view, scene and object perception rely on similar neural mechanisms, and their processing pathways are tightly interlinked. According to another, scene gist might follow a separate pathway, relying primarily on global image properties. Recently, this latter idea has been supported with a set of experiments using content priming as a probe into scene and object perception (Brady et al. Journal of Experimental Psychology: Human Perception and Performance, 43, 1160-1176, 2017). The experiments have shown that preserving only structureless global ensemble texture information in the images of scenes could support rapid scene perception; however, preserving the same information in the images of objects failed to support object perception. We were intrigued by these results, since they are at odds with findings showing that scene content is primarily carried by the explicit encoding of scene structure as represented, for instance, by contours and their properties. In an attempt to reconcile these results, we attempted to replicate the experiments. In our replication experiment, we failed to find any evidence for a privileged use of texture information for scene as opposed to object primes. We conclude that there is no sufficient evidence for any fundamental differences in the processing pathways for object and scene perception: both rely on structural features that describe spatial relationships between constituent parts as well as texture information. To address this issue in the most rigorous manner possible, we here present the results of both a pilot experiment and a pre-registered replication attempt.

Domain-specific representation of social inference by neurons in the human amygdala and hippocampus.

Inferring the intentions and emotions of others from behavior is crucial for social cognition. While neuroimaging studies have identified brain regions involved in social inference, it remains unknown whether performing social inference is an abstract computation that generalizes across different stimulus categories or is specific to certain stimulus domain. We recorded single-neuron activity from the medial temporal lobe (MTL) and the medial frontal cortex (MFC) in neurosurgical patients performing different types of inferences from images of faces, hands, and natural scenes. Our findings indicate distinct neuron populations in both regions encoding inference type for social (faces, hands) and nonsocial (scenes) stimuli, while stimulus category was itself represented in a task-general manner. Uniquely in the MTL, social inference type was represented by separate subsets of neurons for faces and hands, suggesting a domain-specific representation. These results reveal evidence for specialized social inference processes in the MTL, in which inference representations were entangled with stimulus type as expected from a domain-specific process.

Generation of Warfighter Avatars from Weapon Training Scene Images for Blast Exposure Simulations

Generation of Warfighter Avatars from Weapon Training Scene Images for Blast Exposure Simulations

Correlation reconstruction mechanism based on dual wavelength imaging and neural network

In a natural light-field imaging scene, lighting is usually performed using a mixed light field of multiple wavelengths. To better meet the requirements of human vision, this study proposes a correlation reconstruction method based on dual-wavelength imaging and a neural network (compression sensing correlation fusion-reconstruction by dual-wavelength imaging and auto-encoder neural network, CSCFR-DWI-AENN). Two different wavelengths of light are mixed through an optical multiplexer unit (OMU) to form a dual-wavelength illumination light field, and the light intensity value reflected by the object is received by the detector. The object information was reconstructed using the compressed sensing ghost imaging algorithm, and the distribution of the illumination field was taken as prior information. Two single-wavelength illumination information and the detection value are used for compressed sensing reconstruction, and the high-quality reconstructed image of the target to be measured is obtained by combining the noise reduction advantage of the autoencoder neural network. Then the target information is fused by the New Sum of Modified Laplacian (NSML) algorithm. Two single-wavelength reconstruction, dual-wavelength direct reconstruction and the proposed reconstruction were compared respectively, and non-reference image quality evaluation functions such as EN, MI, EAV and SF were used to process and analyze the reconstruction effect. It is proved that this algorithm can reconstruct the object image comprehensively, reproduce the complete detail information, and achieve a more accurate, more comprehensive and more reliable description of the same object. It provides a theoretical basis for multi-wavelength imaging technology and has a good application prospect.

Remote sensing image scene classification based on residual dense network

Remote sensing image scene classification based on residual dense network

National estimation of regulated water storage of reservoirs in China

The need for man-made reservoirs has increased with the growing population and changing climate. One of the most intensive human-induced alterations of the hydrological cycle is the regulated water storage in reservoirs, however, its quantification in large-scale reservoirs in China is inadequate. The lack of such information limits the rational management and utilization of water resources in reservoirs. To address this issue, we first mapped the dense time series of high-resolution (10 m) water inundation areas based on Sentinel-2 image scenes from 2017 to 2022 and then composed the annual surface water frequency (WF) maps of Chinese reservoirs. We calculated the water area of each reservoir at different WFs and estimated the corresponding water storage using our previously developed machine learning model. The water storage difference between the maximum water area (WF ≥ 5 %) and year-long (WF ≥ 75 %) water area was assumed as the regulated water storage of reservoirs. Results show that the multiyear-averaged water storage of all Chinese reservoirs at the maximum water area is estimated at 784.60 km3 from 2017 to 2022, and the water storage at the year-long water area is approximately 318.86 km3. The average annual regulated water storage of the reservoirs is estimated at 465.74 km3, accounting for 43.72 % of the total designed water storage of reservoirs in China. Among the basins, the Yangtze River Basin and Pearl River Basin have the highest average annual regulated water storage of 152.52 and 84.94 km3, respectively. The basins with larger irrigation areas and higher populations tend to have higher reservoir regulated water storage. The normalized water storage (NS) shows high spatial variations across reservoirs in different basins, with most basins in the south having higher NS than those in the north.

Adaptive multi-resolution single-pixel imaging based on local transform

Single-pixel imaging involves a trade-off between imaging quality and efficiency due to its sequential detection, particularly challenging for key areas under large frame sizes. We propose a novel mathematical model for multi-resolution single-pixel imaging based on orthogonal local Hadamard transform. To adaptively determine the center of key areas, we employ a dual strategy combining attention maps and a novel sparse geometric moment approach. Our method executes multi-resolution imaging across various levels using multi-resolution pattern sequences. This strategy enhances the imaging quality of key areas while significantly reducing computational burden. Through simulations and experiments, we demonstrate that our adaptive multi-resolution SPI yields superior imaging results in large frame size scenarios and sparse imaging scenes, effectively addressing the quality-efficiency trade-off in single-pixel imaging.

Real-World Scene Image Enhancement with Contrastive Domain Adaptation Learning

Image enhancement methods leveraging learning-based approaches have demonstrated impressive results when trained on synthetic degraded-clear image pairs. However, when deployed in real-world scenarios, such models often suffer significant performance degradation due to the inherent domain gap between synthetic and real degradations. To bridge this gap, we propose a novel Two-stage Contrastive Domain Adaptation image Enhancement (TCDAE) framework consisting of two key strategies: (1) Synthetic-to-Real Domain Transfer Learning (S2R-DTL) that effectively translates images from the synthetic degraded domain to the real degraded domain, aligning the domains at the pixel level, and (2) Degraded-to-Clear Domain Transfer Learning (D2C-DTL) that further adapts the enhancement model from the synthetic to the real domain by translating images from the real degraded domain to the real clean domain in both supervised and unsupervised branches. A unique aspect of our approach is the integration of a Domain Noise Contrastive Estimation (DoNCE) loss in both learning strategies. This specialized loss formulation enables TCDAE to robustly translate images across domains, even in scenarios lacking strong positive examples. Consequently, our framework can generate enhanced images with natural, realistic appearances akin to real clear images. Comprehensive experiments on real-world degraded scenes across diverse tasks, including dehazing, deraining, and deblurring, demonstrate the superiority of TCDAE over state-of-the-art methods, achieving improved visual quality, quantitative metrics, and downstream task performance.

Multiple-Input Multiple-Output Synthetic Aperture Radar Waveform and Filter Design in the Presence of Uncertain Interference Environment

Multiple-input multiple-output synthetic aperture radar (MIMO-SAR) anti-jamming waveform design relies on accurate prior information about the interference. However, it is difficult to obtain accurate prior knowledge about uncertain intermittent sampling repeater jamming (ISRJ), leading to a severe decline in the detection performance of MIMO-SAR systems. Therefore, this article studies the robust joint design problem of MIMO radar transmit waveform and filter against uncertain ISRJ. We characterize two categories of uncertain interference, including sample length uncertainty and sample-time uncertainty, modeled as Gaussian distribution in different range bins. Based on the uncertain interference model, we formulate the maximizing SINR as a figure of merit, which is a non-convex quadratic optimization problem under specific waveform constraints. Based on the alternating direction method of multipliers (ADMM) framework, a novel joint design algorithm of waveform and filter is proposed. In order to improve the convergence performance of ADMM, the difference in convex functions (DC) programming is applied to the ADMM iterations framework to solve the problem of waveform energy inequality constraint. Finally, numerical results demonstrate the effectiveness and robustness of the proposed method, compared to the existing methods that utilize deterministic interference models in the uncertain ISRJ environment. Moreover, the spaceborne SAR real scene imaging simulations are conducted to evaluate the anti-ISRJ performance.

A New Symmetry-Based Transformer for Text Spotting in Person and Vehicle Re-Identification Images

Text spotting in person and vehicle re-identification images is complex due to the presence of multiple views of the same person and vehicle. Most existing models focus on text spotting in natural scene images, our work focuses on spotting in person and vehicle re-identification images. The rationale behind this work is that the person and the vehicles share symmetry properties and the bib number in the torso and license plate number in the vehicle are text. The method divides the input image into patches, and it explores vision transformation for encoding the patches into linear patches. The linearly embedded patches are fed to the feature similarity index step, which involves phase congruency and gradient magnitude to detect symmetric patches. The transformer is proposed to encode and capture textual information from the symmetry patches for text detection and recognition. The decoder receives the attention features from the encoder and fetches a multi-task head with the information about the detected and recognized text. The experiments on person and vehicle image benchmark, viz. (Person) Re-ID, RBNR, UFPR-ALPR and RodoSol datasets show significant improvement in performance when compared to other text spotting models. The effectiveness of the proposed model is validated by testing on the benchmark datasets, namely, ICDAR 2015, Total-Text and CTW1500 of natural scene images. Furthermore, cross-data validation shows the proposed method is independent of domains.

A novel framework for predicting 3D scene infrared radiation characteristics through AI-enhanced thermodynamic modeling

In the evolving landscape of modern warfare, virtual battlefield construction technology significantly impacts fields such as weapon development and tactical formulation. Currently, 3D infrared scene simulation face challenges like high computational resource demands, difficulty in depicting interaction mechanisms between dynamic targets and environments, and a lack of reliable validation for algorithms. In this paper, we propose a three-dimensional infrared characteristics prediction framework (3DICPF) that integrates data-driven and theoretical algorithms, capable of rapidly constructing realistic 3D infrared scenes based on limited input information. Considering the extremely high cost of data acquisition, a step-by-step construction approach is adopted to reduce dependence on data. For the aligned image inputs, a 3D shape reconstruction network is utilized to obtain the mesh calculation domain of the target. Subsequently, 3DICPF employs two temperature field reconstruction networks to obtain the temperature distribution of the target. Finally, by using Monte Carlo ray tracing techniques, 3DICPF calculates the radiative interactions between the target and the background, achieving real-time infrared scene rendering. Moreover, confirmatory experiments with a shape-similar tank replica were conducted, demonstrating that 3DICPF could simulate infrared scene images with an average similarity above 88.13%. Overall, our research presents a significant advancement in virtual warfare technology, offering an efficient tool for enhancing strategic military planning and decision-making.

SGEdit: Bridging LLM with Text2Image Generative Model for Scene Graph-based Image Editing

Scene graphs offer a structured, hierarchical representation of images, with nodes and edges symbolizing objects and the relationships among them. It can serve as a natural interface for image editing, dramatically improving precision and flexibility. Leveraging this benefit, we introduce a new framework that integrates large language model (LLM) with Text2Image generative model for scene graph-based image editing. This integration enables precise modifications at the object level and creative recomposition of scenes without compromising overall image integrity. Our approach involves two primary stages: 1) Utilizing a LLM-driven scene parser, we construct an image's scene graph, capturing key objects and their interrelationships, as well as parsing fine-grained attributes such as object masks and descriptions. These annotations facilitate concept learning with a fine-tuned diffusion model, representing each object with an optimized token and detailed description prompt. 2) During the image editing phase, a LLM editing controller guides the edits towards specific areas. These edits are then implemented by an attention-modulated diffusion editor, utilizing the fine-tuned model to perform object additions, deletions, replacements, and adjustments. Through extensive experiments, we demonstrate that our framework significantly outperforms existing image editing methods in terms of editing precision and scene aesthetics. Our code is available at https://bestzzhang.github.io/SGEdit.

A Virtual View Acquisition Technique for Complex Scenes of Monocular Images Based on Layered Depth Images

With the rapid development of stereoscopic display technology, how to generate high-quality virtual view images has become the key in the applications of 3D video, 3D TV and virtual reality. The traditional virtual view rendering technology maps the reference view into the virtual view by means of 3D transformation, but when the background area is occluded by the foreground object, the content of the occluded area cannot be inferred. To solve this problem, we propose a virtual view acquisition technique for complex scenes of monocular images based on a layered depth image (LDI). Firstly, the depth discontinuities of the edge of the occluded area are reasonably grouped by using the multilayer representation of the LDI, and the depth edge of the occluded area is inpainted by the edge inpainting network. Then, the generative adversarial network (GAN) is used to fill the information of color and depth in the occluded area, and the inpainting virtual view is generated. Finally, GAN is used to optimize the color and depth of the virtual view, and the high-quality virtual view is generated. The effectiveness of the proposed method is proved by experiments, and it is also applicable to complex scenes.

Scene-cGAN: A GAN for underwater restoration and scene depth estimation

Despite their wide scope of application, the development of underwater models for image restoration and scene depth estimation is not a straightforward task due to the limited size and quality of underwater datasets, as well as variations in water colours resulting from attenuation, absorption and scattering phenomena in the water column. To address these challenges, we present an all-in-one conditional generative adversarial network (cGAN) called Scene-cGAN. Our cGAN is a physics-based multi-domain model designed for image dewatering, restoration and depth estimation. It comprises three generators and one discriminator. To train our Scene-cGAN, we use a multi-term loss function based on uni-directional cycle-consistency and a novel dataset. This dataset is constructed from RGB-D in-air images using spectral data and concentrations of water constituents obtained from real-world water quality surveys. This approach allows us to produce imagery consistent with the radiance and veiling light corresponding to representative water types. Additionally, we compare Scene-cGAN with current state-of-the-art methods using various datasets. Results demonstrate its competitiveness in terms of colour restoration and its effectiveness in estimating the depth information for complex underwater scenes.

Deep Learning-Driven Virtual Furniture Replacement Using GANs and Spatial Transformer Networks

This study proposes a Generative Adversarial Network (GAN)-based method for virtual furniture replacement within indoor scenes. The proposed method addresses the challenge of accurately positioning new furniture in an indoor space by combining image reconstruction with geometric matching through combining spatial transformer networks and GANs. The system leverages deep learning architectures like Mask R-CNN for executing image segmentation and generating masks, and it employs DeepLabv3+, EdgeConnect algorithms, and ST-GAN networks for carrying out virtual furniture replacement. With the proposed system, furniture shoppers can obtain a virtual shopping experience, providing an easier way to understand the aesthetic effects of furniture rearrangement without putting in effort to physically move furniture. The proposed system has practical applications in the furnishing industry and interior design practices, providing a cost-effective and efficient alternative to physical furniture replacement. The results indicate that the proposed method achieves accurate positioning of new furniture in indoor scenes with minimal distortion or displacement. The proposed system is limited to 2D front-view images of furniture and indoor scenes. Future work would involve synthesizing 3D scenes and expanding the system to replace furniture images photographed from different angles. This would enhance the efficiency and practicality of the proposed system for virtual furniture replacement in indoor scenes.

A novel binary hashing for agricultural scenery classification

In this research, we present PerceptHashing, a technique designed to categorize million-scale agricultural scenic images by incorporating human gaze shifting paths (GSPs) into a hashing framework. For each agricultural image, we identify visually and semantically significant object patches, such as fields, crops, and water bodies. These patches are linked to form a graphlet, establishing a network of spatially adjacent patches, and a GSP is then extracted using an active learning algorithm. The GSP reflects the distribution of human gaze across different regions of each agricultural scene, typically involving fewer than 12 regions, as validated by cross-validation. We then design a binary hashing framework that effectively leverages the semantics encoded in these GSPs. This framework integrates three key elements: (i) refinement of noisy labels, (ii) incorporation of deep image-level agricultural semantics, and (iii) updates to the adaptive data graph. The resulting hash codes from each GSP are converted into a kernelized visual descriptor for classification. To evaluate the influence of GSPs on agricultural image classification both qualitatively and quantitatively, we conducted an extensive user study comparing GSPs from typical observers with those from Alzheimer’s patients. The results demonstrate that: (1) the classification accuracy of 1.22 million agricultural aerial images using our approach is significantly higher than those processed by other methods, and (2) GSPs from 33 Alzheimer’s patients differ markedly from those of 37 normal observers, leading to notable differences in classification accuracy (accnormal=0.694\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$acc_{normal}=0.694$$\\end{document} versus accpatient=0.322\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$acc_{patient}=0.322$$\\end{document}).

Energy-Saving Design of Smart City Buildings Based on Deep Learning Algorithms and Remote Sensing Image Scenes

Energy-Saving Design of Smart City Buildings Based on Deep Learning Algorithms and Remote Sensing Image Scenes

High‐Performance Infrared Scene Generation Using a Double‐S Microbridge Photothermal Film Emitter With Ultralarge Array

AbstractInfrared emitters are highly desirable for applications in infrared imaging and stealth technology. Photothermal film emitters, which are driven by light, have garnered considerable interest due to their excellent photothermal properties and ease of fabrication. However, existing photothermal films often suffer from low photothermal conversion efficiency (PCE) and limited array scale. Here, a photothermal film emitter featuring a double‐S microbridge structure is presented, which comprehensively considering the PCE and frame rate. Initially, Si microbridges are fabricated within each microstructure to support the photothermal film. The pattern structure is then arranged into a double‐S configuration to manipulate the thermal conductivity. By optimizing the length and width of the double‐S pattern, high PCE and fast frame rates are achieved. The double‐S microbridge structure photothermal film emitter is fabricated with a 4‐inch diameter, forming an ultra‐large array with over 3000 × 3000 pixels. The absorption and photothermal conversion characteristics are studied, revealing a maximum temperature of 582.70 K and a highest PCE of 16.32%. Time‐dependent photothermal response demonstrated the emitter's rapid heat storage and release capabilities. Additionally, an infrared scene generator is realized. These results position the photothermal film emitter as a promising solution for infrared imaging and dynamic scene generation.

Unmanned Aerial Vehicle-Neural Radiance Field (UAV-NeRF): Learning Multiview Drone Three-Dimensional Reconstruction with Neural Radiance Field

In traditional 3D reconstruction using UAV images, only radiance information, which is treated as a geometric constraint, is used in feature matching, allowing for the restoration of the scene’s structure. After introducing radiance supervision, NeRF can adjust the geometry in the fixed-ray direction, resulting in a smaller search space and higher robustness. Considering the lack of NeRF construction methods for aerial scenarios, we propose a new NeRF point sampling method, which is generated using a UAV imaging model, compatible with a global geographic coordinate system, and suitable for a UAV view. We found that NeRF is optimized entirely based on the radiance while ignoring the direct geometry constraint. Therefore, we designed a radiance correction strategy that considers the incidence angle. Our method can complete point sampling in a UAV imaging scene, as well as simultaneously perform digital surface model construction and ground radiance information recovery. When tested on self-acquired datasets, the NeRF variant proposed in this paper achieved better reconstruction accuracy than the original NeRF-based methods. It also reached a level of precision comparable to that of traditional photogrammetry methods, and it is capable of outputting a surface albedo that includes shadow information.

Remote sensing image scene recognition using MobileNet

Abstract. Remote sensing image scene recognition plays a pivotal role in various applications, including environmental monitoring, disaster response, urban planning, precision agriculture, and aids in resource management and policy formulation. However, utilizing established convolutional neural networks(CNNs) models like AlexNet and VGG9 for this task can be computationally intensive and time-consuming due to their extensive parameter requirements. This dissertation introduces a MobileNet-based CNN optimized for remote sensing image scene recognition. This lightweight model significantly reduces computational load and model size without compromising accuracy, thereby enhancing efficiency. Empirical results on the NWPU45 dataset demonstrate MobileNet's superiority, achieving an accuracy of 91.16%, a Kappa coefficient of 90.96%, and an F1 score of 91.16% on the test set. Moreover, MobileNet's compact architecture, with merely 3.2531 million parameters and 587.9342 million FLOPs, underscores its efficiency and makes it a promising candidate for practical deployment in remote sensing applications. The findings suggest that MobileNet not only addresses the challenge of computational intensity but also opens new avenues for advancing scene recognition technology in the field of remote sensing.