Multiple Scenes Research Articles

SCARF: Scalable Continual Learning Framework for Memory‐efficient Multiple Neural Radiance Fields

AbstractThis paper introduces a novel continual learning framework for synthesising novel views of multiple scenes, learning multiple 3D scenes incrementally, and updating the network parameters only with the training data of the upcoming new scene. We build on Neural Radiance Fields (NeRF), which uses multi‐layer perceptron to model the density and radiance field of a scene as the implicit function. While NeRF and its extensions have shown a powerful capability of rendering photo‐realistic novel views in a single 3D scene, managing these growing 3D NeRF assets efficiently is a new scientific problem. Very few works focus on the efficient representation or continuous learning capability of multiple scenes, which is crucial for the practical applications of NeRF. To achieve these goals, our key idea is to represent multiple scenes as the linear combination of a cross‐scene weight matrix and a set of scene‐specific weight matrices generated from a global parameter generator. Furthermore, we propose an uncertain surface knowledge distillation strategy to transfer the radiance field knowledge of previous scenes to the new model. Representing multiple 3D scenes with such weight matrices significantly reduces memory requirements. At the same time, the uncertain surface distillation strategy greatly overcomes the catastrophic forgetting problem and maintains the photo‐realistic rendering quality of previous scenes. Experiments show that the proposed approach achieves state‐of‐the‐art rendering quality of continual learning NeRF on NeRF‐Synthetic, LLFF, and TanksAndTemples datasets while preserving extra low storage cost.

A transient network model for cross-regional layered fire smoke diffusion in high-rise buildings

At present, research on smoke diffusion in high-rise building fires mainly focuses on the single smoke layer propagation within a limited area, with less attention paid to the smoke propagation under the joint action of the fire floor and the vertical shaft area. Moreover, research on fire spread often focuses on the composition of combustible materials and building structures, neglecting natural ventilation. To address the above issues, we propose a transient network model for layered smoke diffusion (CLDTN) based on multi-region integration and complex control parameters. This model considers the smoke stratification mode of the fire floor affected by the fire source and ventilation, as well as the vertical stratification mode of the shaft affected by the stack effect, achieving cross-regional smoke stratification and diffusion modeling through smoke coupling. Then, a time-dependent transient network model is used to analyze furniture materials and open ventilation conditions of doors and windows, and a differential equation for energy conservation in multiple scene layers is constructed to analyze the dynamic behavior of smoke propagation.The comparative experiment of smoke propagation between CLDTN and the Fire Dynamics Simulator (FDS) shows that CLDTN is more efficient in computation time and the model is more accurate.

BEW-YOLOv8: A deep learning model for multi-scene and multi-scale flood depth estimation

As global climate change intensifies, the frequency of extreme weather events, including urban flooding, has risen, posing a significant threat to the safety of urban areas. Efficient and precise flood depth monitoring is crucial for a dynamic understanding of disaster conditions, enabling informed decision-making and reducing the potential for significant loss of life and property. Although object detection models have shown promise in estimating flood depths and have achieved notable results, enhancements are still needed in applications across multi-scene and multi-scale. This research enhances the YOLOv8 model by integrating Bidirectional Feature Pyramid Network (BiFPN), Effective Squeeze and Excitation (EffectiveSE), and Wise-IoU (WIoU), targeting submerged cars to develop the BEW-YOLOv8 model. Testing shows that the BEW-YOLOv8 model improves Precision, Recall, F1 Score, mAP@0.5, and mAP@0.5–0.95 by 14.1%, 2.2%, 7.9%, 8.7%, and 6.3%, respectively, compared to the unmodified YOLOv8 model. Compared with other existing models, BEW-YOLOv8 achieves the best performance with less than one-tenth of the parameters. Furthermore, this study verifies that the BEW-YOLOv8 model holds considerable promise for further enhancements with dataset expansions. Its precision and dependability make the BEW-YOLOv8 model well-suited for real-time flood depth assessment across multiple scenes and scales. The model’s capability for real-time processing supports urgent flood response needs, offering solid technical assistance for managing urban floods and mitigating disaster risks.

Rapid motion planning of manipulator in three-dimensional space under multiple scenes

Most working scenes of industrial robots are static scenes and from a previous static scene to a current static scene is a multi-scene. Finding optimal paths with limited time is difficult for motion planning in a high-dimensional space or in multiple scenes. The low efficiency of motion planning of rapidly-exploring random tree star in high-dimensional spaces, low adaptability of global replanning to multiple scenes are addressed by proposing the local replanning based on goal dynamically-guiding rapidly-exploring random tree star (LR-GD-RRT*). The algorithm contributes to fast path tree exploration and multi-scene motion planning. For path tree exploration, sampling points heuristically generating and new nodes growth by goal dynamically-guiding are proposed to reduce the blind and ineffective searches. Moreover, dynamic adjustment of size of new node neighborhood according to density of the obstacles is proposed to search for more neighbor nodes to optimize the path and also to reduce ineffective computation to improve efficiency. For multi-scene, three steps of trimming, re-exploration, and reconnection for local replanning are proposed. For path tree exploration, simulations in 2D plane, 3D space, and the manipulator show that GD-RRT* improves convergence speed, shortens path length and search time, compared with RRT*. For multi-scene, simulations in 3D space and with the manipulator show that the local replanning of the current scene has both lower path cost and higher planning efficiency compared with the global replanning of the previous scene. Motion of the six-degree-of-freedom robot end in a real scene also verifies the effectiveness of the LR-GD-RRT*.

Crowd Scenes in Péter Nádas’ Parallel Stories

The article examines the representation of the crowd in Péter Nádas’s Parallel Stories. In the plot, there are multiple scenes including the masses that highlight different features of the phenomenon. I argue that the 1956 Hungarian Revolution appears in the novel as a fundamental experience of society, influencing the nature of people’s gatherings even after its defeat. Firstly, I show that the characteristics of the swarm appear in the cooperation of people foraging for food during the revolution. However, the bread queue represents the extreme behaviour and the aggression of the acquisitive masses, whose collective action is always directed at acquiring something. Thereafter, I examine the representations of crowd panic in the basement scene of the 1956 cannonade and of the 1957 railway station gathering. Finally, I scrutinise two scenes with the seemingly liberated and intoxicated crowd. The analysis focuses on the relationship between the crowd and the individual, on the narrative representation of the crowd, on its influence on the participants’ perceptions, and the associated attributes.

LiDAR‐based place recognition for mobile robots in ground/water surface multiple scenes

AbstractLiDAR‐based 3D place recognition is an essential component of simultaneous localization and mapping systems in multi‐scene robotic applications. However, extracting discriminative and generalizable global descriptors of point clouds is still an open issue due to the insufficient use of the information contained in the LiDAR scans in existing approaches. In this paper, we propose a novel spatial‐temporal point cloud encoding network for multiple scenes, dubbed STM‐Net, to fully fuse the multi‐view spatial information and temporal information of LiDAR point clouds. Specifically, we first develop a spatial feature encoding module consisting of the single‐view transformer and multi‐view transformer. The module learns the correlation both within a single view and between two views by utilizing the multi‐layer range images generated by spherical projection and multi‐layer bird's eye view images generated by top‐down projection. Then in the temporal feature encoding module, we exploit the temporal transformer to mine the temporal information in the sequential point clouds, and a NetVLAD layer is applied to aggregate features and generate sub‐descriptors. Furthermore, we use a GeM pooling layer to fuse more information along the time dimension for the final global descriptors. Extensive experiments conducted on unmanned ground/surface vehicles with different LiDAR configurations indicate that our method (1) achieves superior place recognition performance than state‐of‐the‐art algorithms, (2) generalizes well to diverse sceneries, (3) is robust to viewpoint changes, (4) can operate in real‐time, demonstrating the effectiveness and satisfactory capability of the proposed approach and highlighting its promising applications in multi‐scene place recognition tasks.

Deep Edge-Based Fault Detection for Solar Panels.

Solar panels may suffer from faults, which could yield high temperature and significantly degrade their power generation. To detect faults of solar panels in large photovoltaic plants, drones with infrared cameras have been implemented. Drones may capture a huge number of infrared images. It is not realistic to manually analyze such a huge number of infrared images. To solve this problem, we develop a Deep Edge-Based Fault Detection (DEBFD) method, which applies convolutional neural networks (CNNs) for edge detection and object detection according to the captured infrared images. Particularly, a machine learning-based contour filter is designed to eliminate incorrect background contours. Then faults of solar panels are detected. Based on these fault detection results, solar panels can be classified into two classes, i.e., normal and faulty ones (i.e., macro ones). We collected 2060 images in multiple scenes and achieved a high macro F1 score. Our method achieved a frame rate of 28 fps over infrared images of solar panels on an NVIDIA GeForce RTX 2080 Ti GPU.

MLA-MFL: A Smartphone Indoor Localization Method for Fusing Multisource Sensors Under Multiple Scene Conditions

MLA-MFL: A Smartphone Indoor Localization Method for Fusing Multisource Sensors Under Multiple Scene Conditions

Transformer-based berm detection for automated bulldozer safety in edge dumping

Ensuring the safety of automated bulldozers during edge dumping operations in open-pit mines requires accurate detection of the berm's edge. However, precisely detecting berm edges in open-pit mining scenes is challenging due to the presence of similar texture structures. To address this issue, this paper created a dataset for berm edge detection that covers multiple scenes and proposed the edge detection model with transformer (MEDTER). This model, based on a transformer with a two-stage detection structure, aims to restore thin edges. Additionally, the deep multilevel aggregation decoder enhances the information flow in the transformer to improve the sensitivity to detailed information. The BCE-Dice loss functions effectively handle the problem of pixel imbalance. Results demonstrate that the proposed model achieves significant improvements compared to others. Furthermore, the model exhibits excellent detection performance despite variations in scenes and detection distances, suggesting its practical value in ensuring the safety of automated bulldozers during edge dumping.

Research on a Near-Field Millimeter Wave Imaging Algorithm and System Based on Multiple-Input Multiple-Output Sparse Sampling

In order to reduce the hardware cost and data acquisition time in near-field scenarios, such as airport security imaging systems, this paper discusses the layout of a multiple-input multiple-output (MIMO) radar array. In view of the existing multi-input multiple-output imaging algorithm, the reconstructed image artifacts and aliasing problems caused by sparse sampling are discussed. In this paper, a multi-station radar array and a corresponding sparse MIMO imaging algorithm based on combined sparse sub-channels are proposed. By studying the wave–number spectrum of backscattered MIMO synthetic aperture radar (SAR) data, the nonlinear relationship between the wave number spectrum and reconstructed image is established. By selecting a complex gain vector, multiple channels are coherently combined effectively, thus eliminating aliasing and artifacts in the reconstructed image. At the same time, the algorithm can be used for the MIMO–SAR configuration of arbitrarily distributed transmitting and receiving arrays. A new multi-station millimeter wave imaging system is designed by using a frequency-modulated continuous wave (FMCW) chip and sliding rail platform as a planar SAR. The combination of the hardware system provides reconfiguration, convenience and economy for the combination of millimeter wave imaging systems in multiple scenes.

Few-Shot Remote Sensing Novel View Synthesis with Geometry Constraint NeRF

Novel View Synthesis (NVS) is an important task for 3D interpretation in remote sensing scenes, which also benefits vicinagearth security by enhancing situational awareness capabilities. Recently, NVS methods based on Neural Radiance Fields (NeRFs) have attracted increasing attention for self-supervised training and highly photo-realistic synthesis results. However, it is still challenging to synthesize novel view images in remote sensing scenes, given the complexity of land covers and the sparsity of input multi-view images. In this paper, we propose a novel NVS method named FReSNeRF, which combines Image-Based Rendering (IBR) and NeRF to achieve high-quality results in remote sensing scenes with sparse input. We effectively solve the degradation problem by adopting the sampling space annealing method. Additionally, we introduce depth smoothness based on the segmentation mask to constrain the scene geometry. Experiments on multiple scenes show the superiority of our proposed FReSNeRF over other methods.

The Batman and the redemption of first responders: The culmination of a rhetoric of rescue during COVID-19, Trump and protest

The timing of Matt Reeves’s The Batman (2022) situated the film as a symbol of and response to the social troubles that resulted from the political unrest in 2020 and 2021, presenting solution-driven rhetoric amidst the numerous public crises in American discourse and identity. Captured and represented in his opening prologue speech in juxtaposition to his rhetorical culmination speech, Batman’s orations mirror his own arc and the politically prescribed reawakening of confidence in political figures and institutions. Paralleling his addresses, the evolution of Batman’s excessive violence when he initially works under the identity of ‘Vengeance’, compared to his subsequent performance as a first responder at the end of the film, serves as a rhetoric of redemption for first responders. Multiple scenes focus on Batman’s learning to recognize humans’ need for compassion as he becomes a student of social progress. Working as a unifying symbol of a diverse leader, Gotham’s newly elected mayor, Bella Reál, simultaneously offers a vision for transitioning away from leaders associated with racist and sexist discourse. With Batman’s rhetorical culmination and transformation into an emergency rescuer, The Batman serves as unifying verbal and visual discourse in response to contemporary political unrest.

A Transformer and Visual Foundation Model-Based Method for Cross-View Remote Sensing Image Retrieval

Abstract. Retrieving UAV images that lack POS information with georeferenced satellite orthoimagery is challenging due to the differences in angles of views. Most existing methods rely on deep neural networks with a large number of parameters, leading to substantial time and financial investments in network training. Consequently, these methods may not be well-suited for downstream tasks that have high timeliness requirements. In this work, we propose a cross-view remote sensing image retrieval method based on transformer and visual foundation model. We investigated the potential of visual foundation model for extracting common features from cross-view images. Training is only conducted on a small, self-designed retrieval head, alleviating the burden of network training. Specifically, we designed a CVV module to optimize the features extracted from the visual foundation model, making these features more adept for cross-view image retrieval tasks. And we designed an MLP head to achieve similarity discrimination. The method is verified on a publicly available dataset containing multiple scenes. Our method shows excellent results in terms of both efficiency and accuracy on 15 sub-datasets (10 or 50 scene categories) derived from the public dataset, which holds practical value in engineering applications with streamlined scene categories and constrained computational resources. Furthermore, we initiated a comprehensive discussion and conducted ablation experiments on the network design to validate its efficacy. Additionally, we analyzed the presence of overfitting within the network and deliberated on the limitations of our study, proposing potential avenues for future enhancements.

PCINet: a Prototype- and Concept-based Interpretable Network for Mutli-scene Recognition

Abstract. With the development of remote sensing techniques, a large number of high-resolution aerial images is now available and benefit many applications. Multi-scene recognition plays a key role in applying remote sensing images to these applications, which refers to predicting multiple scenes coexisted in an aerial image and has attracted an increasing attention. Recently, most researchers tend to invent deep learning-based recognition models and has gained great achievements. However, few efforts have been deployed to explaining the success of deep neural networks in multi-scene recognition. To address this, we introduce concept bottleneck model (CBM) to interpreting model performance and propose a novel network, namely Prototype- and Concept-based Interpretable Network (PCINet), that projects aerial imagery into a prototype-concept memory bank and encode their correlations for explaining how a network can identify coexisting scenes in an aerial image. Specifically, the proposed network mainly consists of two branches: prototype matching that measures similarity scores between image features and scene prototypes, and concept bottleneck branches that aligned image features to textual embeddings and compute their relations with concept embeddings. Afterwards, Outputs are integrated for inferring scene categories. Experimental results show that the model enhances interpretability, providing valuable insights for urban planning and resource management, thereby bridging the gap between deep learning models and practical applications.

Research on the countermeasures of the single-phase-to-earth faults in flexibly grounded distribution networks

In a neutral, flexible, grounded distribution network, the phenomenon that a single-phase ground fault with high impedance connected in parallel with a low resistance can result in a relatively lower zero-sequence current flowing through the faulty feeder. This zero-sequence current can make it a challenge to use zero-sequence protection schemes to accurately identify and effectively protect the faulty feeder. In terms of the aforementioned challenge, based on the zero-sequence equivalent circuit of a single-phase-to-earth fault, this paper conducts an investigation on the phase difference between the zero-sequence current and the non-fault phase line voltage at power frequency in flexible grounding distribution network with different grounded conditions, both before and after the introduction of a parallel low-resistor connection. Subsequently, a criterion for selecting the faulty feeder is developed based on the observed phase difference characteristics following the implementation of the parallel low-resistor connection. Furthermore, a disposal strategy for single-phase-to-earth faults is formulated, which takes fault resistance estimation into account. To validate the proposed method, a simulation model of a flexible grounding distribution network is implemented with PSCAD/EMTDC. The effectiveness and reliability of this method in multiple scenes, including the initial fault conditions, sampling frequency, and distributed generation (DG), are proved through simulation results.

OF-DFN: Optical flow prediction network for different perspective image fusion

Currently, non-decision-level image fusion algorithms require extremely high registration precision of the images to be fused. In the face of different perspective image fusion scenarios, traditional feature registration algorithms and learning-based methods have poor robustness and are unsuitable for large image differences because of the Registration-Fusion separation. In addition, the lack of relevant datasets also hinders the development of different perspective image fusion methods. Given the above problems, we collect 5000 sets of different perspective RGB-MONO datasets in multiple scenes for raw data support. We present an end-to-end learned system for fusing two different perspective photographs into a chosen target view. The cascaded feature extraction based on encoder–decoder structure enables learning optical flow at different feature levels systematically. Then the optical flow module enables the image to be continuously registered and optimized during the fusion process, thus avoiding the deviations introduced by non-end-to-end algorithms. Extensive quantitative and qualitative experiments demonstrate that our proposed system can effectively fuse images from different perspectives in our self-built dataset. Compared with non-end-to-end fusion, our method provides superior performance in several fusion evaluation indicators.

Visual Recognition Memory of Scenes Is Driven by Categorical, Not Sensory, Visual Representations.

When we perceive a scene, our brain processes various types of visual information simultaneously, ranging from sensory features, such as line orientations and colors, to categorical features, such as objects and their arrangements. Whereas the role of sensory and categorical visual representations in predicting subsequent memory has been studied using isolated objects, their impact on memory for complex scenes remains largely unknown. To address this gap, we conducted an fMRI study in which female and male participants encoded pictures of familiar scenes (e.g., an airport picture) and later recalled them, while rating the vividness of their visual recall. Outside the scanner, participants had to distinguish each seen scene from three similar lures (e.g., three airport pictures). We modeled the sensory and categorical visual features of multiple scenes using both early and late layers of a deep convolutional neural network. Then, we applied representational similarity analysis to determine which brain regions represented stimuli in accordance with the sensory and categorical models. We found that categorical, but not sensory, representations predicted subsequent memory. In line with the previous result, only for the categorical model, the average recognition performance of each scene exhibited a positive correlation with the average visual dissimilarity between the item in question and its respective lures. These results strongly suggest that even in memory tests that ostensibly rely solely on visual cues (such as forced-choice visual recognition with similar distractors), memory decisions for scenes may be primarily influenced by categorical rather than sensory representations.

Learning single and multi-scene camera pose regression with transformer encoders

Contemporary state-of-the-art localization methods perform feature matching against a structured scene model or learn to regress the scene 3D coordinates. The resulting matches between 2D query pixels and 3D scene coordinates are used to estimate the camera pose using PnP and RANSAC, requiring the camera intrinsics for both the query and reference images. An alternative approach is to directly regress the camera pose from the query image. Although less accurate, absolute camera pose regression does not require any additional information at inference time and is typically lightweight and fast. Recently, Transformers were proposed for learning multi-scene camera pose regression, employing encoders to attend to spatially varying deep features while using decoders to embed multiple scene queries at once. In this work, we show that Transformer Encoders can aggregate and extract task-informative latent representations for learning both single- and multi- scene camera pose regression, without Transformer-Decoders. Our approach is shown to reduce the runtime and memory of previous Transformer-based multi-scene solutions, while comparing favorably with contemporary pose regression schemes and achieving state-of-the-art accuracy on multiple indoor and outdoor regression benchmarks. In particular, to the best of our knowledge, our approach is the first absolute regression approach to attain sub-meter average accuracy across outdoor scenes. We make our code publicly available at: https://github.com/yolish/transposenet.

Dark Light Image-Enhancement Method Based on Multiple Self-Encoding Prior Collaborative Constraints

The purpose of dark image enhancement is to restore dark images to visual images under normal lighting conditions. Due to the ill-posedness of the enhancement process, previous enhancement algorithms often have overexposure, underexposure, noise increases and artifacts when dealing with complex and changeable images, and the robustness is poor. This article proposes a new enhancement approach consisting in constructing a dim light enhancement network with more robustness and rich detail features through the collaborative constraint of multiple self-coding priors (CCMP). Specifically, our model consists of two prior modules and an enhancement module. The former learns the feature distribution of the dark light image under normal exposure as an a priori term of the enhancement process through multiple specific autoencoders, implicitly measures the enhancement quality and drives the network to approach the truth value. The latter fits the curve mapping of the enhancement process as a fidelity term to restore global illumination and local details. Through experiments, we concluded that the new method proposed in this article can achieve more excellent quantitative and qualitative results, improve detail contrast, reduce artifacts and noise, and is suitable for dark light enhancement in multiple scenes.

Photoelectricity Theory-Based Concrete Crack Image Segmentation and Optimal Exposure Interval Research

To solve the problem of low accuracy in automatic concrete crack image segmentation and the non-standardization of concrete crack image datasets, an exposure-based concrete crack image capture scene characterization method was proposed, and the optimal exposure interval for crack segmentation was presented by multiple scene image capture experiments. First, current public crack datasets were collected and analyzed, and it was shown that improper spatial resolution, mislabeling, overexposure, and defocus are frequent non-standardization problems in crack dataset production. Through the analysis of the photoelectric principle in concrete crack imaging, an equivalent exposure was set as a core indicator for scene characterization. Twenty-one indoor scenes were designed by varying the illumination intensity and exposure time, and the experimental results showed that an equivalent exposure can be a core control index for scene characterization. The grayscale distribution law of concrete crack images was analyzed with four specimens’ images captured indoors in 50 exposure scenes, and the segmentation accuracy of an image from each scene was calculated through comparison with corresponding manually labeled binary files. The experiment’s results revealed that 5~50 lx·s was the optimal equivalent exposure interval for concrete crack image segmentation, in which better segmentation accuracy was achieved with an F1 score of up to 96.3%.