Scene Reconstruction Research Articles

GauLoc: 3D Gaussian Splatting‐based Camera Relocalization

Abstract3D Gaussian Splatting (3DGS) has emerged as a promising representation for scene reconstruction and novel view synthesis for its explicit representation and real‐time capabilities. This technique thus holds immense potential for use in mapping applications. Consequently, there is a growing need for an efficient and effective camera relocalization method to complement the advantages of 3DGS. This paper presents a camera relocalization method, namely GauLoc, in a scene represented by 3DGS. Unlike previous methods that rely on pose regression or photometric alignment, our proposed method leverages the differential rendering capability provided by 3DGS. The key insight of our work is the proposed implicit featuremetric alignment, which effectively optimizes the alignment between rendered keyframes and the query frames, and leverages the epipolar geometry to facilitate the convergence of camera poses conditioned explicit 3DGS representation. The proposed method significantly improves the relocalization accuracy even in complex scenarios with large initial camera rotation and translation deviations. Extensive experiments validate the effectiveness of our proposed method, showcasing its potential to be applied in many real‐world applications. Source code will be released at https://github.com/xinzhe11/GauLoc.

Bacterial profile-based body fluid identification using a machine learning approach.

Identifying the origins of biological traces is critical for the reconstruction of crime scenes in forensic investigations. Traditional methods for body fluid identification rely on chemical, enzymatic, immunological, and spectroscopic techniques, which can be sample-consuming and depend on simple color-change reactions. However, these methods have limitations when residual samples are insufficient after DNA extraction. This study aimed to develop a method for body fluid identification by leveraging bacterial DNA profiling to overcome the limitations of the conventional approaches. Bacterial profiles were determined by sequencing the hypervariable region of the 16S rRNA gene,using DNA metabarcoding of evidence collected from criminal cases. Amplicon sequence variants (ASVs) were analyzed to identify significant microbial patterns in different body fluid samples. The bacterial profile-based method demonstrated high discriminatory power with a machine learning model trained using the naïve Bayes algorithm, achieving an accuracy of over 98% in classifying samples into one of four body fluid types: blood, saliva, vaginal secretion, and mixture traces of vaginal secretions and semen. Bacterial profiling enhances the accuracy and robustness of body fluid identification in forensic analysis, providing a valuable alternative to traditional methods by utilizing DNA and microbial community data despite the uncontrollable conditions. This approach offers significant improvements in the classification accuracy and practical applicability in forensic investigations.

Gunshot Defense Wounds: Three Case Reports and a Literature Review

Defense wounds generally occur when a victim attempts to protect vital organs and blood vessels during an assault, and are therefore typically located on the forearms or hands. Gunshot-induced defense wounds are less frequent compared to stab wounds, which have been extensively described in the literature. We present three cases where victims unsuccessfully attempted to defend themselves from gunshots, and where the detailed description of injuries played a key role in the reconstruction of the crime scenes. In the first case, a man was shot with a smooth-bore shotgun, presenting a “through-and-through” gunshot wound on the lateral surface of the left forearm and a large gaping wound on the posterior surface of the same forearm. The second and third cases involved two men who were shot with handguns. The first displayed an entrance wound on the dorsal surface of the right hand, with burned edges and smoke soiling, and an exit wound on the palmar surface. The second case involved two wounds on the left hand: one on the dorsum and the other on the palm. To the best of our knowledge, few studies in the literature emphasize the role of gunshot defense wounds in aiding the reconstruction of crime dynamics. The cases presented in this study highlight the importance of precisely defining the pathological and morphological features of the wounds, as well as the bullet trajectories, to accurately identify defense wounds. These findings are valuable for reconstructing the dynamics of the assault and providing critical information to the public prosecutor.

Fast Building Instance Proxy Reconstruction for Large Urban Scenes.

Digitalization of large-scale urban scenes (in particular buildings) has been a long-standing open problem, which attributes to the challenges in data acquisition, such as incomplete scene coverage, lack of semantics, low efficiency, and low reliability in path planning. In this paper, we address these challenges in urban building reconstruction from aerial images, and we propose an effective workflow and a few novel algorithms for efficient 3D building instance proxy reconstruction for large urban scenes. Specifically, we propose a novel learning-based approach to instance segmentation of urban buildings from aerial images followed by a voting-based algorithm to fuse the multi-view instance information to a sparse point cloud (reconstructed using a standard Structure from Motion pipeline). Our method enables effective instance segmentation of the building instances from the point cloud. We also introduce a layer-based surface reconstruction method dedicated to the 3D reconstruction of building proxies from extremely sparse point clouds. Extensive experiments on both synthetic and real-world aerial images of large urban scenes have demonstrated the effectiveness of our approach. The generated scene proxy models can already provide a promising 3D surface representation of the buildings in large urban scenes, and when applied to aerial path planning, the instance-enhanced building proxy models can significantly improve data completeness and accuracy, yielding highly detailed 3D building models.

Filtering-Assisted Airborne Point Cloud Semantic Segmentation for Transmission Lines.

Point cloud semantic segmentation is crucial for identifying and analyzing transmission lines. Due to the number of point clouds being huge, complex scenes, and unbalanced sample proportion, the mainstream machine learning methods of point cloud segmentation cannot provide high efficiency and accuracy when extending to transmission line scenes. This paper proposes a filter-assisted airborne point cloud semantic segmentation for transmission lines. First, a large number of ground point clouds is identified by introducing the well-developed cloth simulation filter to alleviate the impact of the imbalance of the target object proportion on the classifier's performance. The multi-dimensional features are then defined, and the classification model is trained to achieve the multi-element semantic segmentation of the transmission line scene. The experimental results and analysis indicate that the proposed filter-assisted algorithm can significantly improve the semantic segmentation performance of the transmission line point cloud, enhancing both the point cloud segmentation efficiency and accuracy by more than 25.46% and 3.15%, respectively. The filter-assisted point cloud semantic segmentation method reduces the volume of sample data, the number of sample classes, and the sample imbalance index in power line scenarios to a certain extent, thereby improving the classification accuracy of classifiers and reducing time consumption. This research holds significant theoretical reference value and engineering application potential for scene reconstruction and intelligent understanding of airborne laser point cloud transmission lines.

Integrating artificial intelligence in forensic science

Thesis. The thesis of this article is to explore the integration of artificial intelligence (AI) methodology in forensic science. An assessment of the potential implications of AI for improving investigative processes and outcomes will also be addressed. Concept. The concept focuses on exploring the application of AI technologies in ar- eas of science such as analysing evidence recognising patterns and supporting decision- making systems. The article emphasises the practical use of AI algorithms in investiga- tions. on exploring how artificial intelligence technologies can be implemented in various aspects of forensic science. Results. An analysis of existing literature and case studies shows that integrating AI into forensic science can improve efficiency, accuracy, and objectivity in investigations. AI tools can automate tasks analyse datasets and identify patterns that might be challeng- 16 “ABoUT The INTeRNeT” - TheoRy ing for humans to detect. However utilising AI in forensic science poses challenges like algorithms, privacy issues with data handling, and the necessity for oversight. Further- more, when employing intelligence for inquiries it is essential to prioritise transparency, and accountability and uphold integrity in decision-making processes. Originality. This article adds to the discussion about integrating AI into science by of- fering a thorough examination of its potential advantages and obstacles faced along, with ethical concerns. By merging research findings and offering perspectives on emerging trends we can gain insights, into the impact of AI on advancing investigations in the years ahead. Additionally, the theoretical structure presented here establishes a foundation for research studies and practical implementations, within the field of forensic science. Keywords: Artificial intelligence, forensic science, AI applications in forensic science, machine learning for evidence analysis, crime scene reconstruction with AI

CoupNeRF: Property‐aware Neural Radiance Fields for Multi‐Material Coupled Scenario Reconstruction

AbstractNeural Radiance Fields (NeRFs) have achieved significant recognition for their proficiency in scene reconstruction and rendering by utilizing neural networks to depict intricate volumetric environments. Despite considerable research dedicated to reconstructing physical scenes, rare works succeed in challenging scenarios involving dynamic, multi‐material objects. To alleviate, we introduce CoupNeRF, an efficient neural network architecture that is aware of multiple material properties. This architecture combines physically grounded continuum mechanics with NeRF, facilitating the identification of motion systems across a wide range of physical coupling scenarios. We first reconstruct specific‐material of objects within 3D physical fields to learn material parameters. Then, we develop a method to model the neighbouring particles, enhancing the learning process specifically in regions where material transitions occur. The effectiveness of CoupNeRF is demonstrated through extensive experiments, showcasing its proficiency in accurately coupling and identifying the behavior of complex physical scenes that span multiple physics domains.

DSEM-NeRF: Multimodal feature fusion and global–local attention for enhanced 3D scene reconstruction

3D scene understanding often faces the problems of insufficient detail capture and poor adaptability to multi-view changes. To this end, we proposed a NeRF-based 3D scene understanding model DSEM-NeRF, which effectively improves the reconstruction quality of complex scenes through multimodal feature fusion and global–local attention mechanism. DSEM-NeRF extracts multimodal features such as color, depth, and semantics from multi-view 2D images, and accurately captures key areas by dynamically adjusting the importance of features. Experimental results show that DSEM-NeRF outperforms many existing models on the LLFF and DTU datasets, with PSNR reaching 20.01, 23.56, and 24.58 respectively, and SSIM reaching 0.834. In particular, it shows strong robustness in complex scenes and multi-view changes, verifying the effectiveness and reliability of the model.

Damage analysis and assessment of concrete T-girder bridge based on fire scene numerical reconstruction

Fire is a sporadic disaster of concrete bridges, with diverse fire environments and complex damage mechanisms. Accurately evaluating the damage situation of concrete bridges after a fire is exceedingly challenging. This study formulates a damage analysis and assessment method based on the step-by-step and progressively deepening working principle. The method relies on fire scene numerical reconstruction and encompasses key technical aspects, including bridge detection and analysis during the fire incident, fire scene numerical reconstruction, and subsequent bridge damage assessment. Building upon these principles, the study utilizes results from the detection and analysis of the concrete T-girder bridge during a fire incident to establish Computational Fluid Dynamics (CFD) and Finite Element Method (FEM) models for the numerical reconstruction of the fire scene. These models enable the calculation of varying temperature distributions and the evolution of the bridge under fire. Compared with the parameters obtained through the ISO834 method, the numerical reconstruction approaches not only enhances the accuracy of replicating the bridge combustion process but also enables the extraction of temperature field distribution patterns within the bridge fire space and its concrete components.

EIDU-Net: edge-preserved inception DenseGCN U-Net for LiDAR point cloud segmentation

With the development of laser scanners and machine learning, point cloud semantic segmentation plays a significant role in autonomous driving, scene reconstruction, human-computer interaction, and other fields. In recent years, point cloud semantic segmentation based on deep learning has become one of the key research directions in point cloud processing. Due to the limited ability to exploit geometric details and contextual information in point clouds, most methods that adopt encoder-decoder architecture lose local structural information easily, especially detailed features, and extract features insufficiently. To address this issue, the edge-preserving inception DenseGCN U-Net (entitled as EIDU-Net) is proposed. EIDU-Net makes full use of the complementation between geometric details in the original point cloud and high-level features. The edge-preserved graph pooling (EGP) layer, the key module of the EIDU-Net, is designed to retain additional edge feature information from the original point cloud during pooling operations. Accordingly, the edge-preserved graph unpooling (EGU) layer can restore the feature graph more efficiently based on the additionally retained edge features. Extensive experiments demonstrate that our proposed EIDU-Net has remarkable improvements on semantic segmentation tasks under whatever S3DIS or Terracotta Warrior fragments. Our code is publicly available at https://github.com/caoxin918/EIDU-Net.

A virtual scene reconstruction algorithm for 3D digital images based on machine learning

In order to obtain a better quality image of virtual scene reconstruction of three-dimensional digital image, a virtual scene reconstruction algorithm of three-dimensional digital image based on machine learning is proposed. Firstly, the state information and presentation instructions of the scene are analyzed to obtain the distribution position of the vertices of the mesh model of the image reconstruction. The global image is approximately calculated by the local coordinate method, and the local details are corrected to complete the rendering of the three-dimensional digital image. Then, the space and scale are used as feature points to build a window detection template on the image, and the classifier is applied to suppress discrete feature points and remove redundant features. Finally, the smoothed three-dimensional coordinates are obtained according to the fitting function method to reconstruct the three-dimensional surface, and the local two-dimensional triangle is segmented and mapped to the three-dimensional space to realize the reconstruction of the virtual scene of the three-dimensional digital image. The experimental results show that the algorithm has a fast convergence speed, the reconstructed image details and edge contours are complete, and the overall effect is good.

Deep-Learning-Based Real-Time Passive Non-Line-of-Sight Imaging for Room-Scale Scenes.

Non-line-of-sight imaging is a technique for reconstructing scenes behind obstacles. We report a real-time passive non-line-of-sight (NLOS) imaging method for room-scale hidden scenes, which can be applied to smart home security monitoring sensing systems and indoor fast fuzzy navigation and positioning under the premise of protecting privacy. An unseen scene encoding enhancement network (USEEN) for hidden scene reconstruction is proposed, which is a convolutional neural network designed for NLOS imaging. The network is robust to ambient light interference conditions on diffuse reflective surfaces and maintains a fast reconstruction speed of 12.2 milliseconds per estimation. The consistency of the mean square error (MSE) is verified, and the peak signal-to-noise ratio (PSNR) values of 19.21 dB, 15.86 dB, and 13.62 dB are obtained for the training, validation, and test datasets, respectively. The average values of the structural similarity index (SSIM) are 0.83, 0.68, and 0.59, respectively, and are compared and discussed with the corresponding indicators of the other two models. The sensing system built using this method will show application potential in many fields that require accurate and real-time NLOS imaging, especially smart home security systems in room-scale scenes.

HyperDehazing: A hyperspectral image dehazing benchmark dataset and a deep learning model for haze removal

Haze contamination severely degrades the quality and accuracy of optical remote sensing (RS) images, including hyperspectral images (HSIs). Currently, there are no paired benchmark datasets containing hazy and haze-free scenes in HSI dehazing, and few studies have analyzed the distributional properties of haze in the spatial and spectral domains. In this paper, we developed a new hazy synthesis strategy and constructed the first hyperspectral dehazing benchmark dataset (HyperDehazing), which contains 2000 pairs synthetic HSIs covering 100 scenes and another 70 real hazy HSIs. By analyzing the distribution characteristics of haze, we further proposed a deep learning model called HyperDehazeNet for haze removal from HSIs. Haze-insensitive longwave information injection, novel attention mechanisms, spectral loss function, and residual learning are used to improve dehazing and scene reconstruction capability. Comprehensive experimental results demonstrate that the HyperDehazing dataset effectively represents complex haze in real scenes with synthetic authenticity and scene diversity, establishing itself as a new benchmark for training and assessment of HSI dehazing methods. Experimental results on the HyperDehazing dataset demonstrate that our proposed HyperDehazeNet effectively removes complex haze from HSIs, with outstanding spectral reconstruction and feature differentiation capabilities. Furthermore, additional experiments conducted on real HSIs as well as the widely used Landsat-8 and Sentinel-2 datasets showcase the exceptional dehazing performance and robust generalization capabilities of HyperDehazeNet. Our method surpasses other state-of-the-art methods with high computational efficiency and a low number of parameters.

Neural Impedance Boundary (NeIB): a neural-network based framework for acoustic surface impedance estimation utilizing sparse measurement data

Amidst recent advancements in the 3D digital representation that have significantly enhanced the modeling of geometric attributes of pre-existing environments, accurate estimation of acoustic boundary conditions remains a complex challenge. This paper presents a novel way to determine what we refer to as a neural boundary field, using physics-informed neural networks (PINN). The aim is to estimate the surface impedance measured in-situ by utilizing several points of sound field pressure. This approach couples the Helmholtz equation with automatic differentiation in the PINN framework to estimate accurately the surface impedance using a hybrid modeling approach where measurement data and domain knowledge in the form of equations for the acoustic waves are combined. As a proof-of-concept, we train the neural networks using 2D sound field data obtained from high-fidelity numerical acoustical simulations that incorporate actual surface materials parameters. We discuss the measurement techniques associated with this method and outline our vision for its application to 3D scene reconstructions in the future.

Development of two step reverse transcription droplet digital PCR (RT-ddPCR) for simultaneous identification of saliva and semen

Determination of the type of body fluids is essential for crime scene reconstruction and for improving the reliability of expert testimonies. Messenger RNA (mRNA) analysis by reverse transcription quantitative PCR (RT-qPCR) has been used in forensic genetics, particularly for body fluid identification. It is a relative quantification method that compares the Ct values of target and reference gene. Thus, the method is unsuitable for determining exact copy numbers of the target gene. To address this limitation, this study performed body fluid-specific mRNA analysis using two-step reverse transcription droplet digital PCR (RT-ddPCR), which is capable of absolute quantification. We found that RT-ddPCR was accurate and sensitive enough to detect as little as 1.5 copies/μl of complementary DNA (cDNA), making it suitable for application using casework samples. It was also highly specific for body fluids, as non-specific amplification did not occur. In addition, saliva-semen mixtures with ratios ranging from 1:50 to 50:1 were successfully identified. When comparing the results of RT-qPCR and RT-ddPCR, some samples were difficult to interpret because of the high Ct values of RT-qPCR. However, when the same samples were analyzed using RT-ddPCR, saliva and semen were distinctly identified. Thus, RT-ddPCR is useful for mixed samples (e.g., in sexual assault cases) with low amounts of DNA, which often leads to ambiguous results when using RT-qPCR. Other body fluids (e.g., vaginal fluid and menstrual blood) can also be identified by including additional markers. This study demonstrates the potential of RT-ddPCR for applications in forensic science.

A MLP architecture fusing RGB and CASSI for computational spectral imaging

The coded Aperture Snapshot Spectral Imaging (CASSI) system offers significant advantages in dynamically acquiring hyper-spectral images compared to traditional measurement methods. However, it faces the following challenges: (1) Traditional masks rely on random patterns or analytical design, limiting CASSI’s performance improvement. (2) Existing CASSI reconstruction algorithms do not fully utilize RGB information. (3) High-quality reconstruction algorithms are often slow and limited to offline scene reconstruction. To address these issues, this paper proposes a new MLP architecture, Spectral-Spatial MLP (SSMLP), which replaces the transformer structure with a network using CASSI measurements and RGB as multimodal inputs. This maintains reconstruction quality while significantly improving reconstruction speed. Additionally, we constructed a teacher-student network (SSMLP with a teacher, SSMLP-WT) to transfer the knowledge learned from a large model to a smaller network, further enhancing the smaller network’s accuracy. Extensive experiments show that SSMLP matches the performance of transformer-based structures in spectral image reconstruction while improving inference speed by at least 50%. The reconstruction quality of SSMLP-WT is further improved by knowledge transfer without changing the network, and the teacher boosts the performance by 0.92 dB (44.73 dB vs. 43.81 dB).

Four-frame pixel super-resolution method for lensless imaging systems

Lensless imaging systems have gained significant attention recently due to their advantages in terms of reduced size and weight compared to traditional lens-based systems. However, like other imaging methods, lensless imaging encounters challenges in resolving scenes with more details. In this article, we propose a novel four-frame super-resolution method specifically tailored for lensless imaging systems. Our approach shares similarities with previous lensless imaging systems, involving a sensor and a modulation device placed in front of the image sensor. We develop an explicit degradation downsampling model with sub-pixel shifts and provide the solution to corresponding inverse problem, may offering valuable guidance for other super-resolution imaging algorithms based on spatial displacements. By applying random lateral sub-pixel shifts, acquiring four low-resolution (LR) images, and fusing their spatial information, we achieve high-resolution (HR) sensor recordings, enabling super-resolution reconstruction of the imaging scene. Numerical simulations demonstrate approximately an improvement in spatial resolution compared to single-measurement methods. Furthermore, we evaluate the performance of our method across various lensless imaging systems utilizing different masks, validating its versatility and effectiveness in achieving higher resolution outcomes. Experimental results also support our proposed scheme's ability to achieve higher spatial resolution reconstruction in a real system.

Indoor scene reconstruction from LiDAR point cloud based on roof extraction

Indoor scene reconstruction from LiDAR point clouds is crucial for indoor navigation, scan-to-BIM, and virtual reality. However, occlusions caused by furniture and clutter make this task challenging. This paper introduces a novel framework for reconstructing 3D models using typically unoccluded ceiling point clouds, because the ceiling is difficult to be obstructed by indoor objects and is relatively well preserved. This framework starts with extracting the roof, roof is extracted by a proposed method based on cloth simulation which can extract various shape of roofs. Then a room instance map of the scene is obtained by input the point cloud density map to Mask R-CNN. By overlapping the roof point cloud and room instance map, the whole roof is segmented into roof points of different rooms. Next, we extracted the contours of every room's roof and a new algorithm was proposed to detect connections between different rooms to restore the topological relationship between rooms, generating the scene's floor plan. Finally, by detecting the contours and connections' height, the floor plan is stretched to detected height to generate a 3D model. To evaluate the proposed method's effectiveness, experiments were conducted on two indoor LiDAR point cloud datasets, GibLayout and ISPRS Benchmark, comprising a total of 13 scenes. The results were compared with representative methods, including Floor-sp and Heat for floor plan extraction, and Polyfit and KSR for 3D model reconstruction. The experimental results show that the new method outperform existing approaches, enabling the reconstruction of highly overlapping 3D models of real scenes.

A robust correspondence-based registration method for large-scale outdoor point cloud

ABSTRACT Large-scale outdoor point cloud registration is essential for the 3D reconstruction of outdoor scenes. Its central objective is to achieve accurate point cloud registration by determining accurate spatial transformation parameters. While feature-based methods eliminate the need for initial position estimation, they encounter challenges in handling high outlier rates. Therefore, a method capable of effectively managing outliers is crucial for enhancing the efficiency and accuracy of large-scale outdoor point cloud registration. This paper introduces the maximal clique with adaptive voting (MCAV) method, which leverages graph-based inlier compatibility to optimize potential matches. MCAV employs adaptive parameter voting (APV) to enhance computational efficiency, demonstrating significant speedup characteristics in datasets with a significant number of inliers. To further reduce outliers in potential matches, we integrate Black-Rangarajan Duality (BRD) and graduated non-convexity (GNC) into the truncated least squares (TLS) framework (BG-TLS). Accordingly, we propose the efficient BG-TLS (EBG-TLS) method for computing the registration model. Comparative analyses with traditional and deep learning-based methods across various real-world environments demonstrate that the proposed method outperforms existing algorithms in terms of rotation error, translation error, and efficiency, particularly in complex, high-noise settings. This method finds broad applications in geospatial mapping and surveying, autonomous navigation, and environmental monitoring.

GrARffiti: The Reconstruction and Deployment of Augmented Reality (AR) Graffiti

Graffiti relies on social instrumentation for its creation on spatial structures. It is questioned whether different mechanisms exist to transfer social and spatial hierarchies under a new model for better engagement, management, and governance. This research aims to replace physical graffiti using augmented reality (AR) in smartphones. Contact-free AR graffiti starts with the creation of 3D graffiti; this is followed by an AR cloud platform upload, quick response (QR) code access, and site deployment, leading to the secondary reconstruction of a field scene using smartphone screenshots. The working structure was created based on the first 3D reconstruction of graffiti details as AR models and second 3D reconstruction of field graffiti on different backgrounds using a photogrammetry method. The 3D graffiti can be geotagged as a personal map and 3D printed for collections. This culture-engaged AR creates a two-way method of interacting with spatial structures where the result is collected as a self-governed form of social media. The reinterpreted context is represented by a virtual 3D sticker or symbolized name card shared on the cloud. The hidden or social hierarchy was reinterpreted by a sense of ritual without altering any space. The application of digital stickers in AR redefines the spatial order, typology, and governance of graffiti.