Implicit Representation Research Articles

StegaINR4MIH: steganography by implicit neural representation for multi-image hiding

StegaINR4MIH: steganography by implicit neural representation for multi-image hiding

The neural instantiation of spontaneous counterfactual thought.

Many of the most interesting cognitive feats that humans perform require us to consider not just the things that actually occur but also alternative possibilities. We often do this explicitly (e.g., when imagining precisely how a first date could have gone better), but other times we do it spontaneously and implicitly (e.g., when thinking, "I have to catch this bus," implying bad alternatives if the bus is not caught). A growing body of research has identified a core set of neural processes involved in explicit, episodic counterfactual thinking. Little is known, however, about the processes supporting the spontaneous, possibly implicit representation of alternatives. To make progress on this question, we induced participants to spontaneously generate counterfactual alternatives by asking them to judge whether agents were forced to make a particular choice or chose freely-a judgment that implicitly depends on their alternative options. Using functional magnetic resonance imaging, we found 14 clusters that were preferentially engaged when participants were making force judgments (which elicit the spontaneous consideration of alternatives) compared to judgments of what actually occurred (which do not elicit alternatives). These clusters were widely distributed throughout the brain, including in the bilateral prefrontal cortex, bilateral inferior parietal lobule, bilateral middle and inferior temporal gyri, bilateral posterior cingulate, and bilateral caudate. In many of these regions, we additionally show that variability in the neural signal correlates with trial-by-trial variability in participants' force judgments. Our findings provide a first characterization of the neural substrates of the spontaneous representation of counterfactual alternatives. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

GSEditPro: 3D Gaussian Splatting Editing with Attention‐based Progressive Localization

AbstractWith the emergence of large‐scale Text‐to‐Image(T2I) models and implicit 3D representations like Neural Radiance Fields (NeRF), many text‐driven generative editing methods based on NeRF have appeared. However, the implicit encoding of geometric and textural information poses challenges in accurately locating and controlling objects during editing. Recently, significant advancements have been made in the editing methods of 3D Gaussian Splatting, a real‐time rendering technology that relies on explicit representation. However, these methods still suffer from issues including inaccurate localization and limited manipulation over editing. To tackle these challenges, we propose GSEditPro, a novel 3D scene editing framework which allows users to perform various creative and precise editing using text prompts only. Leveraging the explicit nature of the 3D Gaussian distribution, we introduce an attention‐based progressive localization module to add semantic labels to each Gaussian during rendering. This enables precise localization on editing areas by classifying Gaussians based on their relevance to the editing prompts derived from cross‐attention layers of the T2I model. Furthermore, we present an innovative editing optimization method based on 3D Gaussian Splatting, obtaining stable and refined editing results through the guidance of Score Distillation Sampling and pseudo ground truth. We prove the efficacy of our method through extensive experiments.

Open source tool for Micro-CT aided meso-scale modeling and meshing of complex textile composite structures

Volumetric image-based modeling of textile reinforcements and composites is favored over ideal geometric modeling because of its ability to represent complex structures in sufficient detail. Although several approaches were devised, there is still a scarcity of dedicated tools capable of effectively transferring pertinent information from images to high-fidelity models. This work presents the open source project, PolyTex, a Python-based object-oriented application that establishes a streamlined and reproducible workflow for such tasks. Dual kriging serves as the foundational theory for the parametric approach developed to represent, simplify, and approximate the morphology and topology of fiber tows. The code takes two types of input, either an explicit representation of tow geometry using point clouds or implicit representations, such as image masks representing fiber tows separately with grayscale values. Tailored APIs allow for smooth integration between PolyTex’s modeling capabilities and the simulation environments offered by OpenFOAM and Abaqus. Case studies on virtual testing of textile permeability were presented to demonstrate this capability. The modular and object-oriented design makes PolyTex a highly reusable and extensible tool that allows users to create a customized pipeline.

Neural Surfel Reconstruction: Addressing Loop Closure Challenges in Large-Scale 3D Neural Scene Mapping.

Efficiently reconstructing complex and intricate surfaces at scale remains a significant challenge in 3D surface reconstruction. Recently, implicit neural representations have become a popular topic in 3D surface reconstruction. However, how to handle loop closure and bundle adjustment is a tricky problem for neural methods, because they learn the neural parameters globally. We present an algorithm that leverages the concept of surfels and expands relevant definitions to address such challenges. By integrating neural descriptors with surfels and framing surfel association as a deformation graph optimization problem, our method is able to effectively perform loop closure detection and loop correction in challenging scenarios. Furthermore, the surfel-level representation simplifies the complexity of 3D neural reconstruction. Meanwhile, the binding of neural descriptors to corresponding surfels produces a dense volumetric signed distance function (SDF), enabling the mesh reconstruction. Our approach demonstrates a significant improvement in reconstruction accuracy, reducing the average error by 16.9% compared to previous methods, while also generating modeling files that are up to 90% smaller than those produced by traditional methods.

Spatiotemporal implicit neural representation as a generalized traffic data learner

Spatiotemporal Traffic Data (STTD) measures the complex dynamical behaviors of the multiscale transportation system. Existing methods aim to reconstruct STTD using low-dimensional models. However, they are limited to data-specific dimensions or source-dependent patterns, restricting them from unifying representations. Here, we present a novel paradigm to address the STTD learning problem by parameterizing STTD as an implicit neural representation. To discern the underlying dynamics in low-dimensional regimes, coordinate-based neural networks that can encode high-frequency structures are employed to directly map coordinates to traffic variables. To unravel the entangled spatial–temporal interactions, the variability is decomposed into separate processes. We further enable modeling in irregular spaces such as sensor graphs using spectral embedding. Through continuous representations, our approach enables the modeling of a variety of STTD with a unified input, thereby serving as a generalized learner of the underlying traffic dynamics. It is also shown that it can learn implicit low-rank priors and smoothness regularization from the data, making it versatile for learning different dominating data patterns. We validate its effectiveness through extensive experiments in real-world scenarios, showcasing applications from corridor to network scales. Empirical results not only indicate that our model has significant superiority over conventional low-rank models, but also highlight that the versatility of the approach extends to different data domains, output resolutions, and network topologies. Comprehensive model analyses provide further insight into the inductive bias of STTD. We anticipate that this pioneering modeling perspective could lay the foundation for universal representation of STTD in various real-world tasks. PyTorch implementations of this project is publicly available at:https://github.com/tongnie/traffic_dynamics.

Symmetric Piecewise Developable Approximations

AbstractWe propose a novel method for generating symmetric piecewise developable approximations for shapes in approximately global reflectional or rotational symmetry. Given a shape and its symmetry constraint, the algorithm contains two crucial steps: (i) a symmetric deformation to achieve a nearly developable model and (ii) a symmetric segmentation aided by the deformed shape. The key to the deformation step is the use of the symmetric implicit neural representations of the shape and the deformation field. A new mesh extraction from the implicit function is introduced to construct a strictly symmetric mesh for the subsequent segmentation. The symmetry constraint is carefully integrated into the partition to achieve the symmetric piecewise developable approximation. We demonstrate the effectiveness of our algorithm over various meshes.

GS‐Octree: Octree‐based 3D Gaussian Splatting for Robust Object‐level 3D Reconstruction Under Strong Lighting

AbstractThe 3D Gaussian Splatting technique has significantly advanced the construction of radiance fields from multi‐view images, enabling real‐time rendering. While point‐based rasterization effectively reduces computational demands for rendering, it often struggles to accurately reconstruct the geometry of the target object, especially under strong lighting conditions. Strong lighting can cause significant color variations on the object's surface when viewed from different directions, complicating the reconstruction process. To address this challenge, we introduce an approach that combines octree‐based implicit surface representations with Gaussian Splatting. Initially, it reconstructs a signed distance field (SDF) and a radiance field through volume rendering, encoding them in a low‐resolution octree. This initial SDF represents the coarse geometry of the target object. Subsequently, it introduces 3D Gaussians as additional degrees of freedom, which are guided by the initial SDF. In the third stage, the optimized Gaussians enhance the accuracy of the SDF, enabling the recovery of finer geometric details compared to the initial SDF. Finally, the refined SDF is used to further optimize the 3D Gaussians via splatting, eliminating those that contribute little to the visual appearance. Experimental results show that our method, which leverages the distribution of 3D Gaussians with SDFs, reconstructs more accurate geometry, particularly in images with specular highlights caused by strong lighting. The source code can be downloaded from https://github.com/LaoChui999/GS-Octree.

Neural Vector Fields for Implicit Surface Representation and Inference

AbstractNeural implicit fields have recently shown increasing success in representing, learning and analysis of 3D shapes. Signed distance fields and occupancy fields are still the preferred choice of implicit representations with well-studied properties, despite their restriction to closed surfaces. With neural networks, unsigned distance fields as well as several other variations and training principles have been proposed with the goal to represent all classes of shapes. In this paper, we develop a novel and yet a fundamental representation considering unit vectors in 3D space and call it Vector Field (VF). At each point in $$\mathbb {R}^3$$ R 3 , VF is directed to the closest point on the surface. We theoretically demonstrate that VF can be easily transformed to surface density by computing the flux density. Unlike other standard representations, VF directly encodes an important physical property of the surface, its normal. We further show the advantages of VF representation, in learning open, closed, or multi-layered surfaces. We show that, thanks to the continuity property of the neural optimization with VF, a separate distance field becomes unnecessary for extracting surfaces from the implicit field via Marching Cubes. We compare our method on several datasets including ShapeNet where the proposed new neural implicit field shows superior accuracy in representing any type of shape, outperforming other standard methods. Codes are available at https://github.com/edomel/ImplicitVF.

An interface tracking method with triangle edge cuts

This paper introduces a volume-conserving interface tracking algorithm on unstructured triangle meshes. We propose to discretize the interface via triangle edge cuts which represent the intersections between the interface and the triangle mesh edges using a compact 6 numbers per triangle. This enables an efficient implicit representation of the sub-triangle polygonal material regions without explicitly storing connectivity information. Moreover, we propose an efficient advection algorithm for this interface representation that is based on geometric queries and does not require an optimization process. This advection algorithm is extended via an area correction step that enforces volume-conservation of the materials. We demonstrate the efficacy of our method on a variety of advection problems on a triangle mesh and compare its performance to existing interface tracking methods including VOF and MOF.

Representation of Cultures in Local Italian Language Textbooks for Chinese Universities: A Diachronic Content Analysis

ABSTRACT This study explores the evolving landscape of cultural representations within locally produced Italian as a foreign language (IFL) textbooks in the Chinese educational context. Employing a diachronic content analysis, three distinct textbooks spanning different chronological periods (1972, 1991, 2022) were scrutinised to discern the nuanced evolution of cultural representations. Findings reveal a notable transformation in emphasis, shifting from an initial spotlight on source culture towards an increasing prominence of target culture across successive textbooks. What is more, a consistent emphasis on cultural products, cultural practices, and cultural persons (this latter following Moran, 2009), with an emergence of cultural perspectives in the latest textbook, aligns with an evolving understanding of culture in the domain of foreign language teaching. The observed dynamics, evident through both explicit and implicit representations, underscore the multifaceted relationship between educational priorities, socio-historical contexts, and cultural representations in language textbooks.

Advances in vision-based deep learning methods for interacting hands reconstruction: A survey

Vision-based hand reconstructions have become noteworthy tools in enhancing interactive experiences in various applications such as virtual reality, augmented reality, and autonomous driving, which enable sophisticated interactions by reconstructing complex motions of human hands. Despite significant progress driven by deep-learning methodologies, the quest for high-fidelity interacting hands reconstruction faces challenges such as limited dataset diversity, lack of detailed hand representation, occlusions, and differentiation between similar hand structures. This survey thoroughly reviews deep learning-based methods, diverse datasets, loss functions, and evaluation metrics addressing the complexities of interacting hands reconstruction. Mainstream algorithms of the past five years are systematically classified into two main categories: algorithms that employ explicit representations, such as parametric meshes and 3D Gaussian splatting, and those that utilize implicit representations, including signed distance fields and neural radiance fields. Novel deep-learning models like graph convolutional networks and transformers are applied to solve the aforementioned challenges in hand reconstruction effectively. Beyond summarizing these interaction-aware algorithms, this survey also briefly discusses hand tracking in virtual reality and augmented reality. To the best of our knowledge, this is the first survey specifically focusing on the reconstruction of both hands and their interactions with objects. The survey contains the various facets of hand modeling, deep learning approaches, and datasets, broadening the horizon of hand reconstruction research and future innovation in natural user interactions.

Generating implicit object fragment datasets for machine learning

One of the primary challenges inherent in utilizing deep learning models is the scarcity and accessibility hurdles associated with acquiring datasets of sufficient size to facilitate effective training of these networks. This is particularly significant in object detection, shape completion, and fracture assembly. Instead of scanning a large number of real-world fragments, it is possible to generate massive datasets with synthetic pieces. However, realistic fragmentation is computationally intensive in the preparation (e.g., pre-factured models) and generation. Otherwise, simpler algorithms such as Voronoi diagrams provide faster processing speeds at the expense of compromising realism. In this context, it is required to balance computational efficiency and realism. This paper introduces a GPU-based framework for the massive generation of voxelized fragments derived from high-resolution 3D models, specifically prepared for their utilization as training sets for machine learning models. This rapid pipeline enables controlling how many pieces are produced, their dispersion and the appearance of subtle effects such as erosion. We have tested our pipeline with an archaeological dataset, producing more than 1M fragmented pieces from 1,052 Iberian vessels (). Although this work primarily intends to provide pieces as implicit data represented by voxels, triangle meshes and point clouds can also be inferred from the initial implicit representation. To underscore the unparalleled benefits of CPU and GPU acceleration in generating vast datasets, we compared against a realistic fragment generator that highlights the potential of our approach, both in terms of applicability and processing time. We also demonstrate the synergies between our pipeline and realistic simulators, which frequently cannot select the number and size of resulting pieces. To this end, a deep learning model was trained over realistic fragments and our dataset, showcasing similar results.

Cloud-Edge Collaborative Submap-Based VSLAM Using Implicit Representation Transmission

Cloud-Edge Collaborative Submap-Based VSLAM Using Implicit Representation Transmission

Offline Reinforcement Learning with Constrained Hybrid Action Implicit Representation Towards Wargaming Decision-Making

Offline Reinforcement Learning with Constrained Hybrid Action Implicit Representation Towards Wargaming Decision-Making

PST-Diff: Achieving High-Consistency Stain Transfer by Diffusion Models With Pathological and Structural Constraints.

Histopathological examinations heavily rely on hematoxylin and eosin (HE) and immunohistochemistry (IHC) staining. IHC staining can offer more accurate diagnostic details but it brings significant financial and time costs. Furthermore, either re-staining HE-stained slides or using adjacent slides for IHC may compromise the accuracy of pathological diagnosis due to information loss. To address these challenges, we develop PST-Diff, a method for generating virtual IHC images from HE images based on diffusion models, which allows pathologists to simultaneously view multiple staining results from the same tissue slide. To maintain the pathological consistency of the stain transfer, we propose the asymmetric attention mechanism (AAM) and latent transfer (LT) module in PST-Diff. Specifically, the AAM can retain more local pathological information of the source domain images, while ensuring the model's flexibility in generating virtual stained images that highly confirm to the target domain. Subsequently, the LT module transfers the implicit representations across different domains, effectively alleviating the bias introduced by direct connection and further enhancing the pathological consistency of PST-Diff. Furthermore, to maintain the structural consistency of the stain transfer, the conditional frequency guidance (CFG) module is proposed to precisely control image generation and preserve structural details according to the frequency recovery process. To conclude, the pathological and structural consistency constraints provide PST-Diff with effectiveness and superior generalization in generating stable and functionally pathological IHC images with the best evaluation score. In general, PST-Diff offers prospective application in clinical virtual staining and pathological image analysis.

CardiacField: computational echocardiography for automated heart function estimation using two-dimensional echocardiography probes

Abstract Aims Accurate heart function estimation is vital for detecting and monitoring cardiovascular diseases. While two-dimensional echocardiography (2DE) is widely accessible and used, it requires specialized training, is prone to inter-observer variability, and lacks comprehensive three-dimensional (3D) information. We introduce CardiacField, a computational echocardiography system using a 2DE probe for precise, automated left ventricular (LV) and right ventricular (RV) ejection fraction (EF) estimations, which is especially easy to use for non-cardiovascular healthcare practitioners. We assess the system’s usability among novice users and evaluate its performance against expert interpretations and advanced deep learning (DL) tools. Methods and results We developed an implicit neural representation network to reconstruct a 3D cardiac volume from sequential multi-view 2DE images, followed by automatic segmentation of LV and RV areas to calculate volume sizes and EF values. Our study involved 127 patients to assess EF estimation accuracy against expert readings and two-dimensional (2D) video-based DL models. A subset of 56 patients was utilized to evaluate image quality and 3D accuracy and another 50 to test usability by novice users and across various ultrasound machines. CardiacField generated a 3D heart from 2D echocardiograms with <2 min processing time. The LVEF predicted by our method had a mean absolute error (MAE) of 2.48%, while the RVEF had an MAE of 2.65%. Conclusion Employing a straightforward apical ring scan with a cost-effective 2DE probe, our method achieves a level of EF accuracy for assessing LV and RV function that is comparable to that of three-dimensional echocardiography probes.

Generalized Lipschitz Tracing of Implicit Surfaces

AbstractWe present a versatile and robust framework to render implicit surfaces defined by black‐box functions that only provide function value queries. We assume that the input function is locally Lipschitz continuous; however, we presume no prior knowledge of its Lipschitz constants. Our pre‐processing step generates a discrete acceleration structure, a Lipschitz field, that provides data to infer local and directional Lipschitz upper bounds. These bounds are used to compute safe step sizes along rays during rendering. The Lipschitz field is constructed by generating local polynomial approximations to the input function, then bounding the derivatives of the approximating polynomials. The accuracy of the approximation is controlled by the polynomial degree and the granularity of the spatial resolution used during fitting, which is independent from the resolution of the Lipschitz field. We demonstrate that our process can be implemented in a massively parallel way, enabling straightforward integration into interactive and real‐time modelling workflows. Since the construction only requires function value evaluations, the input surface may be represented either procedurally or as an arbitrarily filtered grid of function samples. We query the original implicit representation upon ray trace, as such, we preserve the geometric and topological details of the input as long as the Lipschitz field supplies conservative estimates. We demonstrate our method on both procedural and discrete implicit surfaces and compare its exact and approximate variants.

Implicit hydromechanical representation of fractures using a continuum approach

Fractures control fluid flow, solute transport, and mechanical deformation in crystalline media. They can be modeled numerically either explicitly or implicitly via an equivalent continuum. The implicit framework implies lower computational cost and complexity. However, upscaling heterogeneous fracture properties for its implicit representation as an equivalent fracture layer remains an open question. In this study, we propose an approach, the Equivalent Fracture Layer (EFL), for the implicit representation of fractures surrounded by low-permeability rock matrix to accurately simulate hydromechanical coupled processes. The approach assimilates fractures as equivalent continua with a manageable scale (≫1 μm) that facilitates spatial discretization, even for large-scale models including multiple fractures. Simulation results demonstrate that a relatively thick equivalent continuum layer (in the order of cm) can represent a fracture (with aperture in the order of μm) and accurately reproduce the hydromechanical behavior (i.e., fluid flow and deformation/stress behavior). There is an upper bound restriction due to the Young's modulus because the equivalent fracture layer should have a lower Young's modulus than that of the surrounding matrix. To validate the approach, we model a hydraulic stimulation carried out at the Bedretto Underground Laboratory for Geosciences and Geoenergies in Switzerland by comparing numerical results against measured data. The method further improves the ability and simplicity of continuum methods to represent fractures in fractured media.

Space-time reconstruction for lensless imaging using implicit neural representations

Many computational imaging inverse problems are challenged by noise, model mismatch, and other imperfections that decrease reconstruction quality. For data taken sequentially in time, instead of reconstructing each frame independently, space-time algorithms simultaneously reconstruct multiple frames, thereby taking advantage of temporal redundancy through space-time priors. This helps with denoising and provides improved reconstruction quality, but often requires significant computational and memory resources. Designing effective but flexible temporal priors is also challenging. Here, we propose using an implicit neural representation to model dynamics and act as a computationally tractable and flexible space-time prior. We demonstrate this approach on video captured with a lensless imager, DiffuserCam, and show improved reconstruction results and robustness to noise compared to frame-by-frame methods.