Continuous Representation Research Articles

NeRF-OR: neural radiance fields for operating room scene reconstruction from sparse-view RGB-D videos.

RGB-D cameras in the operating room (OR) provide synchronized views of complex surgical scenes. Assimilation of this multi-view data into a unified representation allows for downstream tasks such as object detection and tracking, pose estimation, and action recognition. Neural radiance fields (NeRFs) can provide continuous representations of complex scenes with limited memory footprint. However, existing NeRF methods perform poorly in real-world OR settings, where a small set of cameras capture the room from entirely different vantage points. In this work, we propose NeRF-OR, a method for 3D reconstruction of dynamic surgical scenes in the OR. Where other methods for sparse-view datasets use either time-of-flight sensor depth or dense depth estimated from color images, NeRF-OR uses a combination of both. The depth estimations mitigate the missing values that occur in sensor depth images due to reflective materials and object boundaries. We propose to supervise with surface normals calculated from the estimated depths, because these are largely scale invariant. We fit NeRF-OR to static surgical scenes in the 4D-OR dataset and show that its representations are geometrically accurate, where state of the art collapses to sub-optimal solutions. Compared to earlier work, NeRF-OR grasps fine scene details while training 30 faster. Additionally, NeRF-OR can capture whole-surgery videos while synthesizing views at intermediate time values with an average PSNR of 24.86dB. Last, we find that our approach has merit in sparse-view settings beyond those in the OR, by benchmarking on the NVS-RGBD dataset that contains as few as three training views. NeRF-OR synthesizes images with a PSNR of 26.72dB, a 1.7% improvement over state of the art. Our results show that NeRF-OR allows for novel view synthesis with videos captured by a small number of cameras with entirely different vantage points, which is the typical camera setting in the OR. Code is available via: github.com/Beerend/NeRF-OR .

Data-Driven Radial Compressor Design Space Mapping

Abstract Estimates of turbomachinery performance trends inform system-level compromises during preliminary design. Existing empirical correlations for efficiency use limited experimental data, while analytical loss models require calibration to yield predictive results. From a set of 3708 radial compressor computations, this paper maps efficiency as a function of mean-line aerodynamics, and determines the governing loss mechanisms. An open-source turbomachinery design code creates annulus and blade geometry, then runs a Reynolds-averaged Navier–Stokes simulation for compressors sampled from the mean-line design space. Polynomial surface fits yield a continuous eight-dimensional representation of the design space for analysis, predicting efficiency with a root-mean-square error of 1.2% points. The results show a balance between surface dissipation in boundary layers and mixing loss due to casing separations sets optimum values for inlet Mach number, hub-to-tip ratio, de Haller number, and backsweep angle. Surface dissipation drives the effect of flow coefficient, with high surface areas at low values, and high velocities at high values. Compact compressor designs are achieved by increasing inlet Mach number, reducing hub-to-tip ratio, and minimizing the radial loading coefficient—all of which reduce efficiency approaching design space boundaries. An interactive web-based tool makes the results available to practising engineers, demonstrating large ensembles of automated designs and simulations as a higher-fidelity replacement for legacy empirical correlations in preliminary design.

Bridging auditory perception and natural language processing with semantically informed deep neural networks

Sound recognition is effortless for humans but poses a significant challenge for artificial hearing systems. Deep neural networks (DNNs), especially convolutional neural networks (CNNs), have recently surpassed traditional machine learning in sound classification. However, current DNNs map sounds to labels using binary categorical variables, neglecting the semantic relations between labels. Cognitive neuroscience research suggests that human listeners exploit such semantic information besides acoustic cues. Hence, our hypothesis is that incorporating semantic information improves DNN’s sound recognition performance, emulating human behaviour. In our approach, sound recognition is framed as a regression problem, with CNNs trained to map spectrograms to continuous semantic representations from NLP models (Word2Vec, BERT, and CLAP text encoder). Two DNN types were trained: semDNN with continuous embeddings and catDNN with categorical labels, both with a dataset extracted from a collection of 388,211 sounds enriched with semantic descriptions. Evaluations across four external datasets, confirmed the superiority of semantic labeling from semDNN compared to catDNN, preserving higher-level relations. Importantly, an analysis of human similarity ratings for natural sounds, showed that semDNN approximated human listener behaviour better than catDNN, other DNNs, and NLP models. Our work contributes to understanding the role of semantics in sound recognition, bridging the gap between artificial systems and human auditory perception.

A feature extension and reconstruction method with incremental learning capabilities under limited samples for intelligent diagnosis

Existing data-driven Intelligent Fault Diagnosis (IFD) methods often struggle with continuous learning and integrating new diagnostic knowledge. These methods necessitate a complete retraining process when new fault data emerges. Furthermore, collecting sufficient data for model training is impractical in real-world applications. This study proposes a Feature Extension and Reconstruction (FER) method for model incremental updating with limited samples. FER employs scalable and modular feature extractors for continuous representation learning of new fault modes. In addition, a domain mapping algorithm based on optimal transport is devised to leverage old classification nodes in aiding new nodes’ training, enhancing models’ generalization performance under limited samples. A feature reconstruction technique is also developed to compress the expanded multi-backbone model, eliminating redundant parameters and dimensions for adaptability in subsequent incremental tasks. Validated on simulated transmission systems and real planetary gearboxes, FER demonstrates a better stability-plasticity trade-off in incremental tasks with limited training samples, offering a novel solution for adaptive IFD model updates in sample-constrained scenarios.

Dynamic Graph Representation Learning via Coupling-Process Model.

Representation learning based on dynamic graphs has received a lot of attention in recent years due to its wide range of application scenarios. Although many discrete or continuous dynamic graph representation learning methods have been proposed, many of them ignore the role of edge types. Through the observation of dynamic graphs in the real world, it is found that the types of various edges are very different in nature. They are roughly divided into two categories according to the frequency of occurrence: evolutive edges that appear infrequently and interactive edges that appear frequently. For both types of edges, we propose a coupling-process model (DyCPM) to capture the dynamic mechanisms of them. The model not only generates low-dimensional embedding vectors of nodes, but also aggregates the structural information and temporal information of two kinds of edges. In particular, we design a neural network parameterized discrete process to depict the change law of the topology of evolutive edges and a neural network parameterized temporal point process (TPP) to characterize the temporal dynamic rule of interactive edges. More importantly, we propose a coupling mechanism to transfer the information of the two processes through a shared embedding matrix and finally generate an embedding matrix that aggregates the topology information and temporal information of the two kinds of edges for the dynamic link prediction task. We evaluate our model and several baselines on real datasets. The experimental results show that our model can better aggregate the topology information and temporal information of the two kinds of edges according to their properties and outperforms several state-of-the-art baselines in the performance of dynamic link prediction.

VQ-NeRF: Neural Reflectance Decomposition and Editing With Vector Quantization.

We propose VQ-NeRF, a two-branch neural network model that incorporates Vector Quantization (VQ) to decompose and edit reflectance fields in 3D scenes. Conventional neural reflectance fields use only continuous representations to model 3D scenes, despite the fact that objects are typically composed of discrete materials in reality. This lack of discretization can result in noisy material decomposition and complicated material editing. To address these limitations, our model consists of a continuous branch and a discrete branch. The continuous branch follows the conventional pipeline to predict decomposed materials, while the discrete branch uses the VQ mechanism to quantize continuous materials into individual ones. By discretizing the materials, our model can reduce noise in the decomposition process and generate a segmentation map of discrete materials. Specific materials can be easily selected for further editing by clicking on the corresponding area of the segmentation outcomes. Additionally, we propose a dropout-based VQ codeword ranking strategy to predict the number of materials in a scene, which reduces redundancy in the material segmentation process. To improve usability, we also develop an interactive interface to further assist material editing. We evaluate our model on both computer-generated and real-world scenes, demonstrating its superior performance. To the best of our knowledge, our model is the first to enable discrete material editing in 3D scenes.

In-Clinic and Natural Gait Observations master protocol (I-CAN-GO) to validate gait using a lumbar accelerometer

Traditional measurements of gait are typically performed in clinical or laboratory settings where functional assessments are used to collect episodic data, which may not reflect naturalistic gait and activity patterns. The emergence of digital health technologies has enabled reliable and continuous representation of gait and activity in free-living environments. To provide further evidence for naturalistic gait characterization, we designed a master protocol to validate and evaluate the performance of a method for measuring gait derived from a single lumbar-worn accelerometer with respect to reference methods. This evaluation included distinguishing between participants’ self-perceived different gait speed levels, and effects of different floor surfaces such as carpet and tile on walking performance, and performance under different bouts, speed, and duration of walking during a wide range of simulated daily activities. Using data from 20 healthy adult participants, we found different self-paced walking speeds and floor surface effects can be accurately characterized. Furthermore, we showed accurate representation of gait and activity during simulated daily living activities and longer bouts of outside walking. Participants in general found that the devices were comfortable. These results extend our previous validation of the method to more naturalistic setting and increases confidence of implementation at-home.

The continuous Choquet integral representation theorems in an abstract setting

The purpose of the paper is to formulate Choquet integral representation theorems for a monotone functional on a collection of functions in such a way that the representing measures are simultaneously inner and outer continuous on appropriate collections of sets such as open, closed, compact, and measurable. This type of theorem is referred to as the continuous Choquet integral representation theorem and will be discussed in a setting general enough for practical use. The benefits of our results are as follows:(i)the representing measures are simultaneously inner and outer continuous,(ii)the collections of sets for which the representing measures are inner and outer continuous are larger than those in previous studies,(iii)the regularity of the representing measures is also considered, and(iv)it is possible to handle not only σ-continuous but also τ-continuous functionals.

Neural implicit surface reconstruction of freehand 3D ultrasound volume with geometric constraints

Three-dimensional (3D) freehand ultrasound (US) is a widely used imaging modality that allows non-invasive imaging of medical anatomy without radiation exposure. Surface reconstruction of US volume is vital to acquire the accurate anatomical structures needed for modeling, registration, and visualization. However, traditional methods cannot produce a high-quality surface due to image noise. Despite improvements in smoothness, continuity, and resolution from deep learning approaches, research on surface reconstruction in freehand 3D US is still limited. This study introduces FUNSR, a self-supervised neural implicit surface reconstruction method to learn signed distance functions (SDFs) from US volumes. In particular, FUNSR iteratively learns the SDFs by moving the 3D queries sampled around volumetric point clouds to approximate the surface, guided by two novel geometric constraints: sign consistency constraint and on-surface constraint with adversarial learning. Our approach has been thoroughly evaluated across four datasets to demonstrate its adaptability to various anatomical structures, including a hip phantom dataset, two vascular datasets and one publicly available prostate dataset. We also show that smooth and continuous representations greatly enhance the visual appearance of US data. Furthermore, we highlight the potential of our method to improve segmentation performance, and its robustness to noise distribution and motion perturbation.

Mechanical Failure of Core-Shell Cathode Particles: The Effects of Concentration-Dependent Material Properties and Phase Field Fracture Modelling

The use of core-shell cathode particles in lithium-ion batteries is an attractive approach to enhancing energy density whilst retaining lifetime, through reducing undesired reactions at the electrode-electrolyte interface and limiting the volume change of the electrode particles. However, mechanical failure through the fracture and debonding of the core-shell interface is a major challenge. In this work, we employ a coupled finite-element model to predict and mitigate the mechanical failure of core-shell cathode structures, taking as example a particle of NMC811 (core) coated with NMC111 (shell). In particular, we focus on two aspects:The first one involves the assumptions of material properties as inputs of the model. The material properties are often considered constant by battery modelling researchers, yet these parameters can vary significantly during charge/discharge. For example, experiments have unveiled a three-orders-of-magnitude drop in the diffusion coefficient of NMC materials during discharge [1]. Here, we incorporate material properties obtained from experimental data, including concentration-dependent diffusion coefficient obtained from GITT measurement and partial molar volume derived from in situ X-ray diffraction data. Our results indicate that when assuming a concentration-dependent partial molar volume, the maximum values of tensile hoop stress in the shell are nearly three times lower than those predicted with constant average properties, diminishing the likelihood of fracture.When accounting for concentration-dependent diffusion coefficient, large concentration gradient is observed near the outer surface of the core due to reduced lithium mobility at high states of lithiation, hindering full electrode capacity utilisation. The significant concentration gradient and capacity underutilisation align with direct observations from experiments [2]. Notably, these phenomena cannot be captured in modelling if assuming constant diffusion coefficient. These findings offer new perspectives on the performance and design of core-shell electrode particles. Safe design maps in terms of core size and shell thickness to mitigate fracture and debonding are obtained.The second focus of this work is the utilisation of phase field fracture model in predicting cracking behaviours within the core-shell particle. Phase field modelling provides a continuous representation of cracks, enabling the tracking of complex cracking patterns [3]. We predict cracking behaviors caused by volume changes during charge/discharge, as well as inaccessible capacity and newly created active surface area caused by cracking. Our multiphyics model can hopefully act as a catalyst, accelerating the development process of the core-shell structured cathode particles and shortening the time to market of advanced batteries with extended lifetime.

Functional network characteristics in anxiety- and mania-based subgroups of bipolar I disorder

BackgroundBipolar disorder I (BD-I) is a heterogeneous disorder with a high prevalence of comorbid anxiety. The aim of this study was to investigate whether anxiety and mania symptoms define distinct subgroups within BD-I and to explore potential differences in functional network characteristics between these subgroups. MethodsSubgroups were identified using scores from clinical anxiety and mania scales. After dimension reduction of these scores, data-driven clustering analysis with cross-validation was employed to reveal the existence of subgroups. Resting-state functional magnetic resonance imaging (rs-fMRI) scans were pre-processed using fMRIPrep. After parcellation and network construction, global and regional graph theoretical measures were calculated per subgroup. ResultsClustering results revealed that, based on anxiety symptomatology, subjects fell into two distinct subgroups, whereas mania symptoms divided subjects into four unique subgroups. These subgroups varied notably on several symptom scales. Network assortativity was significantly associated with anxiety subgroups. Post-hoc pairwise comparisons did not reveal significant global functional network differences between the anxiety subgroups or between mania subgroups. Regional network differences between clinical subgroups were especially apparent for strength and degree in the temporal and frontal lobes. LimitationsSmall sample size of some subgroups is a limitation of this study as is the categorical rather than continuous representation of anxiety and mania symptoms. ConclusionsBD-I populations may be stratified into robust subgroups based on anxiety and mania symptoms, showing differences in functional network connectivity. Our findings highlight new avenues of research for investigating heterogeneity in psychiatric populations.

Inverse design of TPMS piezoelectric metamaterial based on deep learning

Piezoelectric metamaterials are a class of composite structural materials with a piezoelectric effect that achieve efficient coupling between electrical and mechanical energy through their unique microscopic design. Typically, the tailoring of cellular architectures paves the way for unlocking the potential to achieve unprecedentedly designed materials with specific properties. We have developed three different inverse design frameworks for metamaterials, using deep learning (DL) techniques and underpinned by Bayesian optimisation. These frameworks reveal precise one-to-many correlations between material properties and geometric parameters within the triple periodic minimal surface (TPMS) piezoelectric metamaterial. For the first time, we have introduced the Progressive Latin Hypercube Sampling (PLHS) method to the inverse design process of metamaterials, highlighting the effectiveness of uniform sampling to improve the efficiency of deep learning models. The conditional variational autoencoder (cVAE), the conditional generative adversarial network (cGAN) and the multilayer perceptron (MLP) are studied for comparative analysis. All three models incorporate an inverse design network and a feature predictor to construct a continuous latent representation of a large number of hybrid TPMS (H-TPMS). After extensive training, the models can generate various piezoelectric porous structures that accurately meet the required material property parameters. Our results demonstrate the high accuracy of cVAE in inverse design, with 98.92% agreement for piezoelectric values and 96.44% agreement for mechanical properties. The three inverse design frameworks show different generalisation capabilities, with cVAE performing best. This study thoroughly analyses the strengths and limitations of each model, providing valuable insights for researchers seeking inverse design recommendations.

Hamiltonian formulation of linear non-Hermitian systems

In the case of a linear non-Hermitian system, I prove that it's possible to construct a Hamiltonian in such a way that the equations governing the non-Hermitian system can be exactly expressed using Hamilton's canonical equations. Initially, I demonstrate this within the discrete representation framework and subsequently extend it to continuous representation. Through this formulation employing the Hamiltonian, I can pinpoint a conserved charge using Noether's theorem and identify adiabatic invariants. When this approach is applied to Hermitian systems, all the obtained results converge to the well-known outcomes associated with the Schrödinger equation.

Neural Gaussian Scale-Space Fields

Gaussian scale spaces are a cornerstone of signal representation and processing, with applications in filtering, multiscale analysis, anti-aliasing, and many more. However, obtaining such a scale space is costly and cumbersome, in particular for continuous representations such as neural fields. We present an efficient and lightweight method to learn the fully continuous, anisotropic Gaussian scale space of an arbitrary signal. Based on Fourier feature modulation and Lipschitz bounding, our approach is trained self-supervised, i.e., training does not require any manual filtering. Our neural Gaussian scale-space fields faithfully capture multiscale representations across a broad range of modalities, and support a diverse set of applications. These include images, geometry, light-stage data, texture anti-aliasing, and multiscale optimization.

Phase-field anisotropic damage with bond-slip model for reinforced concrete beam under the service load

Concrete, as a quasi-brittle material, undergoes a latent damage accumulation preceding macroscopic crack formation, which is further complicated by the intricate interaction between steel reinforcements and concrete in reinforced concrete (RC) structures. Traditional approaches inadequately capture these nuances, especially when it comes to the seamless transition from numerous micro cracks to a visible macro cracks. Phase-field damage theory stands out as a sophisticated tool, integrating sharp cracks into a continuous damage field representation and detailing the entire process of micro-to-macroscopic cracking evolution. In this work, a phase-field damage model that accounts for bond-slip behavior is proposed, grounded in the principle of stress space decomposition to address the inherent tension–compression anisotropy. Newton’s method with a viscosity coefficient of 0.001 enhances simulation accuracy and robustness through optimal convergence. The comparison between rigid bond and bond slip mechanisms indicates a significant difference in their impact on crack modes, revealing the necessity of considering bond slip. As the steel reinforcement ratio increases, the RC beam’s load-bearing capacity increases incrementally, concurrently exhibiting a reduction in macroscopic cracks, steel reinforcement stress, and overall damage severity. However, the comparative advantage of rigid bond in improving RC beam’s resistance against cracking begins to wane as the reinforcement proportion rises. This study thus sheds light on the nuanced interplay between bond characteristics and the cracking behavior of RC beams, providing a comprehensive and advanced perspective.

Gradient-based optimization for quantum architecture search

Quantum Architecture Search (QAS) has shown significant promise in designing quantum circuits for Variational Quantum Algorithms (VQAs). However, existing QAS algorithms primarily explore circuit architectures within a discrete space, which is inherently inefficient. In this paper, we propose a Gradient-based Optimization for Quantum Architecture Search (GQAS), which leverages a circuit encoder, decoder, and predictor. Initially, the encoder embeds circuit architectures into a continuous latent representation. Subsequently, a predictor utilizes this continuous latent representation as input and outputs an estimated performance for the given architecture. The latent representation is then optimized through gradient descent within the continuous latent space based on the predicted performance. The optimized latent representation is finally mapped back to a discrete architecture via the decoder. To enhance the quality of the latent representation, we pre-train the encoder on a substantial dataset of circuit architectures using Self-Supervised Learning (SSL). Our simulation results on the Variational Quantum Eigensolver (VQE) indicate that our method outperforms the current Differentiable Quantum Architecture Search (DQAS).

Order embedding theorems and multi-utility representation of the preorder

As an improvement of Fletcher-Lindgren order embedding theorem, we show that every completely regular ordered space X is order embedded in the Tychonoff ordered cube of infinite weight of X. For that purpose, we evaluate the minimal cardinality of continuous functions which represent multi-utility of the (pre)order on X. Moreover, for a topological preordered space X which admits a continuous multi-utility representation (or a completely regular ordered space X) and its compact subspace, similar descriptions by maps to Tychonoff ordered cube are also given.

Opt2Vec - a continuous optimization problem representation based on the algorithm's behavior: A case study on problem classification

Characterization of the optimization problem is a crucial task in many recent optimization research topics (e.g., explainable algorithm performance assessment, and automated algorithm selection and configuration). The state-of-the-art approaches use exploratory landscape analysis to represent the optimization problem, where for each one, a set of features is extracted using a set of candidate solutions sampled by a sampling strategy over the whole decision space. This paper proposes a novel representation of continuous optimization problems by encoding the information found in the interaction between an algorithm and an optimization problem. The new problem representation is learned using the information from the states/positions in the optimization run trajectory (i.e., the candidate solutions visited by the algorithm). With the novel representation, the problem can be characterized dynamically during the optimization run, instead of using a set of candidate solutions from the whole decision space that have never been observed by the algorithm. The novel optimization problem representation is called Opt2Vec and uses an autoencoder type of neural network to encode the information found in the interaction between an optimization algorithm and optimization problem into an embedded subspace. The Opt2Vec representation efficiency is shown by enabling different optimization problems to be successfully identified using only the information obtained from the optimization run trajectory.

Joint inference of discrete cell types and continuous type-specific variability in single-cell datasets with MMIDAS.

Reproducible definition and identification of cell types is essential to enable investigations into their biological function, and understanding their relevance in the context of development, disease and evolution. Current approaches model variability in data as continuous latent factors, followed by clustering as a separate step, or immediately apply clustering on the data. We show that such approaches can suffer from qualitative mistakes in identifying cell types robustly, particularly when the number of such cell types is in the hundreds or even thousands. Here, we propose an unsupervised method, MMIDAS, which combines a generalized mixture model with a multi-armed deep neural network, to jointly infer the discrete type and continuous type-specific variability. Using four recent datasets of brain cells spanning different technologies, species, and conditions, we demonstrate that MMIDAS can identify reproducible cell types and infer cell type-dependent continuous variability in both uni-modal and multi-modal datasets.

Learning Pixel-Wise Continuous Depth Representation via Clustering for Depth Completion

Learning Pixel-Wise Continuous Depth Representation via Clustering for Depth Completion