Continuum Version Research Articles

Unification of symmetries inside neural networks: transformer, feedforward and neural ODE

Understanding the inner workings of neural networks, including transformers, remains one of the most challenging puzzles in machine learning. This study introduces a novel approach by applying the principles of gauge symmetries, a key concept in physics, to neural network architectures. By regarding model functions as physical observables, we find that parametric redundancies of various machine learning models can be interpreted as gauge symmetries. We mathematically formulate the parametric redundancies in neural ODEs, and find that their gauge symmetries are given by spacetime diffeomorphisms, which play a fundamental role in Einstein’s theory of gravity. Viewing neural ODEs as a continuum version of feedforward neural networks, we show that the parametric redundancies in feedforward neural networks are indeed lifted to diffeomorphisms in neural ODEs. We further extend our analysis to transformer models, finding natural correspondences with neural ODEs and their gauge symmetries. The concept of gauge symmetries sheds light on the complex behavior of deep learning models through physics and provides us with a unifying perspective for analyzing various machine learning architectures.

Stability of Regularized Hastings–Levitov Aggregation in the Subcritical Regime

We prove bulk scaling limits and fluctuation scaling limits for a two-parameter class ALE(α,η)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$({\\alpha },\\eta )$$\\end{document} of continuum planar aggregation models. The class includes regularized versions of the Hastings–Levitov family HL(α)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$({\\alpha })$$\\end{document} and continuum versions of the family of dielectric-breakdown models, where the local attachment intensity for new particles is specified as a negative power -η\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$-\\eta $$\\end{document} of the density of arc length with respect to harmonic measure. The limit dynamics follow solutions of a certain Loewner–Kufarev equation, where the driving measure is made to depend on the solution and on the parameter ζ=α+η\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$${\\zeta }={\\alpha }+\\eta $$\\end{document}. Our results are subject to a subcriticality condition ζ⩽1\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$${\\zeta }\\leqslant 1$$\\end{document}: this includes HL(α)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$({\\alpha })$$\\end{document} for α⩽1\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$${\\alpha }\\leqslant 1$$\\end{document} and also the case α=2,η=-1\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$${\\alpha }=2,\\eta =-1$$\\end{document} corresponding to a continuum Eden model. Hastings and Levitov predicted a change in behaviour for HL(α)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$({\\alpha })$$\\end{document} at α=1\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$${\\alpha }=1$$\\end{document}, consistent with our results. In the regularized regime considered, the fluctuations around the scaling limit are shown to be Gaussian, with independent Ornstein–Uhlenbeck processes driving each Fourier mode, which are seen to be stable if and only if ζ⩽1\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$${\\zeta }\\leqslant 1$$\\end{document}.

Probing the large deviations for the beta random walk in random medium.

We consider a discrete-time random walk on a one-dimensional lattice with space- and time-dependent random jump probabilities, known as the beta random walk. We are interested in the probability that, for a given realization of the jump probabilities (a sample), a walker starting at the origin at time t=0 is at position beyond ξsqrt[T/2] at time T. This probability fluctuates from sample to sample and we study the large-deviation rate function, which characterizes the tails of its distribution at large time T≫1. It is argued that, up to a simple rescaling, this rate function is identical to the one recently obtained exactly by two of the authors for the continuum version of the model. That continuum model also appears in the macroscopic fluctuation theory of a class of lattice gases, e.g., in the so-called KMP model of heat transfer. An extensive numerical simulation of the beta random walk, based on an importance sampling algorithm, is found in good agreement with the detailed analytical predictions. A first-order transition in the tilted measure, predicted to occur in the continuum model, is also observed in the numerics.

Nodal force error and its reduction for material point methods

This work shows that with the presence of a prestress, the discretized momentum equation using the material point method (MPM) or the dual domain material point (DDMP) method is inconsistent with the continuum version of the equation. This inconsistency leads to incorrect wave speed predictions in prestressed materials and numerical instabilities. The source of this inconsistency is the nodal force calculation using the material points.To address this issue, we introduce an auxiliary stress field, which can be regarded as a local background stress. We then reformulate the nodal force calculation, such that it only depends on the local stress variation. The effect of the prestress is then subtracted away in the nodal force calculation, and the consistencies of MPM and DDMP are then restored.This reformulation only requires simple modifications to the original nodal force in both MPM and DDMP. After implementing this reformulated nodal force in a numerical code and comparing the results with manufactured solutions, we found significant improvements not only in the accuracy of the solutions, but also in the numerical stability for both MPM and DDMP. With the reformulated nodal force, the DDMP results converge rapidly as the number of material points increases.

The two-gap BCS model in the large-N approximation within a field-theory approach

We study the continuum version of the two-gap BCS model in (3 + 1)D within the large-N approximation. We calculate the effective potential of the model which depends on two independent energy gaps σ and Δ, where σ describes the Cooper pair made of electrons that belong to the same internal symmetry whereas Δ describes the Cooper pair of electrons that have a different internal symmetry. The effective potential is calculated by considering that the Debye frequency is an ultraviolet cutoff Λ, which is meant to describe the physical lattice of 3D superconductors. Our main result shows that the extra gap provides a possible inter-band phase transition that may be either a stable or metastable phase, depending on the competion between the coupling constants of the model. We also derive the critical temperature below which the phases may be observed.

Majorana Neutrinos and Clockworked Yukawa Couplings Contribution to Nonobservation of the Rare Leptonic Decayli→ljγ, Clockwork Photon and Clockwork Graviton

The clockwork is an extra-dimensional set-up for generating light particles with exponentially suppressed or hierarchical couplings of light particles with N massive states having comparable masses near the threshold scale of the mechanism in theories which contain no small parameters at the fundamental level. We explore the prospect of charged lepton flavour violation (cLFV) in a clockwork framework which encompasses Dirac mass terms as well as Majorana mass terms for the new clockwork fermions. We deive the masses of the non zero clockwork Majorana masses, and new particles in a clockwork framework and for their Yukawa couplings to the lepton doublets, in the framework where the clockwork parameters are universal. When the new clockwork Majorana masses are non zero, neutrino masses are generated as a result from the exchange of heavy messenger particles such as right handed iso$-$singlet neutrinos or iso$ - $triplet scalar bosons known as the seesaw mechanism. In the case of non zero clockwork Majorana masses, owing to the sizable effective Yukawa couplings of the higher mass modes neutrino masses can only be made tiny by conjecturing large Majorana mass in the teraelectron volt range for allthe clockwork gears. This is apparent from the constraints on the mass scale of the clockwork fermions due to the non-observation of the rare cLFV decay $ \mu\rightarrow e\gamma $, $ \tau \rightarrow \mu \gamma $, $ \tau\rightarrow e\gamma $. A general description of the clockwork mechanism valid for fermions, gauge bosons, and gravitons is discussed here. This mechanism can be implemented with a discrete set of new fields or, in its continuum version, through an extra spatial dimension.

An Isogeometric Cloth Simulation Based on Fast Projection Method

A novel continuum-based fast projection scheme is proposed for cloth simulation. Cloth geometry is described by NURBS, and the dynamic response is modeled by a displacement-only Kirchhoff-Love shell element formulated directly on NURBS geometry. The fast projection method, which solves strain limiting as a constrained Lagrange problem, is extended to the continuum version. Numerical examples are studied to demonstrate the performance of the current scheme. The proposed approach can be applied to grids of arbitrary topology and can eliminate unrealistic over-stretching efficiently if compared to spring-based methodologies.

The one-sided lipschitz condition in the follow-the-leader approximation of scalar conservation laws

We consider the follow-the-leader particle approximation scheme for a [Formula: see text] scalar conservation law with non-negative compactly supported [Formula: see text] initial datum and with a [Formula: see text] concave flux, which is known to provide convergence towards the entropy solution [Formula: see text] to the corresponding Cauchy problem. We provide two novel contributions to this theory. First, we prove that the one-sided Lipschitz condition satisfied by the approximate density [Formula: see text] is a “discrete version of an entropy condition”; more precisely, under fairly general assumptions on [Formula: see text] (which imply concavity of [Formula: see text]) we prove that the continuum version [Formula: see text] of said condition allows to select a unique weak solution, despite [Formula: see text] is apparently weaker than the classical Oleinik–Hoff one-sided Lipschitz condition [Formula: see text]. Said result relies on an improved version of Hoff’s uniqueness. A byproduct of it is that the entropy condition is encoded in the particle scheme prior to the many-particle limit, which was never proven before. Second, we prove that in case [Formula: see text] the one-sided Lipschitz condition can be improved to a discrete version of the classical (and “sharp”) Oleinik–Hoff condition. In order to make the paper self-contained, we provide proofs (in some cases “alternative” ones) of all steps of the convergence of the particle scheme.

Deposition, diffusion, and nucleation on an interval

Motivated by nanoscale growth of ultra-thin films, we study a model of deposition, on an interval substrate, of particles that perform Brownian motions until any two meet, when they nucleate to form a static island, which acts as an absorbing barrier to subsequent particles. This is a continuum version of a lattice model studied in the applied literature. We show that the associated interval-splitting process converges in the sparse deposition limit to a Markovian process (in the vein of Brennan and Durrett) governed by a splitting density with a compact Fourier series expansion but, apparently, no simple closed form. We show that the same splitting density governs the fixed deposition rate, large time asymptotics of the normalized gap distribution, so these asymptotics are independent of deposition rate. The splitting density is derived by solving an exit problem for planar Brownian motion from a right-angled triangle, extending work of Smith and Watson.

Examining Teacher Candidates’ Backgrounds, Experiences, and Beliefs as Precursors for Developing Dispositions for Democracy

Background/Context: In this article, we draw on Villegas’s explanations of beliefs as precursors to social justice dispositions and Mills et al.’s Australian research about social justice dispositions as either affirmative or transformative. These conceptual positions assist us in understanding the beliefs that are derived from applicants’ schooling and life experiences. Purpose of Study: The article reports on how 46 candidates applying for an Australian equity-oriented initial teacher education (ITE) program described their backgrounds and beliefs around social justice. The purpose of the research is to understand starting point beliefs in order to design a strongly democratic teacher education program for historically hard-to-staff schools. Setting: Expressions of interest were analyzed from applications to an Australian ITE program designed to prepare teachers for hard-to-staff urban and rural secondary schools in Victoria, Australia. Research Design: This research is a qualitative critical discourse analysis. Our initial activity involved grouping statements using the following five codes: their firsthand, personal backgrounds of disadvantage; their secondhand experience with disadvantage, such as seeing racism from afar or volunteering; their belief statements about equity or social justice; their deficit positions or stereotypes; and their overtly activist statements. Findings/Results: Finding that some of these discourses were much more common than others, we were able to examine applicants’ social justice discourses more deeply. We contend that democratic discourses can be seen on a kind of continuum, with weaker (or thinner) and stronger versions. In general, a weaker version of equity includes generic inclusive beliefs about everyone being equal, whereas a stronger version demonstrates a more complex understanding of social inequities and a commitment to changing systems rather than individuals. Within each category, there were glimpses of weaker and stronger activist experiences and beliefs that we consider precursors to strong social justice or democratic teacher dispositions. Conclusions/Recommendations: Rather than determining selection into the program, the candidates’ social justice statements made in their expressions of interest (EOIs) became starting points to design a program that prepares teachers who are knowledgeable agents of social change. Their statements guided us to understand these starting points in relation to how the candidates narrated their backgrounds, experiences, and beliefs, and enabled us to better explore how social justice is interpreted.

Mechanical Torque Promotes Bipolarity of the Mitotic Spindle Through Multi-centrosomal Clustering.

Intracellular forces shape cellular organization and function. One example is the mitotic spindle, a cellular machine consisting of multiple chromosomes and centrosomes which interact via dynamic microtubule filaments and motor proteins, resulting in complicated spatially dependent forces. For a cell to divide properly, it is important for the spindle to be bipolar, with chromosomes at the center and multiple centrosomes clustered into two 'poles' at opposite sides of the chromosomes. Experimental observations show that in unhealthy cells, the spindle can take on a variety of patterns. What forces drive each of these patterns? It is known that attraction between centrosomes is key to bipolarity, but what prevents the centrosomes from collapsing into a monopolar configuration? Here, we explore the hypothesis that torque rotating chromosome arms into orientations perpendicular to the centrosome-centromere vector promotes spindle bipolarity. To test this hypothesis, we construct a pairwise-interaction model of the spindle. On a continuum version of the model, an integro-PDE system, we perform linear stability analysis and construct numerical solutions which display a variety of spatial patterns. We also simulate a discrete particle model resulting in a phase diagram that confirms that the spindle bipolarity emerges most robustly with torque. Altogether, our results suggest that rotational forces may play an important role in dictating spindle patterning.

Reflectionless Canonical Systems, I: Arov Gauge and Right Limits

In spectral theory, $j$-monotonic families of $2\times 2$ matrix functions appear as transfer matrices of many one-dimensional operators. We present a general theory of such families, in the perspective of canonical systems in Arov gauge. This system resembles a continuum version of the Schur algorithm, and allows to restore an arbitrary Schur function along the flow of associated boundary values at infinity. In addition to results in Arov gauge, this provides a gauge-independent perspective on the Krein-de Branges formula and the reflectionless property of right limits on the absolutely continuous spectrum. This work has applications to inverse spectral problems which have better behavior with respect to a normalization at an internal point of the resolvent domain.

Lp-improving estimates for Radon-like operators and the Kakeya-Brascamp-Lieb inequality

This paper considers the problem of establishing Lp-improving inequalities for Radon-like operators in intermediate dimensions (i.e., for averages overs submanifolds which are neither curves nor hypersurfaces). Due to limitations in existing approaches, previous results in this regime are comparatively sparse and tend to require special numerical relationships between the dimension n of the ambient space and the dimension k of the submanifolds. This paper develops a new approach to this problem based on a continuum version of the Kakeya-Brascamp-Lieb inequality, established by Zhang [28] and extended by Zorin-Kranich [29], and on recent results for geometric nonconcentration inequalities [11]. As an initial application of this new approach, this paper establishes sharp restricted strong type Lp-improving inequalities for certain model quadratic submanifolds in the range k<n≤2k.

Deconstructing higher order clockwork gravity

We consider the higher order clockwork theory of gravitational interactions, whereby a number of gravitons are coupled together with TeV strength, but nevertheless generate a Planck scale coupling to matter without the need for a dilaton. It is shown that the framework naturally lends itself to a five-dimensional geometry, and we find the 5D continuum version of such deconstructed 4D gravitational clockwork models. Moreover, the clockwork picture has matter coupled to particular gravitons, which in the 5D framework looks like a braneworld model, with the Randall-Sundrum model being a special case. More generally, the gravitational clockwork leads to a family of scalar-tensor braneworld models, where the scalar is not a dilaton.

Translational symmetries of quadratic Lagrangians

In this paper, we show that a quadratic Lagrangian, with no constraints, containing ordinary time derivatives up to the order [Formula: see text] of [Formula: see text] dynamical variables, has [Formula: see text] symmetries consisting in the translation of the variables with solutions of the equations of motion. We construct explicitly the generators of these transformations and prove that they satisfy the Heisenberg algebra. We also analyze other specific cases which are not included in our previous statement: the Klein–Gordon Lagrangian, [Formula: see text] Fermi oscillators and the Dirac Lagrangian. In the first case, the system is described by an equation involving partial derivatives, the second case is described by Grassmann variables and the third shows both features. Furthermore, the Fermi oscillator and the Dirac field Lagrangians lead to second class constraints. We prove that also in these last two cases there are translational symmetries and we construct the algebra of the generators. For the Klein–Gordon case we find a continuum version of the Heisenberg algebra, whereas in the other cases, the Grassmann generators satisfy, after quantization, the algebra of the Fermi creation and annihilation operators.

Consistency of Archetypal Analysis

Archetypal analysis is an unsupervised learning method that uses a convex polytope to summarize multivariate data. For fixed $k$, the method finds a convex polytope with $k$ vertices, called archetype points, such that the polytope is contained in the convex hull of the data and the mean squared distance between the data and the polytope is minimal. In this paper, we prove a consistency result that shows if the data is independently sampled from a probability measure with bounded support, then the archetype points converge to a solution of the continuum version of the problem, of which we identify and establish several properties. We also obtain the convergence rate of the optimal objective values under appropriate assumptions on the distribution. If the data is independently sampled from a distribution with unbounded support, we also prove a consistency result for a modified method that penalizes the dispersion of the archetype points. Our analysis is supported by detailed computational experiments of the archetype points for data sampled from the uniform distribution in a disk, the normal distribution, an annular distribution, and a Gaussian mixture model.

Isomorphisms between determinantal point processes with translation-invariant kernels and Poisson point processes

AbstractWe prove the Bernoulli property for determinantal point processes on $ \mathbb{R}^d $ with translation-invariant kernels. For the determinantal point processes on $ \mathbb{Z}^d $ with translation-invariant kernels, the Bernoulli property was proved by Lyons and Steif [Stationary determinantal processes: phase multiplicity, bernoullicity, and domination. Duke Math. J.120 (2003), 515–575] and Shirai and Takahashi [Random point fields associated with certain Fredholm determinants II: fermion shifts and their ergodic properties. Ann. Probab.31 (2003), 1533–1564]. We prove its continuum version. For this purpose, we also prove the Bernoulli property for the tree representations of the determinantal point processes.

Numerical evidences of almost convergence of wave speeds for the Burridge\u2013Knopoff model

This paper deals with the numerical approximation of a stick–slip system, known in the literature as Burridge–Knopoff model, proposed as a simplified description of the mechanisms generating earthquakes. Modelling of friction is crucial and we consider here the so-called velocity-weakening form. The aim of the article is twofold. Firstly, we establish the effectiveness of the classical Predictor–Corrector strategy. To our knowledge, such approach has never been applied to the model under investigation. In the first part, we determine the reliability of the proposed strategy by comparing the results with a collection of significant computational tests, starting from the simplest configuration to the more complicated (and more realistic) ones, with the numerical outputs obtained by different algorithms. Particular emphasis is laid on the Gutenberg–Richter statistical law, a classical empirical benchmark for seismic events. The second part is inspired by the result by Muratov (Phys Rev 59:3847–3857, 1999) providing evidence for the existence of traveling solutions for a corresponding continuum version of the Burridge–Knopoff model. In this direction, we aim to find some appropriate estimate for the crucial object describing the wave, namely its propagation speed. To this aim, motivated by LeVeque and Yee (J Comput Phys 86:187–210, 1990) (a paper dealing with the different topic of conservation laws), we apply a space-averaged quantity (which depends on time) for determining asymptotically an explicit numerical estimate for the velocity, which we decide to name LeVeque–Yee formula after the authors’ name of the original paper. As expected, for the Burridge–Knopoff, due to its inherent discontinuity of the process, it is not possible to attach to a single seismic event any specific propagation speed. More regularity is expected by performing some temporal averaging in the spirit of the Cesàro mean. In this direction, we observe the numerical evidence of the almost convergence of the wave speeds for the Burridge–Knopoff model of earthquakes.

The Linear Dilaton: from the Clockwork Mechanism to Its Supergravity Embedding

In this talk we review the minimal embedding in $$\mathcal{N} = 2$$ , $$D = 5$$ supergravity of a five–dimensional dilaton–gravity theory that serves as a toy model for the holographic dual of six–dimensional little string theory. Interestingly, the toy model also arises in the continuum version of the clockwork mechanism that was recently proposed as a means of addressing the hierarchy problem in a new way. Via a $$U(1)$$ gauging that can account for the dilaton potential, it turns out that the linear dilaton background that is present in the holographic dual induces partial supersymmetry breaking in the supergravity theory.

Spatial gradient sensing and chemotaxis via excitability in Dictyosteliumdiscoideum.

The social amoeba Dictyostelium discoideum performs chemotaxis under starvation conditions, aggregating towards clusters of cells following waves of the signaling molecule cAMP. Cells sense extracellular cAMP and produce internal caches of cAMP to be released, relaying the signal. These events lead to traveling waves of cAMP washing over the population of cells. While much research has been performed to understand the functioning of the chemotaxis network in D. discoideum, limited work has been done to link the operation of the signal relay network with the chemotaxis network to provide a holistic view of the system. We take inspiration from D. discoideum and propose a model that directly links the relaying of a chemical message to the directional sensing of that signal. Utilizing an excitable dynamical systems model that has been previously validated experimentally, we show that it is possible to have both signal amplification and perfect adaptation in a single module. We show that noise plays a vital role in chemotaxing to static gradients, where stochastic tunneling of transient bursts biases the system towards accurate gradient sensing. Moreover, this model also automatically matches its internal time scale of adaptation to the naturally occurring periodicity of the traveling chemical waves generated in the population. Numerical simulations were performed to study the qualitative phenomenology of the system and explore how the system responds to diverse dynamic spatiotemporal stimuli. Finally, we address dynamical instabilities that impede chemotactic ability in a continuum version of the model.