Infinite Limit Research Articles

An analytic theory of shallow networks dynamics for hinge loss classification* *This article is an updated version of: Pellegrini F and Biroli G 2020 An analytic theory of shallow networks dynamics for hinge loss classification Advances in Neural Information Processing Systems vol 33 ed H Larochelle, M Ranzato, R Hadsell, M F Balcan and H Lin (New York: Curran Associates)

Neural networks have been shown to perform incredibly well in classification tasks over structured high-dimensional datasets. However, the learning dynamics of such networks is still poorly understood. In this paper we study in detail the training dynamics of a simple type of neural network: a single hidden layer trained to perform a classification task. We show that in a suitable mean-field limit this case maps to a single-node learning problem with a time-dependent dataset determined self-consistently from the average nodes population. We specialize our theory to the prototypical case of a linearly separable data and a linear hinge loss, for which the dynamics can be explicitly solved in the infinite dataset limit. This allows us to address in a simple setting several phenomena appearing in modern networks such as slowing down of training dynamics, crossover between rich and lazy learning, and overfitting. Finally, we assess the limitations of mean-field theory by studying the case of large but finite number of nodes and of training samples.

Asymptotics of streamwise Reynolds stress in wall turbulence

The scaling of different features of streamwise normal stress profiles$\langle uu\rangle ^+(y^+)$in turbulent wall-bounded flows is the subject of a long-running debate. Particular points of contention are the scaling of the ‘inner’ and ‘outer’ peaks of$\langle uu\rangle ^+$at$y^+\approxeq ~15$and$y^+ ={O}(10^3)$, respectively, their infinite Reynolds number limit, and the rate of logarithmic decay in the outer part of the flow. Inspired by the thought-provoking paper of Chen & Sreenivasan (J. Fluid Mech., vol. 908, 2021, p. R3), two terms of an inner asymptotic expansion of$\langle uu\rangle ^+$in the small parameter$Re_{\tau }^{-1/4}$are constructed from a set of direct numerical simulations (DNS) of channel flow. This inner expansion is for the first time matched through an overlap layer to an outer expansion, which not only fits the same set of channel DNS within 1.5 % of the peak stress, but also provides a good match of laboratory data in pipes and the near-wall part of boundary layers, up to the highest$Re_{\tau }$values of$10^5$. The salient features of the new composite expansion are first, an inner$\langle uu\rangle ^+$peak, which saturates at 11.3 and decreases as$Re_{\tau }^{-1/4}$. This inner peak is followed by a short ‘wall log law’ with a slope that becomes positive for$Re_{\tau }$beyond${O}(10^4)$, leading up to an outer peak, followed by the logarithmic overlap layer with a negative slope going continuously to zero for$Re_{\tau }\to \infty$.

Fermionic Condensate in de Sitter Spacetime

Fermionic condensate is investigated in $(D+1)$-dimensional de Sitter spacetime by using the cutoff function regularization. In order to fix the renormalization ambiguity for massive fields an additional condition is imposed, requiring the condensate to vanish in the infinite mass limit. For large values of the field mass the condensate decays exponentially in odd dimensional spacetimes and follows a power law decay in even dimensional spacetimes. For a massless field the fermionic condensate vanishes for odd values of the spatial dimension $D$ and is nonzero for even $D$. Depending on the spatial dimension the fermionic condensate can be either positive or negative. The change in the sign of the condensate may lead to instabilities in interacting field theories.

Hydrophobicity of Self-Assembled Monolayers of Alkanes: Fluorination, Density, Roughness, and Lennard-Jones Cutoffs.

The interplay of fluorination and structure of alkane self-assembled monolayers and how these affect hydrophobicity are explored via molecular dynamics simulations, contact angle goniometry, and surface-enhanced infrared absorption spectroscopy. Wetting coefficients are found to grow linearly in the monolayer density for both alkane and perfluoroalkane monolayers. The larger contact angles of monolayers of perfluorinated alkanes are shown to be primarily caused by their larger molecular volume, which leads to a larger nearest-neighbor grafting distance and smaller tilt angle. Increasing the Lennard-Jones force cutoff in simulations is found to increase hydrophilicity. Specifically, wetting coefficients scale like the inverse square of the cutoff, and when extrapolated to the infinite cutoff limit, they yield contact angles that compare favorably to experimental values. Nanoscale roughness is also found to reliably increase monolayer hydrophobicity, mostly via the reduction of the entropic part of the work of adhesion. Analysis of depletion lengths shows that droplets on nanorough surfaces partially penetrate the surface, intermediate between Wenzel and Cassie-Baxter states.

On the empirical spectral distribution for certain models related to sample covariance matrices with different correlations

Given [Formula: see text], we study two classes of large random matrices of the form [Formula: see text] where for every [Formula: see text], [Formula: see text] are iid copies of a random variable [Formula: see text], [Formula: see text], [Formula: see text] are two (not necessarily independent) sets of independent random vectors having different covariance matrices and generating well concentrated bilinear forms. We consider two main asymptotic regimes as [Formula: see text]: a standard one, where [Formula: see text], and a slightly modified one, where [Formula: see text] and [Formula: see text] while [Formula: see text] for some [Formula: see text]. Assuming that vectors [Formula: see text] and [Formula: see text] are normalized and isotropic “in average”, we prove the convergence in probability of the empirical spectral distributions of [Formula: see text] and [Formula: see text] to a version of the Marchenko–Pastur law and the so-called effective medium spectral distribution, correspondingly. In particular, choosing normalized Rademacher random variables as [Formula: see text], in the modified regime one can get a shifted semicircle and semicircle laws. We also apply our results to the certain classes of matrices having block structures, which were studied in [G. M. Cicuta, J. Krausser, R. Milkus and A. Zaccone, Unifying model for random matrix theory in arbitrary space dimensions, Phys. Rev. E 97(3) (2018) 032113, MR3789138; M. Pernici and G. M. Cicuta, Proof of a conjecture on the infinite dimension limit of a unifying model for random matrix theory, J. Stat. Phys. 175 (2) (2019) 384–401, MR3968860].

Linking ghost penalty and aggregated unfitted methods

In this work, we analyse the links between ghost penalty stabilisation and aggregation-based discrete extension operators for the numerical approximation of elliptic partial differential equations on unfitted meshes. We explore the behaviour of ghost penalty methods in the limit as the penalty parameter goes to infinity, which returns a strong version of these methods. We observe that these methods suffer locking in that limit. On the contrary, aggregated finite element spaces are locking-free because they can be expressed as an extension operator from well-posed to ill-posed degrees of freedom. Next, we propose novel ghost penalty methods that penalise the distance between the solution and its aggregation-based discrete extension. These methods are locking-free and converge to aggregated finite element methods in the infinite penalty parameter limit. We include an exhaustive set of numerical experiments in which we compare weak (ghost penalty) and strong (aggregated finite elements) schemes in terms of error quantities, condition numbers and sensitivity with respect to penalty coefficients on different geometries, intersection locations and mesh topologies.

Explaining universality: infinite limit systems in the renormalization group method

I analyze the role of infinite idealizations used in the renormalization group (RG hereafter) method in explaining universality across microscopically different physical systems in critical phenomena. I argue that despite the reference to infinite limit systems such as systems with infinite correlation lengths during the RG process, the key to explaining universality in critical phenomena need not involve infinite limit systems. I develop my argument by introducing what I regard as the explanatorily relevant property in RG explanations: linearization* property; I then motivate and prove a proposition about the linearization* property in support of my view. As a result, infinite limit systems in RG explanations are dispensable.

The quantum p-spin glass model: a user manual for holographers

We study a large-N bosonic quantum mechanical sigma-model with a spherical target space subject to disordered interactions, more colloquially known as the p-spin spherical model. Replica symmetry is broken at low temperatures and for sufficiently weak quantum fluctuations, which drives the system into a spin glass (SG) phase. The first half of this paper is dedicated to a discussion of this model’s thermodynamics, with particular emphasis on the marginally stable SG. This phase exhibits an emergent conformal symmetry in the strong coupling regime, which dictates its thermodynamic properties. It is associated with an extensive number of nearby states in the free energy landscape. We discuss in detail an elegant approximate solution to the SG equations, which interpolates between the conformal regime and an ultraviolet-complete short distance solution. In the second half of this paper we explore the real-time dynamics of the model and uncover quantum chaos as measured by out-of-time-order four-point functions, both numerically and analytically. We find exponential Lyapunov growth, which intricately depends on the model’s couplings and becomes strongest in the quantum critical regime. We emphasize that the SG phase also exhibits quantum chaos, albeit with parametrically smaller Lyapunov exponent than in the replica symmetric phase. An analytical calculation in the marginal SG phase suggests that this Lyapunov exponent vanishes in a particular infinite coupling limit. We comment on the potential meaning of these observations from the perspective of holography.

Wiener, edge-Wiener, and vertex-edge-Wiener index of Basilica graphs

We determine the exact value of the Wiener index, the edge-Wiener index, and the vertex-edge-Wiener index of the Basilica graphs, i.e., the sequence of finite Schreier graphs associated with the action of the Basilica group on the rooted binary tree. Moreover, we give a formula for the total distance of every vertex in the Basilica graphs, and we are able to make it explicit for some special vertices. We finally introduce the notions of asymptotic Wiener index and asymptotic total distance, which are compatible with that of convergence of the sequence of finite Basilica graphs to an infinite orbital limit graph in the Gromov–Hausdorff topology: the asymptotic values are explicitly computed.

Local density of states and scattering rates across the many-body localization transition

Characterizing the many-body localization (MBL) transition in strongly disordered and interacting quantum systems is an important issue in the field of condensed matter physics. We study the single particle Green's functions for a disordered interacting system in one dimension using exact diagnonalization in the infinite temperature limit and provide strong evidence that single particle excitations carry signatures of delocalization to MBL transition. In the delocalized phase, the typical values of the local density of states and the scattering rate are finite while in the MBL phase, the typical values for both the quantities become vanishingly small. The probability distribution functions of the local density of states and the scattering rate are broad log-normal distributions in the delocalized phase while the distributions become very narrow and sharply peaked close to zero in the MBL phase. We also study the eigenstate Green's function for all the many-body eigenstates and demonstrate that both, the energy resolved typical scattering rate and the typical local density of states, can track the many-body mobility edges.

When and why PINNs fail to train: A neural tangent kernel perspective

Physics-informed neural networks (PINNs) have lately received great attention thanks to their flexibility in tackling a wide range of forward and inverse problems involving partial differential equations. However, despite their noticeable empirical success, little is known about how such constrained neural networks behave during their training via gradient descent. More importantly, even less is known about why such models sometimes fail to train at all. In this work, we aim to investigate these questions through the lens of the Neural Tangent Kernel (NTK); a kernel that captures the behavior of fully-connected neural networks in the infinite width limit during training via gradient descent. Specifically, we derive the NTK of PINNs and prove that, under appropriate conditions, it converges to a deterministic kernel that stays constant during training in the infinite-width limit. This allows us to analyze the training dynamics of PINNs through the lens of their limiting NTK and find a remarkable discrepancy in the convergence rate of the different loss components contributing to the total training error. To address this fundamental pathology, we propose a novel gradient descent algorithm that utilizes the eigenvalues of the NTK to adaptively calibrate the convergence rate of the total training error. Finally, we perform a series of numerical experiments to verify the correctness of our theory and the practical effectiveness of the proposed algorithms. The data and code accompanying this manuscript are publicly available at https://github.com/PredictiveIntelligenceLab/PINNsNTK.

Insertion and diffusion of N and C in [formula omitted]–TiAl: Theoretical study and comparison with O

Nitrogen and carbon play an important role in the oxidation of Ti-Al intermetallic alloys in air. The insertion and diffusion of these elements in the γ-TiAl phase are investigated by first-principle computations. The accommodation of C and N atoms in several likely interstitial positions has been evaluated using density functional theory (DFT) calculations. The results show that both carbon and nitrogen prefer Ti-rich environments. Then, considering the possible jumps among the stable and metastable interstitial sites, the diffusion coefficients have also been obtained from ab initio calculations. According to the Transitional State Theory, in order to compute atomic jump rates, diffusion energy barriers and vibrational modes need to be known. Herein, barrier energies are obtained using the Climbing Image Nudged Elastic Band method. Vibrational properties are computed using the finite displacement method. Finally, diffusion coefficients are obtained solving the transport equation in the infinite time limit, using an analytical approach. The obtained results are compared to the diffusion of oxygen in γ-TiAl, investigated in previous studies. An anisotropic diffusion is obtained for all the interstitial species.

Deformed Morse-like potential

We introduce an exactly solvable one-dimensional potential that supports both bound and/or resonance states. This potential is a generalization of the well-known 1D Morse potential where we introduced a deformation that preserves the finite spectrum property. On the other hand, in the limit of zero deformation, the potential reduces to the exponentially confining potential well introduced recently by Alhaidari [Theor. Math. Phys. 206, 84–96 (2021)]. The latter potential supports an infinite spectrum, which means that the zero deformation limit is a critical point where our system will transition from the finite spectrum limit to the infinite spectrum limit. We solve the corresponding Schrodinger equation and obtain the energy spectrum and the eigenstates using the tridiagonal representation approach.

Pulse shape effects in high-field Bethe–Heitler pair production

Electron–positron pair production via the nonlinear Bethe–Heitler effect in the combined fields of a bare nucleus and a high-intensity laser pulse is studied theoretically. The calculations are performed within the framework of strong-field quantum electrodynamics using a flat-top laser profile with raising and falling edges. This way, the dependence of the pair production process on the precise shape of the laser field is analyzed. Our approach allows us, in particular, to follow the evolution of the created particles’ energy spectra from ultra-short few-cycle pulses to the monochromatic infinite pulse-train limit. We show how the various portions of the pulse influence these spectra and determine conditions for which the outcome from a laser pulse closely resembles the predictions from monochromatic theory.

The EFT stringy viewpoint on large distances

We observe a direct relation between the existence of fundamental axionic strings, dubbed EFT strings, and infinite distance limits in 4d mathcal{N} = 1 EFTs coupled to gravity. The backreaction of EFT strings can be interpreted as RG flow of their couplings, and allows one to probe different regimes within the field space of the theory. We propose that any 4d EFT infinite distance limit can be realised as an EFT string flow. We show that along such limits the EFT string becomes asymptotically tensionless, and so the EFT eventually breaks down. This provides an upper bound for the maximal field range of an EFT with a finite cut-off, and reproduces the Swampland Distance Conjecture from a bottom-up perspective. Even if there are typically other towers of particles becoming light, we propose that the mass of the leading tower scales as m2 ∼ mathcal{T} w in Planck units, with mathcal{T} the EFT string tension and w a positive integer. Our results hold even in the presence of a non-trivial potential, as long as its energy scale remains well below the cut-off. We check both proposals for large classes of 4d mathcal{N} = 1 string compactifications, finding that only the values w = 1, 2, 3 are realised.

Dissipation-Driven Selection under Finite Diffusion: Hints from Equilibrium and Separation of Time Scales.

When exposed to a thermal gradient, reaction networks can convert thermal energy into the chemical selection of states that would be unfavourable at equilibrium. The kinetics of reaction paths, and thus how fast they dissipate available energy, might be dominant in dictating the stationary populations of all chemical states out of equilibrium. This phenomenology has been theoretically explored mainly in the infinite diffusion limit. Here, we show that the regime in which the diffusion rate is finite, and also slower than some chemical reactions, might bring about interesting features, such as the maximisation of selection or the switch of the selected state at stationarity. We introduce a framework, rooted in a time-scale separation analysis, which is able to capture leading non-equilibrium features using only equilibrium arguments under well-defined conditions. In particular, it is possible to identify fast-dissipation sub-networks of reactions whose Boltzmann equilibrium dominates the steady-state of the entire system as a whole. Finally, we also show that the dissipated heat (and so the entropy production) can be estimated, under some approximations, through the heat capacity of fast-dissipation sub-networks. This work provides a tool to develop an intuitive equilibrium-based grasp on complex non-isothermal reaction networks, which are important paradigms to understand the emergence of complex structures from basic building blocks.

Distribution Amplitudes of K^{*} and ϕ at the Physical Pion Mass from Lattice QCD.

We present the first lattice QCD calculation of the distribution amplitudes of longitudinally and transversely polarized vector mesons K^{*} and ϕ using large momentum effective theory. We use the clover fermion action on three ensembles with 2+1+1 flavors of highly improved staggered quarks action, generated by the MIMD Lattice Computation Collaboration, at physical pion mass and {0.06,0.09,0.12} fm lattice spacings and choose three different hadron momenta P_{z}={1.29,1.72,2.15} GeV. The resulting lattice matrix elements are nonperturbatively renormalized in a recently proposed hybrid scheme. An extrapolation to the continuum and infinite momentum limit is carried out. We find that, while the longitudinal distribution amplitudes tend to be close to the asymptotic form, the transverse ones deviate rather significantly from the asymptotic form. Our final results provide crucial abinitio theory inputs for analyzing pertinent exclusive processes.

Non-negativity of BMN two-point functions with three string modes

Recently, we proposed a novel entry of the pp-wave holographic dictionary, which equated the Berenstein-Maldacena-Nastase (BMN) two-point functions in free mathcal{N} = 4 super-Yang-Mills theory with the norm squares of the quantum unitary transition amplitudes between the corresponding tensionless strings in the infinite curvature limit, for the cases with no more than three string modes in different transverse directions. A seemingly highly non-trivial conjectural consequence, particularly in the case of three string modes, is the non-negativity of the BMN two-point functions at any higher genus for any mode numbers. In this paper, we further perform the detailed calculations of the BMN two-point functions with three string modes at genus two, and explicitly verify that they are always non-negative through mostly extensive numerical tests.

On inferring evolutionary stability in finite populations using infinite population models

Models of evolution by natural selection often make the simplifying assumption that populations are infinitely large. In this infinite population limit, rare mutations that are selected against always go extinct, whereas in finite populations they can persist and even reach fixation. Nevertheless, for mutations of arbitrarily small phenotypic effect, it is widely believed that in sufficiently large populations, if selection opposes the invasion of rare mutants, then it also opposes their fixation. Here, we identify circumstances under which infinite-population models do or do not accurately predict evolutionary outcomes in large, finite populations. We show that there is no population size above which considering only invasion generally suffices: for any finite population size, there are situations in which selection opposes the invasion of mutations of arbitrarily small effect, but favours their fixation. This is not an unlikely limiting case; it can occur when fitness is a smooth function of the evolving trait, and when the selection process is biologically sensible. Nevertheless, there are circumstances under which opposition of invasion does imply opposition of fixation: in fact, for the n-player snowdrift game (a common model of cooperation) we identify sufficient conditions under which selection against rare mutants of small effect precludes their fixation—in sufficiently large populations—for any selection process. We also find conditions under which—no matter how large the population—the trait that fixes depends on the selection process, which is important because any particular selection process is only an approximation of reality.

Spin-state energetics of manganese spin crossover complexes: Comparison of single-reference and multi-reference ab initio approaches

Manganese spin crossover (SCO) complexes form a small but ever expanding family of compounds with thermally accessible states of different electronic configuration and total spin. Accurate prediction of spin-state energetics is essential for the theoretical description of these systems. However, this represents a challenging problem that necessitates recourse to correlated wave function methods rather than the more approximate density functional theory (DFT). Here we present a detailed study of spin-state energetics for eight Mn(III) and Mn(II) SCO complexes using the domain-based local pair natural orbital approach to coupled cluster theory with singles, doubles, and perturbative triples, DLPNO-CCSD(T). The effects of reference determinants, basis set, triples excitations, and pair natural orbitals (PNO) thresholds are evaluated and analysed in detail, enabling us to propose a robust and efficient computational protocol based on a combined and balanced mix of extrapolation to the complete basis set and infinite PNO space limits. The results are subsequently used to evaluate multireference wavefunction-based (CASSCF/NEVPT2) and DFT approaches, highlighting their inability to provide a balanced description of spin-state energetics for these complexes. The DLPNO-CCSD(T) protocol proposed in this study can serve as a generally applicable reference-quality quantum chemical method for studying spin crossover systems.