Statistical Physics Perspective Research Articles

A Statistical Physics Perspective: Understanding the Causality Behind Convolutional Neural Network Adversarial Vulnerability.

The adversarial vulnerability of convolutional neural networks (CNNs) refers to the performance degradation of CNNs under adversarial attacks, leading to incorrect decisions. However, the causes of adversarial vulnerability in CNNs remain unknown. To address this issue, we propose a unique cross-scale analytical approach from a statistical physics perspective. It reveals that the huge amount of nonlinear effects inherent in CNNs is the fundamental cause for the formation and evolution of system vulnerability. Vulnerability is spontaneously formed on the macroscopic level after the symmetry of the system is broken through the nonlinear interaction between microscopic state order parameters. We develop a cascade failure algorithm, visualizing how micro perturbations on neurons' activation can cascade and influence macro decision paths. Our empirical results demonstrate the interplay between microlevel activation maps and macrolevel decision-making and provide a statistical physics perspective to understand the causality behind CNN vulnerability. Our work will help subsequent research to improve the adversarial robustness of CNNs.

Finite-size analysis in neural network classification of critical phenomena.

We analyze the problem of supervised learning of ferromagnetic phase transitions from the statistical physics perspective. We consider two systems in two universality classes, the two-dimensional Ising model and two-dimensional Baxter-Wu model, and perform careful finite-size analysis of the results of the supervised learning of the phases of each model. We find that the variance of the neural network (NN) output function (VOF) as a function of temperature has a peak in the critical region. Qualitatively, the VOF is related to the classification rate of the NN. We find that the width of the VOF peak displays the finite-size scaling governed by the correlation length exponent ν of the universality class of the model. We check this conclusion using several NN architectures-a fully connected NN, a convolutional NN, and several members of the ResNet family-and discuss the accuracy of the extracted critical exponents ν.

Survival strategies of artificial active agents

Artificial cells can be engineered to display dynamics sharing remarkable features in common with the survival behavior of living organisms. In particular, such active systems can respond to stimuli provided by the environment and undertake specific displacements to remain out of equilibrium, e.g. by moving towards regions with higher fuel concentration. In spite of the intense experimental activity aiming at investigating this fascinating behavior, a rigorous definition and characterization of such “survival strategies” from a statistical physics perspective is still missing. In this work, we take a first step in this direction by adapting and applying to active systems the theoretical framework of Transition Path Theory, which was originally introduced to investigate rare thermally activated transitions in passive systems. We perform experiments on camphor disks navigating Petri dishes and perform simulations in the paradigmatic active Brownian particle model to show how the notions of transition probability density and committor function provide the pivotal concepts to identify survival strategies, improve modeling, and obtain and validate experimentally testable predictions. The definition of survival in these artificial systems paves the way to move beyond simple observation and to formally characterize, design and predict complex life-like behaviors.

Guest Editorial: Scientific and Physics-Informed Machine Learning for Industrial Applications

Deep learning technology has become one of the core driving forces to promote the in-depth development of industrial automation. In [A1], Wang et al. interpreted the decision process of the convolutional neural network (CNN) by constructing a percolation model from a statistical physics perspective. In this perspective, the decision-making basis of CNN is difficult to understand, because CNN is usually used as a black box model. Furthermore, a novel concept of the differentiation degree and summarized an empirical formula for quantifying the differentiation degree is presented and discussed.

Confined Brownian suspensions: Equilibrium diffusion, thermodynamics, and rheology

We examine the impact of confinement on the structure, dynamics, and rheology of spherically confined macromolecular suspensions, with a focus on the role played by entropic forces, by comparing the limits of strong hydrodynamics and no hydrodynamics. We present novel measurements of the osmotic pressure, intrinsic viscosity, and long-time self-diffusivity in spherical confinement and find confinement induces strong structural correlations and restrictions on configurational entropy that drive up osmotic pressure and viscosity and drive down self-diffusion. Even in the absence of hydrodynamics, confinement produces distinct short-time and long-time self-diffusion regimes. This finding revises the previous understanding that short-time self-diffusion is a purely hydrodynamic quantity. The entropic short-time self-diffusion is proportional to an entropic mobility, a direct analog to the hydrodynamic mobility. A caging plateau following the short-time regime is stronger and more durable without hydrodynamics, and entropic drift—a gradient in volume fraction—drives particles out of their cages. The distinct long-time regime emerges when an entropic mobility gradient arising from heterogeneous distribution of particle volume drives particles out of local cages. We conclude that entropic mobility gradients produce a distinct long-time dynamical regime in confinement and that hydrodynamic interactions weaken this effect. From a statistical physics perspective, confinement restricts configurational entropy, driving up confined osmotic pressure, viscosity, and (inverse) long-time dynamics as confinement tightens. We support this claim by rescaling the volume fraction as the distance from confinement-dependent maximum packing, which collapses the data for each rheological measure onto a single curve.

Non-extensive entropy and fragment–asperity interaction model for earthquakes

In this study, we developed a fragment–asperity interaction model for earthquakes. Based on Tsallis entropy, this model can provide mathematical expression for the non-extensive entropy of fragments and asperities in the gouge area. From a statistical physics perspective, we hypothesized that non-extensive entropy decreases with energy relaxation after crustal rupture (i.e., following an earthquake). By using a windowing process, the non-extensive entropy value was monitored for three relatively recent earthquakes: the 2009 M6.1 L’Aquila (Italy) earthquake, 2011 M9.0 Tohoku (Japan) megathrust earthquake, and 2016 M7.8 Kaikoura (New Zealand) earthquake. The results support our hypothesis and suggest that the non-extensive entropy formula obtained in this study can provide a good characterization of seismicity for a given area.

An Introduction to Machine Learning: a perspective from Statistical Physics

The recent progresses in Machine Learning opened the door to actual applications of learning algorithms but also to new research directions both in the field of Machine Learning directly and, at the edges with other disciplines. The case that interests us is the interface with physics, and more specifically Statistical Physics. In this short lecture, I will try to present first a brief introduction to Machine Learning from the angle of neural networks. After explaining quickly some fundamental models and global aspects of the training procedure, I will discuss into more detail two examples illustrate what can be done from the Statistical Physics perspective.

Exact results on high-dimensional linear regression via statistical physics.

It is clear that conventional statistical inference protocols need to be revised to deal correctly with the high-dimensional data that are now common. Most recent studies aimed at achieving this revision rely on powerful approximation techniques that call for rigorous results against which they can be tested. In this context, the simplest case of high-dimensional linear regression has acquired significant new relevance and attention. In this paper we use the statistical physics perspective on inference to derive several exact results for linear regression in the high-dimensional regime.

DLR Equations and Rigidity for the Sine‐Beta Process

AbstractWe investigate Sineβ, the universal point process arising as the thermodynamic limit of the microscopic scale behavior in the bulk of one‐dimensional log‐gases, or β‐ensembles, at inverse temperature β > 0. We adopt a statistical physics perspective, and give a description of Sineβ using the Dobrushin‐Lanford‐Ruelle (DLR) formalism by proving that it satisfies the DLR equations: the restriction of Sineβ to a compact set, conditionally on the exterior configuration, reads as a Gibbs measure given by a finite log‐gas in a potential generated by the exterior configuration. In short, Sineβ is a natural infinite Gibbs measure at inverse temperature β > 0 associated with the logarithmic pair potential interaction. Moreover, we show that Sineβ is number‐rigid and tolerant in the sense of Ghosh‐Peres; i.e., the number, but not the position, of particles lying inside a compact set is a deterministic function of the exterior configuration. Our proof of the rigidity differs from the usual strategy and is robust enough to include more general long‐range interactions in arbitrary dimension. © 2020 Wiley Periodicals, Inc.

Criticality in the conformational phase transition among self-similar groups in intrinsically disordered proteins: Probed by salt-bridge dynamics

Intrinsically disordered proteins (IDP) serve as one of the key components in the global proteome. In contrast to globular proteins, they harbor an enormous amount of physical flexibility enforcing them to be retained in conformational ensembles rather than stable folds. Previous studies in an aligned direction have revealed the importance of transient dynamical phenomena like that of salt-bridge formation in IDPs to support their physical flexibility and have further highlighted their functional relevance. For this characteristic flexibility, IDPs remain amenable and accessible to different ordered binding partners, supporting their potential multi-functionality. The current study further addresses this complex structure-functional interplay in IDPs using phase transition dynamics to conceptualize the underlying (avalanche type) mechanism of their being distributed across and hopping around degenerate structural states (conformational ensembles). For this purpose, extensive molecular dynamics simulations have been done and the data analyzed from a statistical physics perspective. Investigation of the plausible scope of 'self-organized criticality' (SOC) to fit into the complex dynamics of IDPs was found to be assertive, relating the conformational degeneracy of these proteins to their functional multiplicity. In accordance with the transient nature of 'salt-bridge dynamics', the study further uses it as a probe to explain the structural basis of the proposed criticality in the conformational phase transition among self-similar groups in IDPs. The analysis reveal scale-invariant self-similar fractal geometries in the structural conformations of different IDPs. The insights from the study has the potential to be extended further to benefit structural tinkering of IDPs in their functional characterization and drugging.

Critical Behavior and Universality Classes for an Algorithmic Phase Transition in Sparse Reconstruction

Recovery of an N-dimensional, K-sparse solution $${\mathbf {x}}$$ from an M-dimensional vector of measurements $${\mathbf {y}}$$ for multivariate linear regression can be accomplished by minimizing a suitably penalized least-mean-square cost $$||{\mathbf {y}}-{\mathbf {H}} {\mathbf {x}}||_2^2+\lambda V({\mathbf {x}})$$ . Here $${\mathbf {H}}$$ is a known matrix and $$V({\mathbf {x}})$$ is an algorithm-dependent sparsity-inducing penalty. For ‘random’ $${\mathbf {H}}$$ , in the limit $$\lambda \rightarrow 0$$ and $$M,N,K\rightarrow \infty $$ , keeping $$\rho =K/N$$ and $$\alpha =M/N$$ fixed, exact recovery is possible for $$\alpha $$ past a critical value $$\alpha _c = \alpha (\rho )$$ . Assuming $${\mathbf {x}}$$ has iid entries, the critical curve exhibits some universality, in that its shape does not depend on the distribution of $${\mathbf {x}}$$ . However, the algorithmic phase transition occurring at $$\alpha =\alpha _c$$ and associated universality classes remain ill-understood from a statistical physics perspective, i.e. in terms of scaling exponents near the critical curve. In this article, we analyze the mean-field equations for two algorithms, Basis Pursuit ( $$V({\mathbf {x}})=||{\mathbf {x}}||_{1} $$ ) and Elastic Net ( $$V({\mathbf {x}})= ||{\mathbf {x}}||_{1} + \tfrac{g}{2} ||{\mathbf {x}}||_{2}^2$$ ) and show that they belong to different universality classes in the sense of scaling exponents, with mean squared error (MSE) of the recovered vector scaling as $$\lambda ^\frac{4}{3}$$ and $$\lambda $$ respectively, for small $$\lambda $$ on the critical line. In the presence of additive noise, we find that, when $$\alpha >\alpha _c$$ , MSE is minimized at a non-zero value for $$\lambda $$ , whereas at $$\alpha =\alpha _c$$ , MSE always increases with $$\lambda $$ .

Statistical physics perspective of fracture in brittle and quasi-brittle materials.

We discuss the physics of fracture in terms of the statistical physics associated with the failure of elastic media under applied stresses in presence of quenched disorder. We show that the development and the propagation of fracture are largely determined by the strength of the disorder and the stress field around them. Disorder acts as nucleation centres for fracture. We discuss Griffith's law for a single crack-like defect as a source for fracture nucleation and subsequently consider two situations: (i) low disorder concentration of the defects, where the failure is determined by the extreme value statistics of the most vulnerable defect (nucleation regime) and (ii) high disorder concentration of the defects, where the scaling theory near percolation transition is applicable. In this regime, the development of fracture takes place through avalanches of a large number of tiny microfractures with universal statistical features. We discuss the transition from brittle to quasi-brittle behaviour of fracture with the strength of disorder in the mean-field fibre bundle model. We also discuss how the nucleation or percolation mode of growth of fracture depends on the stress distribution range around a defect. We discuss the corresponding numerical simulation results on random resistor and spring networks.This article is part of the theme issue 'Statistical physics of fracture and earthquakes'.

Elasticity and plasticity of disordered systems, a statistical physics perspective

Elasticity and plasticity are generally considered as engineering topics. In these lectures, I will try to show that they can also be seen as interesting problems from a statistical physics standpoint, emphasising the universal, material independent features. I will focus on the case of amorphous solids, which has undergone many recent experimental and theoretical developments.

A Statistical Physics Perspective to Understand Social Visual Attention in Autism Spectrum Disorder

This study investigated social visual attention in children with Autism Spectrum Disorder (ASD) and with typical development (TD) in the light of Brockmann and Geisel's model of visual attention. The probability distribution of gaze movements and clustering of gaze points, registered with eye-tracking technology, was studied during a free visual exploration of a gaze stimulus. A data-driven analysis of the distribution of eye movements was chosen to overcome any possible methodological problems related to the subjective expectations of the experimenters about the informative contents of the image in addition to a computational model to simulate group differences. Analysis of the eye-tracking data indicated that the scanpaths of children with TD and ASD were characterized by eye movements geometrically equivalent to Lévy flights. Children with ASD showed a higher frequency of long saccadic amplitudes compared with controls. A clustering analysis revealed a greater dispersion of eye movements for these children. Modeling of the results indicated higher values of the model parameter modulating the dispersion of eye movements for children with ASD. Together, the experimental results and the model point to a greater dispersion of gaze points in ASD.

A statistical physics viewpoint on the dynamics of the bouncing ball

We compute, in a statistical physics perspective, the dynamics of a bouncing ball maintained in a chaotic regime thanks to collisions with a plate experiencing an aperiodic vibration. We analyze in details the energy exchanges between the bead and the vibrating plate, and show that the coupling between the bead and the plate can be modeled in terms of both a dissipative process and an injection mechanism by an energy reservoir. An analysis of the injection statistics in terms of fluctuation relation is also provided.

Out-of-equilibrium dynamics and statistics of dispersive PDE

The purpose of this note is to report on some recent advances in the study of out-of-equilibrium behavior of dispersive PDE. One can address this problematic from two different perspectives: a dynamical systems one, and a statistical physics one. The dynamical systems perspective corresponds to constructing solutions exhibiting “energy cascade” between scales, whereas the statistical physics perspective corresponds to deriving effective equations for the dynamics under some “macroscopic limits” in what is often called wave turbulence theory. The rigorous justification of this theory is an outstanding open problem from a rigorous mathematical point of view, and we will touch on it here. We shall discuss some recent attempts to better understand both of the above perspectives.

A statistical physics perspective on alignment-independent protein sequence comparison.

Motivation: Within bioinformatics, the textual alignment of amino acid sequences has long dominated the determination of similarity between proteins, with all that implies for shared structure, function and evolutionary descent. Despite the relative success of modern-day sequence alignment algorithms, so-called alignment-free approaches offer a complementary means of determining and expressing similarity, with potential benefits in certain key applications, such as regression analysis of protein structure-function studies, where alignment-base similarity has performed poorly.Results: Here, we offer a fresh, statistical physics-based perspective focusing on the question of alignment-free comparison, in the process adapting results from ‘first passage probability distribution’ to summarize statistics of ensemble averaged amino acid propensity values. In this article, we introduce and elaborate this approach.Contact: d.r.flower@aston.ac.uk

Heterogeneity, interaction and emergence: effects of composition

The present paper emphasises that socioeconomic systems are complex systems due to two entangled categories characterising their constituents: heterogeneity and interaction as the main sources of complexity and emergence. This perspective is developed by reasoning on the double–face problem of microfoundation and aggregation in macroeconomic modelling taking care of heterogeneity, interaction and composition effects of individual behaviours into the emergence of macroscopic phenomena. With reference to the literature of this field, based on the statistical physics perspective in conjunction with the agent based modelling, an approach is discussed to analyse complex systems dynamics by means of a restricted set of equations explaining the dynamics of the drifting path trajectory and the distribution of fluctuations.

A statistical physics perspective on criticality in financial markets

Stock markets are complex systems exhibiting collective phenomena and particular features such as synchronization, fluctuations distributed as power-laws, non-random structures and similarity to neural networks. Such specific properties suggest that markets operate at a very special point. Financial markets are believed to be critical by analogy to physical systems, but little statistically founded evidence has been given. Through a data-based methodology and comparison to simulations inspired by the statistical physics of complex systems, we show that the Dow Jones and index sets are not rigorously critical. However, financial systems are closer to criticality in the crash neighborhood.

Self-similarity of rRNA secondary structures: A clue to RNA folding

In this paper, we analyze helices in the secondary structures of the 16S and 23S rRNAs from the statistical physics perspective. The results of the analysis lead to propose a possible mechanism of the RNA folding based on the premise that the structure of RNA may bear a trace of its folding. We show that the frequency distribution of the helix contact order approximately follows a power-law, which implies that helices of large contact orders should inevitably exist. Furthermore, the frequencies of helix contact orders can be characterized by the multifractal. Comprehending the multifractality and the power-law of the distribution of the helix contact orders, we suggest a nearest-preferred helix formation as a mechanism for RNA folding via a random binary multiplicative process. The proposed process was supported by reconstructing the multifractal spectrum based on the transfer matrix theory and the binary tree representation of helices in the secondary structures. This justifies, at least partially if not entirely, the relevance of the proposed process as the kinetics of RNA folding.