Abstract
Deep Learning using the eponymous deep neural networks (DNNs) has become an attractive approach towards various data-based problems of theoretical physics in the past decade. There has been a clear trend to deeper architectures containing increasingly more powerful and involved layers. Contrarily, Taylor coefficients of DNNs still appear mainly in the light of interpretability studies, where they are computed at most to first order. However, especially in theoretical physics numerous problems benefit from accessing higher orders, as well. This gap motivates a general formulation of neural network (NN) Taylor expansions. Restricting our analysis to multilayer perceptrons (MLPs) and introducing quantities we refer to as propagators and vertices, both depending on the MLP's weights and biases, we establish a graph-theoretical approach. Similarly to Feynman rules in quantum field theories, we can systematically assign diagrams containing propagators and vertices to the corresponding partial derivative. Examining this approach for S-wave scattering lengths of shallow potentials, we observe NNs to adapt their derivatives mainly to the leading order of the target function's Taylor expansion. To circumvent this problem, we propose an iterative NN perturbation theory. During each iteration we eliminate the leading order, such that the next-to-leading order can be faithfully learned during the subsequent iteration. After performing two iterations, we find that the first- and second-order Born terms are correctly adapted during the respective iterations. Finally, we combine both results to find a proxy that acts as a machine-learned second-order Born approximation.
Highlights
Machine learning (ML) is a highly active field of research that provides a wide range of tools to tackle various data-based problems
We observe two distinct classes of quantities, which we refer to as propagators and vertices, that each depend on the weights, biases, and chosen activation functions and naturally appear in such a tensor formulation. Their naming is intentional, as we discover several similarities between the Taylor expansion of multilayer perceptrons (MLPs) and perturbation theory in quantum field theories: Analogously to Feynman rules, we find underlying rules that specify which combinations of vertices and propagators, i.e., which diagrams, are allowed and contribute to a given Taylor coefficient
Since we observe these MLPs to mainly adapt their derivatives to the leading order, we develop an iterative scheme that can be understood as an neural network (NN) perturbation theory to successively obtain remaining terms of the target function’s Taylor expansion: At each iteration, the idea is to eliminate the leading order from the current targets in the training and test sets, which generates new data sets for the iteration, which a new auxiliary ensemble of MLPs can be trained on
Summary
Machine learning (ML) is a highly active field of research that provides a wide range of tools to tackle various data-based problems. Applying the proposed graph-theoretical formalism, we complement these findings by a quantitative investigation of the dominating analytical structure of MLP ensembles that predict S-wave scattering lengths Since we observe these MLPs to mainly adapt their derivatives to the leading order, we develop an iterative scheme that can be understood as an NN perturbation theory to successively obtain remaining terms of the target function’s Taylor expansion: At each iteration, the idea is to eliminate the leading order from the current targets in the training and test sets, which generates new data sets for the iteration, which a new auxiliary ensemble of MLPs can be trained on.
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
