Invariance-Based Approach Explains Empirical Formulas from Pavement Engineering to Deep Learning

Abstract

In many application areas, there are effective empirical formulas that need explanation. In this paper, we focus on two such challenges: neural networks, where a so-called softplus activation function is known to be very efficient, and pavement engineering, where there are empirical formulas describing the dependence of the pavement strength on the properties of the underlying soil. We show that similar scale-invariance ideas can explain both types of formulas – and, in the case of pavement engineering, invariance ideas can lead to a new formula that combines the advantages of several known ones.