Application of physics encoded neural networks to improve predictability of properties of complex multi-scale systems

Abstract

Predicting physical properties of complex multi-scale systems is a common challenge and demands analysis of various temporal and spatial scales. However, physics alone is often not sufficient due to lack of knowledge on certain details of the system. With sufficient data, however, machine learning techniques may aid. If data are yet relatively cumbersome to obtain, hybrid methods may come to the rescue. We focus in this report on using various types of neural networks (NN) including NN’s into which physics information is encoded (PeNN’s) and also studied effects of NN’s hyperparameters. We apply the networks to predict the viscosity of an emulsion as a function of shear rate. We show that using various network performance metrics as the mean squared error and the coefficient of determination (R2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$R^2$$\\end{document}) that the PeNN’s always perform better than the NN’s, as also confirmed by a Friedman test with a p-value smaller than 0.0002. The PeNN’s capture extrapolation and interpolation very well, contrary to the NN’s. In addition, we have found that the NN’s hyperparameters including network complexity and optimization methods do not have any effect on the above conclusions. We suggest that encoding NN’s with any disciplinary system based information yields promise to better predict properties of complex systems than NN’s alone, which will be in particular advantageous for small numbers of data. Such encoding would also be scalable, allowing different properties to be combined, without repetitive training of the NN’s.