The advance of neural ordinary differential ordinary differential equations

Abstract

Differential methods are widely used to describe complex continuous processes. The main idea of ordinary differential equations is to treat a specific type of neural network as a discrete equation. Therefore, the differential equation solver can be used to optimize the solution process of the neural network. Compared with the conventional neural network solution, the solution process of the neural ordinary differential equation has the advantages of high storage efficiency and adaptive calculation. This paper first gives a brief review of the residual network (ResNet) and the relationship of ResNet to neural ordinary differential equations. Besides, his paper list three advantages of neural ordinary differential equations compared with ResNet and introduce the class of Deep Neural Network (DNN) models that can be seen as numerical discretization of neural ordinary differential equations (N-ODEs). Furthermore, this paper analyzes a defect of neural ordinary differential equations that do not appear in the traditional deep neural network. Finally, this paper demonstrates how to analyze ResNet with neural ordinary differential equations and shows the main application of neural ordinary differential equations (Neural-ODEs).