QR-DeepONet: Resolve abnormal convergence issue in deep operator network

Abstract

Listen

Translate article

Abstract Deep operator network (DeepONet) has been proven to be highly successful in operator learning tasks.
Theoretical analysis indicates that the generation error of DeepONet should decrease as the basis dimension increases, 
thus providing a systematic way to reduce its generalization errors by varying the network hyperparameters.
However, in practice, we found that, depending on the problem being solved and the activation function used, 
the generalization errors fluctuate unpredictably, contrary to theoretical expectations. 
Upon analyzing the output matrix of the trunk net, we determined that this behavior stems from the learned basis 
functions being highly linearly dependent, which limits the expressivity of the vanilla DeepONet.
To address these limitations, we propose QR-DeepONet, an enhanced version of DeepONet using QR decomposition.
These modifications ensured that the learned basis functions were linearly independent and orthogonal to each other.
The numerical results demonstrate that the generalization errors of QR-DeepONet follow theoretical predictions that decrease monotonically as the basis dimension increases and outperform vanilla DeepONet.
Consequently, the proposed method successfully fills the gap between the theory and practice.