A paralleled embedding high-dimensional Bayesian optimization with additive Gaussian kernels for solving CNOP

Abstract

Conditional Nonlinear Optimal Perturbation (CNOP) is widely used in atmospheric and oceanic predictability studies. Solving CNOP is essentially a nonlinear optimization problem with certain constraints. One method to solve CNOP is the intelligent optimization algorithm. But it may not always obtain the satisfactory solution and efficiency because of the local exploitation of individual particles. The Bayesian optimization algorithm can avoid this local stagnation by building a global probability grasp for the optimization space of CNOP. Nonetheless, the limit of approximately 20-dimensional optimization space hampers its application in solving CNOP of high-dimensional numerical models. To overcome this bottleneck, we propose a paralleled embedding high-dimensional Bayesian optimization with additive Gaussian kernels (PEBOA) algorithm, which mainly consists of the feature extraction process and low-dimensional optimization process. In PEBOA, a feature extraction method (deepFE) using the convolutional Autoencoder with residual connections and customized constraint-preserved loss function is designed, which compresses 10 million dimensional data to relatively low-dimensional search spaces, from tens to hundreds of dimensions. Then, we propose the strategy of additive Gaussian kernels, with which the Bayesian optimization algorithm can optimize in relatively low-dimensional search spaces. Concretely, a kernel handles a subspace, and a subspace can handle dimensions approximately not more than 20. Additionally, the modified acquisition function can sample multiple candidates simultaneously with the aim of accelerating the optimization process. We apply PEBOA to solve CNOP of Regional Ocean Modeling System (ROMS) for identifying optimal initial errors of upstream Kuroshio transport variation, which is a 10 million dimensional and time-consuming problem. The computational performance of PEBOA and physical mechanisms of the obtained CNOP are analyzed. Experimental results indicate that deepFE excels Principal Component Analysis (PCA) in terms of relative error ratio, the magnitude of objective function values, and the obtained CNOP pattern. Besides, compared to deepFE-based Particle Swarm Optimization, PEBOA has better solving efficiency with 3.4% larger objective function values and 2.2 times greater likelihood of obtaining the valid CNOP. Furthermore, the modified acquisition function reduces computation time by about 71.0% with four cores. The physical mechanism analysis shows that CNOPs obtained by PEBOA are almost identical to the adjoint method, which can cause an anomalous increase (decrease) in upstream Kuroshio transport and maintain physics consistency.