L-BFGS Research Articles

Stochastic adaptive CL-BFGS algorithms for fully complex-valued dendritic neuron model

This paper proposes two stochastic variance reduced gradient algorithms based on adaptive complex-valued limited-memory BFGS (ACL-BFGS) algorithm, which are established on the calculation of Wirtinger gradient for fully complex-valued dendritic neuron model (FCDNM). Basically, the explicit expression of Wirtinger gradient is derived to enable the quasi-Newton algorithm for the training of FCDNM. A stochastic variance reduction strategy is adopted such that the number of samples per batch is increased during the optimization process to ensure the effectiveness of the stochastic algorithm. The stochastic ACL-BFGS algorithm with variance reduction (SACL-BFGS) can be efficiently applied for the training of FCDNM. Furthermore, by generalizing the A3 accumulative multi-step quasi-Newton method, the A3-SACL-BFGS algorithm is presented to achieve fast convergence. The experimental results on some benchmark prediction and classification problems conclusively imply that the proposed SACL-BFGS and A3-SACL-BFGS algorithms for FCDNM are clearly superior to complex-valued backpropagation (CBP), CL-BFGS and ACL-BFGS algorithms.

Three-Dimensional Inversion of Long-Offset Transient Electromagnetic Method over Topography

The long-offset transient electromagnetic method (LOTEM) is widely employed in geophysical exploration, including environmental investigation, mineral exploration, and geothermal resource exploration. However, most interpretations of LOTEM data assume a flat Earth, and the commonly used one-dimensional (1D) interpretation encounters significant challenges in achieving reliable geological interpretations when topography is ignored. To address these challenges, this study presents an effective three-dimensional (3D) LOTEM inversion method. In this study, we discretize the simulation domain using unstructured tetrahedra to accurately simulate complex geological structures. The finite-element time-domain (FETD) method is utilized to calculate the LOTEM forward responses, and the limited-memory BFGS (L-BFGS) optimization method is employed for 3D LOTEM inversion. To avoid explicit calculation of sensitivity, we obtain the product of the transposed sensitivity matrix and the vector through adjoint forward modeling. Several synthetic models are used to verify the developed program, and the influence of topography on LOTEM inversion is examined. The numerical results demonstrate that topography can significantly impact the inversion result, potentially leading to incorrect geological interpretations. Finally, the developed inversion algorithm is applied to a realistic ore model from Voisey’s Bay, Labrador, Canada. The 3D inversion successfully reconstructs the spatial distribution of the ore body, further confirming the effectiveness of the developed algorithm.

Solving partial differential equations with hybridized physic-informed neural network and optimization approach: Incorporating genetic algorithms and L-BFGS for improved accuracy

Partial differential equations (PDEs) are essential mathematical models for describing a wide range of physical phenomena. Numerically, Physic-Informed Neural Networks (PINNs), a variant of artificial neural networks, present a promising method for solving PDEs. However, due to limitation in accuracy and stability, various adaptive PINN variants have been proposed. We have designed a novel approach that adopted self-adaptive PINN (SA-PINN) with two optimization techniques: the genetic algorithm (GA) and the limited-memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) algorithm. Self-adaptive PINN modifies the weights in the loss function to be fully trainable, enabling the ANN to learn and stabilize the PINN in approximating the difficult regions of the solution. GA initializes the population of ANN trainable parameters to optimize the training process with less number of iterations, while L-BFGS is used to find the best solution accurately. Our proposed approach, named SA-PINN-GA-LBFGS, is tested on solving several benchmark PDE problems including elliptic, parabolic, and hyperbolic types. We compare our results with state-of-the-art methods, demonstrating that SA-PINN-GA-LBFGS provides higher accuracy and greater efficiency.

Classification of CO Environmental Parameter for Air Pollution Monitoring with Grammatical Evolution

Air pollution is a pressing concern in urban areas, necessitating the critical monitoring of air quality to understand its implications for public health. Internet of Things (IoT) devices are widely utilized in air pollution monitoring due to their sensor capabilities and seamless data transmission over the Internet. Artificial intelligence (AI) and machine learning techniques play a crucial role in classifying patterns derived from sensor data. Environmental stations offer a multitude of parameters that can be obtained to uncover hidden patterns showcasing the impact of pollution on the surrounding environment. This paper focuses on utilizing the CO parameter as an indicator of pollution in two datasets collected from wireless environmental monitoring devices in the greater Port area and the Town Hall of Igoumenitsa City in Greece. The datasets are normalized to facilitate their utilization in classification algorithms. The k-means algorithm is applied, and the elbow method is used to determine the optimal number of clusters. Subsequently, the datasets are introduced to the grammatical evolution algorithm to calculate the percentage fault. This method constructs classification programs in a human-readable format, making it suitable for analysis. Finally, the proposed method is compared against four state-of-the-art models: the Adam optimizer for optimizing artificial neural network parameters, a genetic algorithm for training an artificial neural network, the Bayes model, and the limited-memory BFGS method applied to a neural network. The comparison reveals that the GenClass method outperforms the other approaches in terms of classification error.

Full waveform inversion based on hybrid conjugate gradient with BFGS direction

Full waveform inversion (FWI) has the problem of low computational efficiency. The main reasons are low convergence rate and unstable convergence. We mainly use optimization methods to solve computationally inefficient problems. Among the commonly used optimization methods, the L-BFGS (Limited memory Broyden-Fletcher-Goldfarb-Shanno) method converges fast but not stably, while the HCG (Hybrid Conjugate gradient) method converges stably but slowly. Neither of these methods can maximize the computational efficiency of FWI. By using Gauss-Newton direction to approximate the search direction (QN-HCG) of HCG method, the second-order convergence rate of Gauss-Newton method and the strong convergence stability of HCG method can be used at the same time. However, QN-HCG requires the Hessian matrix to be positive definite, which is complicated to compute and occupies a large memory. The memoryless BFGS method is developed from the L-BFGS method. Since the Hessian matrix is not required to be positive definite, the method is simple to compute and occupies less memory, and is therefore more suitable to approximate the search direction of the HCG method. First, in order to improve the convergence speed of FWI and ensure the stability of convergence, we use BFGS without memory variable to approximate the search direction of HCG method (BFGS-HCG), and introduce this method into FWI. Second, the computational efficiency of the proposed method is verified in model tests of acoustic and elastic wave FWI. Model tests show that this approach can improve the computational efficiency of full waveform inversion.

Fast High-Resolution Phase Diversity Wavefront Sensing with L-BFGS Algorithm

The presence of manufacture error in large mirrors introduces high-order aberrations, which can severely influence the intensity distribution of point spread function. Therefore, high-resolution phase diversity wavefront sensing is usually needed. However, high-resolution phase diversity wavefront sensing is restricted with the problem of low efficiency and stagnation. This paper proposes a fast high-resolution phase diversity method with limited memory Broyden–Fletcher–Goldfarb–Shanno (L-BFGS) algorithm, which can accurately detect aberrations in the presence of high-order aberrations. An analytical gradient of the objective function for phase-diversity is integrated into the framework of the L-BFGS nonlinear optimization algorithm. L-BFGS algorithm is specifically suitable for high-resolution wavefront sensing where a large phase matrix is optimized. The performance of phase diversity with L-BFGS is compared to other iterative method through simulations and a real experiment. This work contributes to fast high-resolution image-based wavefront sensing with a high robustness.

An Optimization‐based SPH Solver for Simulation of Hyperelastic Solids

AbstractThis paper proposes a novel method for simulating hyperelastic solids with Smoothed Particle Hydrodynamics (SPH). The proposed method extends the coverage of the state‐of‐the‐art elastic SPH solid method to include different types of hyperelastic materials, such as the Neo‐Hookean and the St. Venant‐Kirchoff models. To this end, we reformulate an implicit integration scheme for SPH elastic solids into an optimization problem and solve the problem using a general‐purpose quasi‐Newton method. Our experiments show that the Limited‐memory BFGS (L‐BFGS) algorithm can be employed to efficiently solve our optimization problem in the SPH framework and demonstrate its stable and efficient simulations for complex materials in the SPH framework. Thanks to the nature of our unified representation for both solids and fluids, the SPH formulation simplifies coupling between different materials and handling collisions.

Elastic and anelastic adjoint tomography with and full Hessian kernels

SUMMARYThe elastic and anelastic structures of the Earth offer fundamental constraints for understanding its physical and chemical properties. Deciphering small variations in the velocity and amplitude of seismic waves can be challenging. Advanced approaches such as full-waveform inversion (FWI) can be useful. We rewrite the anelastic Fréchet kernel expression of Fichtner & van Driel using the displacement–stress formulation. We then derive the full Hessian kernel expression for viscoelastic properties. In these formulations, the anelastic Fréchet kernels are computed by the forward strain and a shift of the adjoint strain. This is complementary to the quality factor Q (i.e., inverse attenuation) Fréchet kernel expressions of Fichtner & van Driel that are explicit for the velocity–stress formulation. To reduce disk space and I/O requirements for computing the full Hessian kernels, the elastic full Hessian kernels are computed on the fly, while the full Hessian kernels for Q are computed by a combination of the on-the-fly approach with the parsimonious storage method. Applications of the Fréchet and full Hessian kernels for adjoint tomography are presented for two synthetic 2-D models, including an idealized model with rectangular anomalies and a model that approximates a subduction zone, and one synthetic 3-D model with an idealized geometry. The calculation of the full Hessian kernel approximately doubles the computationally cost per iteration of the inversion; however, the reduced number of iterations and fewer frequency stages required to achieve the same level of convergence make it overall computationally less expensive than the classical Limited-memory Broyden–Fletcher–Goldfarb–Shanno (L-BFGS) FWI for the 2-D elastic tested models. We find that the use of full Hessian kernels provides comparable results to the L-BFGS inversion using the improved anelastic Fréchet kernels for the 2-D anelastic models tested for the frequency stage up to 0.5 Hz. Given the computational expense of the Q full Hessian kernel calculation, it is not advantageous to use it in Q inversions at this time until further improvements are made. For the 3-D elastic inversion of the tested model, the full Hessian kernel provides similar image quality to the L-BFGS inversion for the frequency stage up to 0.1 Hz. We observe an improved convergence rate for the full Hessian kernel inversion in comparison to L-BFGS at a higher frequency stage, 0.1–0.2 Hz, and we speculate that at higher frequency stages the use of full Hessian kernels may be more computationally advantageous than the classical L-BFGS for the tested models. Finally, we perform 3-D elastic and Q L-BFGS inversions simultaneously using the rederived Q kernels, which can reduce the computational cost of the inversion by about 1/3 when compared to the classical anelastic adjoint tomography using the additionally defined adjoint source. The recovered Q model is smeared when compared to the recovered elastic model at the investigation frequencies up to 0.5 Hz. Q inversion remains challenging and requires further work. The 2-D and 3-D full Hessian kernels may be used for other purposes for instance resolution analysis in addition to the inversions.

Multi-depth phase-only hologram optimization using the L-BFGS algorithm with sequential slicing.

We implement a limited-memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) optimization of phase-only computer-generated hologram for a multi-depth three-dimensional (3D) target. Instead of computing the full 3D reconstruction of the hologram, we use a novel method using L-BFGS with sequential slicing (SS) for partial evaluation of the hologram during optimization that only computes loss for a single slice of the reconstruction at every iteration. We demonstrate that its ability to record curvature information enables L-BFGS to have good quality imbalance suppression under the SS technique.

Machine learning aided near-field acoustic holography based on equivalent source method.

In recent times, equivalent source method-based near-field acoustic holography methods have been extensively applied in sound source localization and characterization. The most commonly used equivalent sources are spherical harmonics. In a non-reverberant environment with no reflections, these equivalent sources could be the best choice since spherical harmonics are derived for the Sommerfeld boundary condition. However, these methods are not the best fit for reverberating environments. In such cases, a new relationship can be calculated between the field weights and the measured pressure with enough training examples. The proposed machine learning models include linear regression (LR) with adaptive moment estimation (Adam), LR with limited-memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS), and multi-layer perceptron with one and two hidden layers. These methods are tested for multiple monopoles and vibrating plate simulations in a room with different wall absorption coefficients. The data-driven methods are also studied on loudspeakers numerically and experimentally in a free field environment. The results from these methods are compared with the results of one norm convex optimization (L1CVX). LR with L-BFGS performed the best among all the methods studied and performed better than L1CVX for less absorption coefficient for geometrically separable sources. LR with L-BFGS also has much faster inference times.

Comparison of Different Weight Optimization Algorithm in Neural Network to Predict Mechanical Properties of AAC Lightweight Brick

This research aims to find the optimal weight optimization algorithm and number of hidden nodes that can be used in Artificial Neural Network to predict mechanical properties (density and compressive strength) of Autoclaved Aerated Concrete (AAC) lightweight brick. The dataset is obtained from secondary source, with a total of 51 data points. From this dataset, the relationship between constituent elements of AAC with its density and compressive strength is modeled using ANN. It was found that the best weight optimization algorithm that can be used for this dataset is the LBFGS (Limited-memory Broyden–Fletcher–Goldfarb–Shanno) algorithm. The optimum hidden layer node is found to be 90 nodes. With this parameters, the ANN can predict density and compressive strength of AAC lightweight brick with accuracy of 93.51% and margin of error around 6.49%. The accuracy of the prediction can be improved by appending the dataset with data points from secondary sources or by doing more experiments and tests.

Data-driven techniques for temperature data prediction: big data analytics approach

For extrapolation, climate change and other meteorological analysis, a study of past and current weather events is a prerequisite. NASA (National Aeronautics and Space Administration) has been able to develop a model capable of predicting various weather data for any location on the Earth, including locations lacking weather stations, weather satellite coverage, and other weather measuring instruments. This paper evaluates the prediction accuracy of the NASA temperature data with respect to NiMet (Nigerian Meteorological Agency) ground truth measurement, using Akwa Ibom Airport as a case study. Exploratory data analysis (descriptive and diagnostic analyses) of temperature retrieved from NiMet and NASA was performed to give a clear path to follow for predictive and prescriptive analyses. Using 2783days of weather data retrieved from NiMet as ground truth, the accuracy of NASA predictions with the corresponding resolution was calculated. Mean absolute error (MAE) of 2.184°C and root mean square error (RMSE) of 2.579°C, with a coefficient of determination (R2) of 0.710 for maximum temperature, then MAE of 0.876°C, RMSE of 1.225°C with a coefficient of determination (R2) of 0.620 for minimum temperature was discovered. There is a good correlation between the two datasets; hence, a model can be developed to generate more accurate predictions, using the NASA data as input. Predictive and prescriptive analyses were performed by employing five prediction algorithms: decision tree regression, XGBoost regression and MLP (multilayer perceptron) with LBFGS (limited-memory Broyden-Fletcher-Goldfarb-Shanno) optimizer, MLP with SGD (stochastic gradient) optimizer and MLP with Adam optimizer. The MLP LBFGS algorithm performed best, by significantly reducing the MAE by 35.35% and RMSE by 31.06% for maximum temperature, accordingly, MAE by 10.05% and RMSE by 8.00% for minimum temperature. Results obtained show that given sufficient data, plugging NASA predictions as input to an LBFGS-MLP model gives more accurate temperature predictions for the study area.

Studies on modified limited-memory BFGS method in full waveform inversion

Full waveform inversion (FWI) is a non-linear optimization problem based on full-wavefield modeling to obtain quantitative information of subsurface structure by minimizing the difference between the observed seismic data and the predicted wavefield. The limited-memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) method is an effective quasi-Newton method in FWI due to its high inversion efficiency with low calculation and storage requirements. Like other conventional quasi-Newton methods, the approximation of the Hessian matrix in the L-BFGS method satisfies the quasi-Newton equation, which only exploits the gradient and model information while the available objective function value is neglected. The modified quasi-Newton equation considers the gradient, model, and objective function information together. Theoretical analysis reveals that the modified quasi-Newton equation is superior to the conventional quasi-Newton equation as it achieves higher-order accuracy in approximating the Hessian matrix. The modified L-BFGS method can be obtained by using the modified quasi-Newton equation to modify the L-BFGS method. This modification improves the accuracy of the Hessian matrix approximation with little increase of calculation for each iteration. We incorporate the modified L-BFGS method into FWI, numerical results show that the modified L-BFGS method has a higher convergence rate, achieves better inversion results, and has stronger anti-noise ability than the conventional L-BFGS method.

Machine Learning-Based GPR with LBFGS Kernel Parameters Selection for Optimal Throughput Mining in 5G Wireless Networks

Considering the ever-growing demand for an efficient method of deductive mining and extrapolative analysis of large-scale dimensional datasets, it is very critical to explore advanced machine learning models and algorithms that can reliably meet the demands of modern cellular networks, satisfying computational efficiency and high precision requirements. One non-parametric supervised machine learning model that finds useful applications in cellular networks is the Gaussian process regression (GPR). The GPR model holds a key controlling kernel function whose hyperparameters can be tuned to enhance its supervised predictive learning and adaptive modeling capabilities. In this paper, the limited-memory Broyden–Fletcher–Goldfarb–Shanno (LBFGS) with kernel parameters selection (KPS) algorithm is employed to tune the GPR model kernel hyperparameters rather than using the standard Bayesian optimization (BOP), which is computationally expensive and does not guarantee substantive precision accuracy in the extrapolative analysis of a large-scale dimensional dataset. In particular, the hybrid GPR–LBFGS is exploited for adaptive optimal extrapolative learning and estimation of throughput data obtained from an operational 5G new radio network. The extrapolative learning accuracy of the proposed GPR–LBFGS with the KPS algorithm was analyzed and compared using standard performance metrics such as the mean absolute error, mean percentage error, root mean square error and correlation coefficient. Generally, results revealed that the GPR model combined with the LBFGS kernel hyperparameter selection is superior to the Bayesian hyperparameter selection method. Specifically, at a 25 m distance, the proposed GPR–LBFGS with the KPS method attained 0.16 MAE accuracy in throughput data prediction. In contrast, the other methods attained 46.06 and 53.68 MAE accuracies. Similarly, at 50 m, 75 m, 100 m, and 160 m measurement distances, the proposed method attained 0.24, 0.18, 0.25, and 0.11 MAE accuracies, respectively, in throughput data prediction, while the two standard methods attained 47.46, 49.93, 29.80, 53.92 and 47.61, 52.54, 53.43, 54.97, respectively. Overall, the GPR–LBFGS with the KPS method would find valuable applications in 5G and beyond 5 G wireless communication systems.

Performance evaluation of soft computing techniques for forecasting daily reference evapotranspiration

Abstract Reference evapotranspiration (ET0) is used to determine crop water requirements under different climatic conditions. In this study, soft computing tools viz. artificial neural network (ANN) and k-nearest neighbors (KNN) models were evaluated for forecasting daily ET0 by comparing their performance with the Penman-Monteith model (PM) using climatic data from 1990 to 2020 of the Indian Agricultural Research Institute (IARI) farm observatory, New Delhi, India. The performance of these models was assessed using statistical performance indices viz., mean absolute error (MAE), mean squared error (MSE), correlation coefficient (r), mean absolute percentage error (MAPE), and index of agreement (d). Results revealed that the ANN model with sigmoid activation function and L-BFGS (Limited memory-Broyden-Fletcher-Goldfarb-Shanno) learning algorithm was selected as the best performing model amongst 36 ANN models. Amongst 4 KNN models developed and tested, the K4 KNN model was observed to be the best in forecasting daily ET0. Overall, the best ANN model (M11) outperformed the K4 KNN model with MAE, MSE, r, MAPE, and d values of 0.075, 0.018, 0.997, 2.76 %, and 0.974, respectively and 0.091, 0.053, 0.984, 3.16 %, and 0.969, respectively during training and testing periods. Thus, we conclude that the ANN technique performed better than the KNN technique in forecasting daily ET0. Sensitivity analysis of the best ANN model revealed that wind speed was the most influential input variable compared to other weather parameters. Thus, the ANN model to forecast daily ET0 accurately for efficient irrigation scheduling of different crops in the study region may be recommended.

An inexact projected gradient method with rounding and lifting by nonlinear programming for solving rank-one semidefinite relaxation of polynomial optimization

We consider solving high-order and tight semidefinite programming (SDP) relaxations of nonconvex polynomial optimization problems (POPs) that often admit degenerate rank-one optimal solutions. Instead of solving the SDP alone, we propose a new algorithmic framework that blends local search using the nonconvex POP into global descent using the convex SDP. In particular, we first design a globally convergent inexact projected gradient method (iPGM) for solving the SDP that serves as the backbone of our framework. We then accelerate iPGM by taking long, but safeguarded, rank-one steps generated by fast nonlinear programming algorithms. We prove that the new framework is still globally convergent for solving the SDP. To solve the iPGM subproblem of projecting a given point onto the feasible set of the SDP, we design a two-phase algorithm with phase one using a symmetric Gauss–Seidel based accelerated proximal gradient method (sGS-APG) to generate a good initial point, and phase two using a modified limited-memory BFGS (L-BFGS) method to obtain an accurate solution. We analyze the convergence for both phases and establish a novel global convergence result for the modified L-BFGS that does not require the objective function to be twice continuously differentiable. We conduct numerical experiments for solving second-order SDP relaxations arising from a diverse set of POPs. Our framework demonstrates state-of-the-art efficiency, scalability, and robustness in solving degenerate SDPs to high accuracy, even in the presence of millions of equality constraints.

Polyphase waveform design for ISRJ suppression based on L-BFGS method

In order to eliminate the false-target peaks caused by the interrupted sampling repeater jamming (ISRJ) and increase the true target detection ability in pulse compression, a polyphase radar waveform consisting of an effective time discontinuous waveform (TDW) and protection pulse (PP) is proposed in this paper. In the waveform design procedure, the ISRJ parameters, including the sampling pulse duration, sampling period and retransmitting delay, are estimated using the sliding-truncation matched filter algorithm. Based on the estimation results, the orthogonal PP and TDW are designed to fill the ISRJ sampling and retransmitting durations according to the jamming operator matrix. The phase code of radar transmitted waveform is calculated iteratively to maximize the orthogonality between the PP and TDW using the limited-memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) algorithm. A mismatched filter for the TDW waveform is optimized to further suppress the false-target peak caused by the ISRJ. Numerical simulations with different jamming parameters are given to demonstrate the efficiency of the proposed method, the designed waveform exhibits a moderate target signal integration loss and a large ISRJ suppression, thereby achieves a high anti-ISRJ performance. The false-target peaks of ISRJ are eliminated and the target signal is retained simultaneously.

Fast-Converging and Low-Complexity Linear Massive MIMO Detection With L-BFGS Method

In massive multiple-input multiple-output (MIMO) uplink, linear minimum mean square error (MMSE) detection achieves near-optimal performance but suffers from undesirable computational burden due to the high-dimensional matrix inversion involved. To address this issue, the limited-memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) quasi-Newton method is introduced in this paper to realize free-matrix-inversion MMSE detection in an iterative way with significant reduction in computational complexity from <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathcal {O}(K^{3})$</tex-math></inline-formula> to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathcal {O}(L K^{2})$</tex-math></inline-formula> , where <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$K$</tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$L$</tex-math></inline-formula> denote the number of user antennas and iterations, respectively. For the better performance and complexity trade-off, three low-complexity L-BFGS based linear MMSE detection algorithms (namely E-LBFGS, I-LBFGS and S-LBFGS) are proposed successively. Furthermore, an efficient initialization strategy for the L-BFGS method based MMSE detection is devised, which helps to accelerate convergence and improve performance with acceptable complexity overhead added compared to the conventional L-BFGS based detection algorithms. Simulation results finally validate that the proposed detection scheme based on the L-BFGS method can closely approach to the MMSE accuracy with a small number of iterations even in poor propagation environments and provide attractive trade-offs between performance and complexity.

Constrained non-linear AVO inversion based on the adjoint-state optimization

Pre-stack AVO inversion of seismic data is a modeling tool for estimating subsurface elastic properties. Our focus is on the model-based inversion method where then unknown variables are estimated by minimizing the misfit to the observed data. Standard approaches for non-linear AVO inversion are based on the gradient descent optimization algorithms that require the calculation of the gradient equations of the objective function. To improve the accuracy and efficiency of these methods, we developed a technique that uses an implementation of the adjoint-state-based gradient computation. The inversion algorithm relies on three basic modeling components consisting of a convolution-based forward model using a linearized approximation of the Zoeppritz equation, the definition of the objective function, and the adjoint-computed gradient. To achieve an accurate solution, we choose a second-order optimization algorithm known as the Limited memory-BFGS (L-BFGS) that implicitly approximates the inverse Hessian matrix. This approach is more efficient than traditional optimization methods. The main novelty of the proposed approach is the derivation of the adjoint-state equations for the gradient of the objective function. The application of the proposed method is demonstrated using 1D and 2D synthetic datasets based on data from the Edvard Grieg oil field. The seismic data for these applications is generated by using both convolutional modeling and finite difference methods. The results of the proposed method are accurate and the computational approach is efficient. The results show that the algorithm reliably retrieves the elastic variables, P- and S-wave velocities and density for both convolutional and finite difference models.

Dual-Level Training of Gaussian Processes with Physically Inspired Priors for Geometry Optimizations.

Gaussian process (GP) regression has been recently developed as an effective method in molecular geometry optimization. The prior mean function is one of the crucial parts of the GP. We design and validate two types of physically inspired prior mean functions: force-field-based priors and posterior-type priors. In this work, we implement a dual-level training (DLT) optimizer for the posterior-type priors. The DLT optimizers can be considered as a class of optimization algorithms that belong to the delta-machine learning paradigm but with several major differences compared to the previously proposed algorithms in the same paradigm. In the first level of the DLT, we incorporate the classical mechanical descriptions of the equilibrium geometries into the prior function, which enhances the performance of the GP optimizer as compared to the one using a constant (or zero) prior. In the second level, we utilize the surrogate potential energy surfaces (PESs), which incorporate the physics learned in the first-level training, as the prior function to refine the model performance further. We find that the force-field-based priors and posterior-type priors reduce the overall optimization steps by a factor of 2-3 when compared to the limited-memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) optimizer as well as the constant-prior GP optimizer proposed in previous works. We also demonstrate the potential of recovering the real PESs with GP with a force-field prior. This work shows the importance of including domain knowledge as an ingredient in the GP, which offers a potentially robust learning model for molecular geometry optimization and for exploring molecular PESs.