Large-scale Nonlinear Problems Research Articles

An extended Gauss-Newton method for full-waveform inversion

Full-waveform inversion (FWI) is a large-scale nonlinear ill-posed problem for which computationally expensive Newton-type methods can become trapped in undesirable local minima, particularly when the initial model lacks a low-wavenumber component and the recorded data lack low-frequency content. A modification to the Gauss-Newton (GN) method is developed to address these issues. The standard GN system for multisource multireceiver FWI is reformulated into an equivalent matrix equation form, with the solution becoming a diagonal matrix rather than a vector as in the standard system. The search direction is transformed from a vector to a matrix by relaxing the diagonality constraint, effectively adding a degree of freedom to the subsurface offset axis. The relaxed system can be explicitly solved with only the inversion of two small matrices that deblur the data residual matrix along the source and receiver dimensions, which simplifies the inversion of the Hessian matrix. When used to solve the extended-source FWI objective function, the extended GN (EGN) method integrates the benefits of model and source extension. The EGN method effectively combines the computational effectiveness of the reduced FWI method with the robustness characteristics of extended formulations and offers a promising solution for addressing the challenges of FWI. It bridges the gap between these extended formulations and the reduced FWI method, enhancing inversion robustness while maintaining computational efficiency. The robustness and stability of the EGN algorithm for waveform inversion are demonstrated numerically.

Zastosowanie programowania nieliniowego dla konfiguracji systemu wymiany ciepła w aparatach kolumnowych przemysłu petrochemicznego przy minimalnych kosztach

For increasing the efficiency of heat transfer processes, the layer-by-layer concept of arrangement of various types of packings in column apparatuses is considered. The results of hydraulic tests of a dry and irrigated cell packing, as well as individual hydraulic tests of a mesh packing, are presented. Methods and algorithms for solving linear programming problems under “uncertainty” conditions were considered. However, in some areas of science it is often difficult or impossible to formalize the problem in an appropriate way and reduce it to a linear programming problem. In this paper, methods for solving non-linear programming problems with a vector objective function are considered. At present, the use of non-linear programming in the vast majority of real situations is reduced to linear approximation models. Along with this, at a significant non-linearity, due to its specificity or influence on the nature of the model, it is necessary to apply optimization methods that are much more complex than, for example, the simplex method. However, the importance of non-linear programming is constantly increasing. This is due to the rapidly growing knowledge of managers and specialists in the use of mathematical models designed to prepare solutions, as well as the increasing availability of computer programs for solving large-scale nonlinear problems. The analysis and studies of the hydrodynamics of a number of regular packings have shown that cell and mesh packings are promising for the implementation of the phase inversion mode. The proposed concept of intensifying heat transfer processes in column apparatuses is based on the use of layers of various packings arranged in the following order in the apparatus, type 1 – cell, type 2 – mesh. At the same time, their main geometric characteristics differ significantly from each other. However, the structure of the bulk packing is inhomogeneous, which makes it difficult to implement a stable mode of local phase inversion under these conditions. This is due to the structure of column apparatuses with bulk packing, in which there is an increased proportion of pores (porosity) near the walls of the apparatus. Porosity is very significant; it can reach up to 40%, and as a result, the local velocity near the walls exceeds the velocity in the centre of the apparatus by up to 70%. By contrast, the use of regular packing structures makes this potentially highly efficient mode technically possible.

Designing an efficient smoothed point interpolation method for modeling slope deformation

PurposeIn this paper, the smoothed point interpolation method (SPIM) is used to model the slope deformation. However, the computational efficiency of SPIM is not satisfying when modeling the large-scale nonlinear deformation problems of geological bodies.Design/methodology/approachIn this paper, the SPIM is used to model the slope deformation. However, the computational efficiency of SPIM is not satisfying when modeling the large-scale nonlinear deformation problems of geological bodies.FindingsA simple slope model with different mesh sizes is used to verify the performance of the efficient face-based SPIM. The first accelerating strategy greatly enhances the computational efficiency of solving the large-scale slope deformation. The second accelerating strategy effectively improves the convergence of nonlinear behavior that occurred in the slope deformation.Originality/valueThe designed efficient face-based SPIM can enhance the computational efficiency when analyzing large-scale nonlinear slope deformation problems, which can help to predict and prevent potential geological hazards.

A modified-upwind with block-centred finite difference scheme based on the two-grid algorithm for convection-diffusion-reaction equations

ABSTRACT A modified-upwind with block-centred finite difference scheme on the basis of the two-grid algorithm is presented for the convection-diffusion-reaction equations. This scheme can keep second-order accuracy in spatial mesh sizes for both state variables and fluxes in the convection–diffusion–reaction problem. Moreover, the two-grid algorithm is constructed in order to solve semilinear convection-dominated problems efficiently, in which the main idea is to settle an original semilinear equation on the coarse space, and next to settle a linearized equation on the fine space. The error estimate of the method proposed in this paper is given through theoretical analysis. It is indicated that the two-grid algorithm can achieve asymptotically optimal approximation as long as the mesh sizes satisfy . Thus, solving such a large-scale nonlinear problem is as easy as linearized problems. Besides, there are some numerical experiments to corroborate in practice that the algorithm is effective and robust to solve convection–diffusion–reaction problems.

Nonlinear greedy relaxed randomized Kaczmarz method

In this paper, we discuss a nonlinear greedy relaxed randomized Kaczmarz (rNGRK) method for solving large-scale nonlinear problems. This method only needs to calculate one row of the Jacobian matrix in each iteration, which greatly reduces the amount of calculation and storage. Furthermore, the convergence of the rNGRK method is proved and the effectiveness of the rNGRK method in the case of noisy-free data is shown in the corresponding numerical experiments.

Recurrent Model Predictive Control: Learning an Explicit Recurrent Controller for Nonlinear Systems

This paper proposes an offline control algorithm, called Recurrent Model Predictive Control (RMPC), to solve large-scale nonlinear finite-horizon optimal control problems. It can be regarded as an explicit solver of traditional Model Predictive Control (MPC) algorithms, which can adaptively select appropriate model prediction horizon according to current computing resources, so as to improve the policy performance. Our algorithm employs a recurrent function to approximate the optimal policy, which maps the system states and reference values directly to the control inputs. The output of the learned policy network after N recurrent cycles corresponds to the nearly optimal solution of N-step MPC. A policy optimization objective is designed by decomposing the MPC cost function according to the Bellman's principle of optimality. The optimal recurrent policy can be obtained by directly minimizing the designed objective function, which is applicable for general nonlinear and non input-affine systems. Both simulation-based and real-robot path-tracking tasks are utilized to demonstrate the effectiveness of the proposed method.

Acceleration Strategies for MR-STAT: Achieving High-Resolution Reconstructions on a Desktop PC Within 3 Minutes.

MR-STAT is an emerging quantitative magnetic resonance imaging technique which aims at obtaining multi-parametric tissue parameter maps from single short scans. It describes the relationship between the spatial-domain tissue parameters and the time-domain measured signal by using a comprehensive, volumetric forward model. The MR-STAT reconstruction solves a large-scale nonlinear problem, thus is very computationally challenging. In previous work, MR-STAT reconstruction using Cartesian readout data was accelerated by approximating the Hessian matrix with sparse, banded blocks, and can be done on high performance CPU clusters with tens of minutes. In the current work, we propose an accelerated Cartesian MR-STAT algorithm incorporating two different strategies: firstly, a neural network is trained as a fast surrogate to learn the magnetization signal not only in the full time-domain but also in the compressed low-rank domain; secondly, based on the surrogate model, the Cartesian MR-STAT problem is re-formulated and split into smaller sub-problems by the alternating direction method of multipliers. The proposed method substantially reduces the computational requirements for runtime and memory. Simulated and in-vivo balanced MR-STAT experiments show similar reconstruction results using the proposed algorithm compared to the previous sparse Hessian method, and the reconstruction times are at least 40 times shorter. Incorporating sensitivity encoding and regularization terms is straightforward, and allows for better image quality with a negligible increase in reconstruction time. The proposed algorithm could reconstruct both balanced and gradient-spoiled in-vivo data within 3 minutes on a desktop PC, and could thereby facilitate the translation of MR-STAT in clinical settings.

Bayesian uncertainty quantification of local volatility model

Local volatility is an important quantity in option pricing, portfolio hedging, and risk management. It is not directly observable from the market; hence calibrations of local volatility models are necessary using observed market data. Unlike most existing point-estimate methods, we cast the large-scale nonlinear inverse problem into the Bayesian framework, yielding a posterior distribution of the local volatility, which naturally quantifies its uncertainty. This extra uncertainty information enables traders and risk managers to make better decisions. To alleviate the computational cost, we apply Karhunen–Lòeve expansion to reduce the dimensionality of the Gaussian Process prior for local volatility. A modified two-stage adaptive Metropolis algorithm is applied to sample the posterior probability distribution, which further reduces computational burdens caused by repetitive numerical forward option pricing model solver and time of heuristic tuning. We demonstrate our methodology with both synthetic and market data.

Online Adaptive Kernel Learning with Random Features for Large-scale Nonlinear Classification

In the field of support vector machines, online random feature map algorithms are very important methods for large-scale nonlinear classification problems. At present, the existing methods have the following shortcomings: (1) If only the hyperplane vector is updated during learning while the random feature components are fixed, there is no guarantee that these online methods can adapt to the change of data distribution shape when the data is coming one by one. (2) When the kernel is selected improperly, the samples mapped to an inappropriate space may not be well classified. In order to overcome these shortcomings, considering the fact that iteratively updating random feature components can make data better fit in the current space and lead to the flexible adjustment of the kernel function, random features based online adaptive kernel learning (RF-OAK) is proposed for large-scale nonlinear classification problems. Theoretical analysis of the proposed algorithm is also provided. The experimental results and the Wilcoxon signed-ranks test show that in terms of test accuracy, the proposed method is significantly better than the state-of-the-art online feature mapping classification methods. Compared with the deep learning algorithms, the training time of RF-OAK is shorter. In terms of test accuracy, RF-OAK is better than online algorithm and comparable with offline algorithms.

Long-term spatial and population-structured planning of non-pharmaceutical interventions to epidemic outbreaks

In this paper, we consider the problem of planning non-pharmaceutical interventions to control the spread of infectious diseases. We propose a new model derived from classical compartmental models; however, we model spatial and population-structure heterogeneity of population mixing. The resulting model is a large-scale non-linear and non-convex optimisation problem. In order to solve it, we apply a special variant of covariance matrix adaptation evolution strategy. We show that results obtained for three different objectives are better than natural heuristics and, moreover, that the introduction of an individual’s mobility to the model is significant for the quality of the decisions. We apply our approach to a six-compartmental model with detailed Poland and COVID-19 disease data. The obtained results are non-trivialand sometimes unexpected; therefore, we believe that our model could be applied to support policy-makers in fighting diseases at the long-term decision-making level.

Scalable subspace methods for derivative-free nonlinear least-squares optimization

We introduce a general framework for large-scale model-based derivative-free optimization based on iterative minimization within random subspaces. We present a probabilistic worst-case complexity analysis for our method, where in particular we prove high-probability bounds on the number of iterations before a given optimality is achieved. This framework is specialized to nonlinear least-squares problems, with a model-based framework based on the Gauss–Newton method. This method achieves scalability by constructing local linear interpolation models to approximate the Jacobian, and computes new steps at each iteration in a subspace with user-determined dimension. We then describe a practical implementation of this framework, which we call DFBGN. We outline efficient techniques for selecting the interpolation points and search subspace, yielding an implementation that has a low per-iteration linear algebra cost (linear in the problem dimension) while also achieving fast objective decrease as measured by evaluations. Extensive numerical results demonstrate that DFBGN has improved scalability, yielding strong performance on large-scale nonlinear least-squares problems.

Flexible operation of modular electrochemical CO2 reduction processes

Electrochemical CO2 reduction (eCO2R) is an emerging technology that is capable of producing various organic chemicals from CO2, but its high electricity cost is a big economic obstacle. One solution to reduce the cumulative electricity cost is demand side management, i.e., to adjust the power load based on time-variant electricity prices. However, varying the power load of CO2-electrolyzers often leads to changes in Faraday efficiency towards target components and thereby influences the product composition. Such deviations from the target product composition may be undesired for downstream processes. We tackle this challenge by proposing a flexible operating scheme for a modular eCO2R process. We formulate the economically optimal operation of an eCO2R process with multiple electrolyzer stacks as a parallel-machine scheduling problem. Adjusting the power load of each sub-process properly, we can save electricity costs while the desired product composition is met at any time. We apply an algorithm based on wavelet transform to solve the resulting large-scale nonlinear scheduling problem in tractable time. We solve each optimization problem with a deterministic global optimization software MAiNGO. We examine flexible operation of a modular eCO2R process for syngas production. The case studies show that the modular structure enables savings in the cumulative electricity cost of the eCO2R process via flexible operation while deviations in the syngas composition could be reduced. Also, the maximum ramping speed of the entire process is found to be a key parameter that strongly influences the cost saving.

Training very large scale nonlinear SVMs using Alternating Direction Method of Multipliers coupled with the Hierarchically Semi-Separable kernel approximations

• Large scale nonlinear Support Vector Machines can be trained using Alternating Direction Method of Multipliers and Hierarchically-Semi- Separable kernel approximations. • Reduced computational times can be obtained without worsening the classification accuracy. • Grid search for the misclassification parameter selection is extremely efficient in this framework. • Our efficient C++ implementation shows clear advantages over LIBSVM. Typically, nonlinear Support Vector Machines (SVMs) produce significantly higher classification quality when compared to linear ones but, at the same time, their computational complexity is prohibitive for large-scale datasets: this drawback is essentially related to the necessity to store and manipulate large, dense and unstructured kernel matrices. Despite the fact that at the core of training an SVM there is a simple convex optimization problem, the presence of kernel matrices is responsible for dramatic performance reduction, making SVMs unworkably slow for large problems. Aiming [at] an efficient solution of large-scale nonlinear SVM problems, we propose the use of the Alternating Direction Method of Multipliers coupled with Hierarchically Semi-Separable (HSS) kernel approximations. As shown in this work, the detailed analysis of the interaction among their algorithmic components unveils a particularly efficient framework and indeed, the presented experimental results demonstrate[, in the case of Radial Basis Kernels,] a significant speed-up when compared to the state-of-the-art nonlinear SVM libraries (without significantly affecting the classification accuracy).

Gradient-augmented Supervised Learning of Optimal Feedback Laws Using State-Dependent Riccati Equations

A supervised learning approach for the solution of large-scale nonlinear stabilization problems is presented. A stabilizing feedback law is trained from a dataset generated from State-dependent Riccati Equation solvers. The training phase is enriched by the use of gradient information in the loss function, which is weighted through the use of hyperparameters. High-dimensional nonlinear stabilization tests demonstrate that real-time sequential large-scale Algebraic Riccati Equation solvers can be substituted by a suitably trained feedforward neural network.

An effective new iterative CG-method to solve unconstrained non-linear optimization issues

In this paper, we proposed a matrix-free double-search direction based on the updated parameter file of the double-search direction with a new mathematical formula for the gamma parameter. When comparing the numerical results of this algorithm with the standard (HWY) algorithm which given by Halilu, Waziri and Yusuf in 2020. We get very robust numerical results. The proposed algorithm is devoid of derivatives to solve large-scale non-linear problems by combining two search directions in one search direction. We demonstrated the overall convergence of the proposed algorithm under certain conditions. The numerical results presented in this paper show that the new search direction is useful for solving widespread non-linear test problems.

Truncated trust region method for nonlinear inverse problems and application in full-waveform inversion

We present a general truncated trust region method to solve large-scale nonlinear inverse problems. The truncated trust region method can serve as an implicit regularization method, and it can take advantage of the second-order derivative information of the misfit functional. The convergence of the truncated trust region method is provided under some smoothness assumptions. To improve the computational efficiency and reduce the memory requirement, we develop a second-order adjoint-state method to efficiently estimate matrix–vector products, and solve the trust region subproblem using the truncated conjugate gradient method with the matrix-free strategy. The full-waveform inversion problem is used to test the numerical performance of the proposed method. Numerical results show that the truncated trust region method can perform better than conventional methods (e.g. nonlinear conjugate gradient method and L-BFGS) for highly nonlinear inverse problems, in terms of inverting resolution.

Thallo – Scheduling for High-Performance Large-Scale Non-Linear Least-Squares Solvers

Large-scale optimization problems at the core of many graphics, vision, and imaging applications are often implemented by hand in tedious and error-prone processes in order to achieve high performance (in particular on GPUs), despite recent developments in libraries and DSLs. At the same time, these hand-crafted solver implementations reveal that the key for high performance is a problem-specific schedule that enables efficient usage of the underlying hardware. In this work, we incorporate this insight into Thallo, a domain-specific language for large-scale non-linear least squares optimization problems. We observe various code reorganizations performed by implementers of high-performance solvers in the literature, and then define a set of basic operations that span these scheduling choices, thereby defining a large scheduling space. Users can either specify code transformations in a scheduling language or use an autoscheduler. Thallo takes as input a compact, shader-like representation of an energy function and a (potentially auto-generated) schedule, translating the combination into high-performance GPU solvers. Since Thallo can generate solvers from a large scheduling space, it can handle a large set of large-scale non-linear and non-smooth problems with various degrees of non-locality and compute-to-memory ratios, including diverse applications such as bundle adjustment, face blendshape fitting, and spatially-varying Poisson deconvolution, as seen in Figure 1. Abstracting schedules from the optimization, we outperform state-of-the-art GPU-based optimization DSLs by an average of 16× across all applications introduced in this work, and even some published hand-written GPU solvers by 30%+.

Unified SVM algorithm based on LS-DC loss

Over the past two decades, support vector machine (SVM) has become a popular supervised machine learning model, and plenty of distinct algorithms are designed separately based on different KKT conditions of the SVM model for classification/regression with different losses, including the convex loss or nonconvex loss. In this paper, we propose an algorithm that can train different SVM models in a \emph{unified} scheme. First, we introduce a definition of the \emph{LS-DC} (\textbf{l}east \textbf{s}quares type of \textbf{d}ifference of \textbf{c}onvex) loss and show that the most commonly used losses in the SVM community are LS-DC loss or can be approximated by LS-DC loss. Based on DCA (difference of convex algorithm), we then propose a unified algorithm, called \emph{UniSVM}, which can solve the SVM model with any convex or nonconvex LS-DC loss, in which only a vector is computed, especially by the specifically chosen loss. Particularly, for training robust SVM models with nonconvex losses, UniSVM has a dominant advantage over all existing algorithms because it has a closed-form solution per iteration, while the existing algorithms always need to solve an L1SVM/L2SVM per iteration. Furthermore, by the low-rank approximation of the kernel matrix, UniSVM can solve the large-scale nonlinear problems with efficiency. To verify the efficacy and feasibility of the proposed algorithm, we perform many experiments on some small artificial problems and some large benchmark tasks with/without outliers for classification and regression for comparison with state-of-the-art algorithms. The experimental results demonstrate that UniSVM can achieve comparable performance in less training time. The foremost advantage of UniSVM is that its core code in Matlab is less than 10 lines; hence, it can be easily grasped by users or researchers.

An adaptive search direction algorithm for the modified projection minimization optimization

In this paper, we proposed a double-search direction algorithm with a free of derivatives based on a different parameter update (n k ) which gives us an update on the double search trend. We used anewly updated formula for the projection parameter within the formula for the proposed search direction. When comparing our numerical results for the proposed algorithm with some standard published algorithmswe obtain efficient numerical results. The proposed algorithm, especially, isused to solve large-scale nonlinear problems by combining two search directions in one search direction. Also, to demonstrate the general convergence of the proposed new algorithm under some circumstances. The numerical performance of the new proposed algorithm on some nonlinear test functions proved the efficiency of this algorithm.

Two-phase differential evolution for solving emergency response supplies optimisation problem

A material supply model is constructed for serious disasters in which a large number of supply centres and disaster areas are involved. We introduce a new method referred to as two-phase differential evolution (TPDE) to solve this kind of complex nonlinear programming problem. In constraint handling phase, the goal is to explore the parameter space to identify a feasible solution quickly. In optimum seeking phase, the aim is to gradually improve the quality of current best solution. Different differential evolution schemes and special handling techniques are utilised in the two phases. Extensive numerical optimisation experiments are conducted where TPDE is compared with results obtained from using commercial software and three evolutionary optimisation methods. We determine that TPDE is always able to find a feasible solution with fewer generations and the optimal solution almost always ranks as the best. This work is beneficial to address large-scale nonlinear optimisation problems with constraints. [Received: 28 August 2019; Revised: 1 February 2020; Accepted: 8 March 2020]