Gradient-type Algorithm Research Articles

Riemannian Smoothing Gradient Type Algorithms for Nonsmooth Optimization Problem on Compact Riemannian Submanifold Embedded in Euclidean Space

Riemannian Smoothing Gradient Type Algorithms for Nonsmooth Optimization Problem on Compact Riemannian Submanifold Embedded in Euclidean Space

A numerical approach to the optimal control of thermally convective flows

The optimal control of thermally convective flows is usually modeled by an optimization problem with constraints of Boussinesq equations that consist of the Navier-Stokes equation and an advection-diffusion equation. This optimal control problem is challenging from both theoretical analysis and algorithmic design perspectives. For example, the nonlinearity and coupling of fluid flows and energy transports prevent direct applications of gradient type algorithms in practice. In this paper, we propose an efficient numerical method to solve this problem based on the operator splitting and optimization techniques. In particular, we employ the Marchuk-Yanenko method leveraged by the L2-projection for the time discretization of the Boussinesq equations so that the Boussinesq equations are decomposed into some easier linear equations without any difficulty in deriving the corresponding adjoint system. Consequently, at each iteration, four easy linear advection-diffusion equations and two degenerated Stokes equations at each time step are needed to be solved for computing a gradient. Then, we apply the Bercovier-Pironneau finite element method for space discretization, and design a BFGS type algorithm for solving the fully discretized optimal control problem. We look into the structure of the problem, and design a meticulous strategy to seek step sizes for the BFGS efficiently. Efficiency of the numerical approach is promisingly validated by the results of some preliminary numerical experiments.

Quantum Gate Generation in Two-Level Open Quantum Systems by Coherent and Incoherent Photons Found with Gradient Search

In this work, we consider an environment formed by incoherent photons as a resource for controlling open quantum systems via an incoherent control. We exploit a coherent control in the Hamiltonian and an incoherent control in the dissipator which induces the time-dependent decoherence rates γk(t) (via time-dependent spectral density of incoherent photons) for generation of single-qubit gates for a two-level open quantum system which evolves according to the Gorini–Kossakowski–Sudarshan–Lindblad (GKSL) master equation with time-dependent coefficients determined by these coherent and incoherent controls. The control problem is formulated as minimization of the objective functional, which is the sum of Hilbert-Schmidt norms between four fixed basis states evolved under the GKSL master equation with controls and the same four states evolved under the ideal gate transformation. The exact expression for the gradient of the objective functional with respect to piecewise constant controls is obtained. Subsequent optimization is performed using a gradient type algorithm with an adaptive step size that leads to oscillating behaviour of the gradient norm vs. iterations. Optimal trajectories in the Bloch ball for various initial states are computed. A relation of quantum gate generation with optimization on complex Stiefel manifolds is discussed. We develop methodology and apply it here for unitary gates as a testing example. The next step is to apply the method for generation of non-unitary processes and to multi-level quantum systems.

Optimal control problems with L0(Ω)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$L^0(\\Omega )$$\\end{document} constraints: maximum principle and proximal gradient method

We investigate optimal control problems with L0\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$L^0$$\\end{document} constraints, which restrict the measure of the support of the controls. We prove necessary optimality conditions of Pontryagin maximum principle type. Here, a special control perturbation is used that respects the L0\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$L^0$$\\end{document} constraint. First, the maximum principle is obtained in integral form, which is then turned into a pointwise form. In addition, an optimization algorithm of proximal gradient type is analyzed. Under some assumptions, the sequence of iterates contains strongly converging subsequences, whose limits are feasible and satisfy a subset of the necessary optimality conditions.

MLE Evolution Equation for Fractional Diffusions and Berry-Esseen Inequality of Stochastic Gradient Descent Algorithm for American Option

We study recursive parameter estimation in fractional diffusion processes. First, stability and asymptotic properties of the global maximum likelihood estimator (MLE) of the drift parameter are obtained under some regularity conditions. Then we obtain an evolution equation for the MLE of the drift parameter in nonhomogeneous fractional stochastic differential equation (fSDE) driven by fractional Brownian motion. This equation is then modified to yield an algorithm which is consistent, asymptotically efficient and converges to the MLE. The gradient and Newton type algorithm are firstorder approximations. Finally we study the Berry-Esseen inequality for stochastic gradient descent in continuous time (SGDCT) algorithm for American option. We compare it with Longstaff-Schwartz regression based method.

A two-grid binary level set method for structural topology optimization

A binary level set method of two-grid type is proposed for structural topology optimization. The method allows both shape and topological changes during evolution with no need for reinitialization. The Galerkin finite element method is used to discretize the linear elasticity equation, the eigenvalue problem in structure and the semi-implicit time discrete level set equation. In order to improve the efficiency of the gradient-type algorithm as well as balance the computational efforts of solvers, a nested two-grid discretization strategy is proposed. The linear elasticity or eigenvalue problem is solved on a coarse mesh, while the level set equation and the smoothing step are both discretized on a fine mesh. A variety of numerical results are given to demonstrate both the efficiency and effectiveness of the algorithm.

Optimal H2 Moment Matching-Based Model Reduction for Linear Systems through (Non)convex Optimization

In this paper, we compute a (local) optimal reduced order model that matches a prescribed set of moments of a stable linear time-invariant system of high dimension. We fix the interpolation points and parametrize the models achieving moment-matching in a set of free parameters. Based on the parametrization and using the H2-norm of the approximation error as the objective function, we derive a nonconvex optimization problem, i.e., we search for the optimal free parameters to determine the model yielding the minimal H2-norm of the approximation error. Furthermore, we provide the necessary first-order optimality conditions in terms of the controllability and the observability Gramians of a minimal realization of the error system. We then propose two gradient-type algorithms to compute the (local) optimal models, with mathematical guarantees on the convergence. We also derive convex semidefinite programming relaxations for the nonconvex Problem, under the assumption that the error system admits block-diagonal Gramians, and derive sufficient conditions to guarantee the block diagonalization. The solutions resulting at each step of the proposed algorithms guarantee the achievement of the imposed moment matching conditions. The second gradient-based algorithm exhibits the additional property that, when stopped, yields a stable approximation with a reduced H2-error norm. We illustrate the theory on a CD-player and on a discretized heat equation.

Nonuniqueness of minimizers for semilinear optimal control problems

A counterexample to uniqueness of global minimizers of semilinear optimal control problems is given. The lack of uniqueness occurs for a special choice of the state-target in the cost functional. Our arguments also show that, for some state-targets, there exist local minimizers which are not global. When this occurs, gradient-type algorithms may be trapped by local minimizers, thus missing global ones. Furthermore, the issue of convexity of a quadratic functional in optimal control is analyzed in an abstract setting. As a corollary of nonuniqueness of minimizers, a nonuniqueness result for a coupled elliptic system is deduced. Numerical simulations have been performed illustrating the theoretical results. We also discuss the possible impact of the multiplicity of minimizers on the turnpike property in long time horizons.

Multi-objective models for the forest harvest scheduling problem in a continuous-time framework

In this study we present several multi-objective models for forest harvest scheduling in forest with single-species, even-aged stands using a continuous formulation. We seek to maximize economic profitability and even-flow of timber harvest volume, both for the first rotation and for the regulated forest. For that, we design new metrics that allow working with continuous decision variables, namely, the harvest time of each stand. Unlike traditional combinatorial formulations, this avoids dividing the planning horizon into periods and simulating alternative management prescriptions before the optimization process. We propose to combine a scalarization technique (weighting method) with a gradient-type algorithm (L-BFGS-B) to obtain the Pareto frontier of the problem, which graphically shows the relationships (trade-offs) between objectives, and helps the decision makers to choose a suitable weighting for each objective. We compare this approach with the widely used in forestry multi-objective evolutionary algorithm NSGA-II. We analyze the model in a Eucalyptus globulus Labill. forest of Galicia (NW Spain). The continuous formulation proves robust in forests with different structures and provides better results than the traditional combinatorial approach. For problem solving, our proposal shows a clear advantage over the evolutionary algorithm in terms of computational time (efficiency), being of the order of 65 times faster for both continuous and discrete formulations.

Proximal Gradient-Type Algorithms for a Class of Bilevel Programming Problems

A class of proximal gradient-type algorithm for bilevel nonlinear nondifferentiable programming problems with smooth substructure is developed in this paper. The original problem is approximately reformulated by explicit slow control technique to a parameterized family function which makes full use of the information of smoothness. At each iteration, we only need to calculate one proximal point analytically or with low computational cost. We prove that the accumulation iterations generated by the algorithms are solutions of the original problem. Moreover, some results of complexity of the algorithms are presented in convergence analysis. Numerical experiments are implemented to verify the efficiency of the proximal gradient algorithms for solving this kind of bilevel programming problems.

On stochastic gradient Langevin dynamics with dependent data streams in the logconcave case

We study the problem of sampling from a probability distribution $\pi $ on $\mathbb{R}^{d}$ which has a density w.r.t. the Lebesgue measure known up to a normalization factor $x\mapsto \mathrm{e}^{-U(x)}/\int _{\mathbb{R}^{d}}\mathrm{e}^{-U(y)}\,\mathrm{d}y$. We analyze a sampling method based on the Euler discretization of the Langevin stochastic differential equations under the assumptions that the potential $U$ is continuously differentiable, $\nabla U$ is Lipschitz, and $U$ is strongly concave. We focus on the case where the gradient of the log-density cannot be directly computed but unbiased estimates of the gradient from possibly dependent observations are available. This setting can be seen as a combination of a stochastic approximation (here stochastic gradient) type algorithms with discretized Langevin dynamics. We obtain an upper bound of the Wasserstein-2 distance between the law of the iterates of this algorithm and the target distribution $\pi $ with constants depending explicitly on the Lipschitz and strong convexity constants of the potential and the dimension of the space. Finally, under weaker assumptions on $U$ and its gradient but in the presence of independent observations, we obtain analogous results in Wasserstein-2 distance.

Two adaptive scaled gradient projection methods for Stiefel manifold constrained optimization

This article is concerned with the problem of minimizing a smooth function over the Stiefel manifold. In order to address this problem, we introduce two adaptive scaled gradient projection methods that incorporate scaling matrices that depend on the step-size and a parameter that controls the search direction. These iterative algorithms use a projection operator based on the QR factorization to preserve the feasibility in each iteration. However, for some particular cases, the proposals do not require the use of any projection operator. In addition, we consider a Barzilai and Borwein-like step-size combined with the Zhang–Hager nonmonotone line-search technique in order to accelerate the convergence of the proposed procedures. We proved the global convergence for these schemes, and we evaluate their effectiveness and efficiency through an extensive computational study, comparing our approaches with other state-of-the-art gradient-type algorithms.

Hyperspectral image denoising based on low-rank coefficients and orthonormal dictionary

Hyperspectral images (HSIs) are unavoidably contaminated by noise during data acquisition and transmission. A variety of noise reduction approaches have been developed for HSIs, in which the low-rank based methods have emerged as a powerful tool. However, most of low-rank based HSI denoising methods are designed to explore the low-rankness in the pixel space under a fixed dictionary, without seeking the optimal dictionary and exploiting the low-rankness of representation coefficients. In this work, a novel HSI denoising model based on Low-Rank Coefficients and Orthonormal Dictionary (LRCOD) is proposed. The novelty of this work lies in LRCOD’s exploiting the low-rankness of representation coefficients and learning an orthonormal dictionary for HSI denoising. To solve the proposed model, an efficient alternating minimization algorithm is developed, since one sub-problem has a closed-form solution and the other can be efficiently solved by the curvilinear search algorithm. Compared with gradient-type algorithms, our algorithm is easy to implement as there is no need to tune optimization parameters like step sizes. The experimental results over both simulated and real datasets verified the superiority of the proposed method. Specifically, under various levels of noise in different scenes, LRCOD yields the best MPSNR in 30 of 35 cases and runs faster than other methods except one non-iterative algorithm.

Convergence rates for an inertial algorithm of gradient type associated to a smooth non-convex minimization

We investigate an inertial algorithm of gradient type in connection with the minimization of a non-convex differentiable function. The algorithm is formulated in the spirit of Nesterov’s accelerated convex gradient method. We prove some abstract convergence results which applied to our numerical scheme allow us to show that the generated sequences converge to a critical point of the objective function, provided a regularization of the objective function satisfies the Kurdyka–Łojasiewicz property. Further, we obtain convergence rates for the generated sequences and the objective function values formulated in terms of the Łojasiewicz exponent of a regularization of the objective function. Finally, some numerical experiments are presented in order to compare our numerical scheme and some algorithms well known in the literature.

Additive manufacturing scanning paths optimization using shape optimization tools

This paper investigates path planning strategies for additive manufacturing processes such as powder bed fusion. The state of the art mainly studies trajectories based on existing patterns. Parametric optimization on these patterns and allocating them to the object areas are the main strategies. We propose in this work a more systematic optimization approach without any a priori restriction on the trajectories. The typical optimization problem is to melt the desired structure, without over-heating (to avoid thermally induced residual stresses) and possibly with a minimal path length. The state equation is the heat equation with a source term depending on the scanning path. First, in a steady-state context, shape optimization tools are applied to trajectories. Second, for time-dependent problems, an optimal control method is considered instead. In both cases, gradient-type algorithms are deduced and tested on 2-D examples. Numerical results are discussed, leading to a better understanding of the problem and thus to short- and long-term perspectives.

A Provably Convergent Projected Gradient-Type Algorithm for TDOA-Based Geolocation Under the Quasi-Parabolic Ionosphere Model

The problem of geolocating an unknown high-frequency emitter based on the quasi-parabolic ionosphere model with time-difference of arrival measurements of the refracted radio rays is of fundamental importance in various military and civilian applications. Such a problem admits a maximum-likelihood (ML) formulation, which is nonlinear and non-convex. By elucidating the geometry of the feasible set of the ML formulation, we develop a first-order algorithm, which we call Generalized Projected Gradient Descent , to solve it. We prove that every limit point of the iterates generated by our proposed algorithm is a critical point of the ML formulation. Simulation results show that our proposed algorithm can more reliably and accurately geolocate the emitter than a state-of-the-art method in various settings.

Parameter-dependent transmission dynamics and optimal control of foot and mouth disease in a contaminated environment

The problem of foot and mouth disease (FMD) is of serious concern to the livestock sector in most nations, especially in developing countries. This paper presents the formulation and analysis of a deterministic model for the transmission dynamics of FMD through a contaminated environment. It is shown that the key parameters that drive the transmission of FMD in a contaminated environment are the shedding, transmission, and decay rates of the virus. Using numerical results, it is depicted that the host-to-host route is more severe than the environmental-to-host route. The model is then transformed into an optimal control problem. Using the Pontryagin’s Maximum Principle, the optimality system is determined. Utilizing a gradient type algorithm with projection, the optimality system is solved for three control strategies: optimal use of vaccination, environmental decontamination, and a combination of vaccination and environmental decontamination. Results show that a combination of vaccination and environmental decontamination is the most optimal strategy. These results indicate that if vaccination and environmental decontamination are used optimally during an outbreak, then FMD transmission can be controlled. Future studies focusing on the control measures for the transmission of FMD in a contaminated environment should aim at reducing the transmission and the shedding rates, while increasing the decay rate.

A gradient-type algorithm for constrained optimization with application to microstructure optimization

We propose a method to optimize periodic microstructures for obtaining homogenized materials with negative Poisson ratio, using shape and/or topology variations in the model hole. The proposed approach employs worst case design in order to minimize the Poisson ratio of the (possibly anisotropic) homogenized elastic tensor in several prescribed directions. We use a minimization algorithm for inequality constraints based on an active set strategy and on a new algorithm for solving minimization problems with equality constraints, belonging to the class of null-space gradient methods. It uses first order derivatives of both the objective function and the constraints. The step is computed as a sum between a steepest descent step (minimizing the objective functional) and a correction step related to the Newton method (aiming to solve the equality constraints). The linear combination between these two steps involves coefficients similar to Lagrange multipliers which are computed in a natural way based on the Newton method. The algorithm uses no projection and thus the iterates are not feasible; the constraints are only satisfied in the limit (after convergence). A local convergence result is proven for a general nonlinear setting, where both the objective functional and the constraints are not necessarily convex functions.

Early Stopping for Kernel Boosting Algorithms: A General Analysis With Localized Complexities

Early stopping of iterative algorithms is a widely-used form of regularization in statistics, commonly used in conjunction with boosting and related gradient-type algorithms. Although consistency results have been established in some settings, such estimators are less well-understood than their analogues based on penalized regularization. In this paper, for a relatively broad class of loss functions and boosting algorithms (including L2-boost, LogitBoost and AdaBoost, among others), we exhibit a direct connection between the performance of a stopped iterate and the localized Gaussian complexity of the associated function class. This connection allows us to show that local fixed point analysis of Gaussian or Rademacher complexities, now standard in the analysis of penalized estimators, can be used to derive optimal stopping rules. We derive such stopping rules in detail for various kernel classes, and illustrate the correspondence of our theory with practice for Sobolev kernel classes.

Single-pixel camera photoacoustic tomography.

.Since it was first demonstrated more than a decade ago, the single-pixel camera concept has been used in numerous applications in which it is necessary or advantageous to reduce the channel count, cost, or data volume. Here, three-dimensional (3-D), compressed-sensing photoacoustic tomography (PAT) is demonstrated experimentally using a single-pixel camera. A large area collimated laser beam is reflected from a planar Fabry–Pérot ultrasound sensor onto a digital micromirror device, which patterns the light using a scrambled Hadamard basis before it is collected into a single photodetector. In this way, inner products of the Hadamard patterns and the distribution of thickness changes of the FP sensor—induced by the photoacoustic waves—are recorded. The initial distribution of acoustic pressure giving rise to those photoacoustic waves is recovered directly from the measured signals using an accelerated proximal gradient-type algorithm to solve a model-based minimization with total variation regularization. Using this approach, it is shown that 3-D PAT of imaging phantoms can be obtained with compression rates as low as 10%. Compressed sensing approaches to photoacoustic imaging, such as this, have the potential to reduce the data acquisition time as well as the volume of data it is necessary to acquire, both of which are becoming increasingly important in the drive for faster imaging systems giving higher resolution images with larger fields of view.