Gradient Optimization Method Research Articles

Совместное применение методов случайного поиска с градиентными методами оптимизации проектных параметров и программ управления ракетным объектом

Совместное применение методов случайного поиска с градиентными методами оптимизации проектных параметров и программ управления ракетным объектом

Exergoeconomic optimization of a combined cycle power plant’s bottoming cycle using organic working fluids

One of the measures to reduce carbon dioxide emissions and increase energy efficiency in a combined cycle power plant is the improving of its thermodynamic efficiency by optimizing the heat utilization s study, a mathematical model of the bottoming cycle of a combined cycle power plant was developed in Matlab. The heat recovery steam generator, which is a crucial element of the bottoming cycle, is modeled as a heat exchanger network. It consists of multiple pressure levels and a reheater that uses organic working fluids in the lower pressure levels. The mathematical model provides the possibility that the heat exchangers in each pressure level could be in parallel and serial arrangements. An exergoeconomic optimization was conducted, where the optimization variables comprised the heat exchanger layout and the operating parameters of the working fluid in each pressure level. The objective of the optimization was to minimize the sum of the cost of exergy destruction in the bottoming cycle and investment costs. The genetic algorithm and gradient optimization methods were used as optimization tools. The results show that lower cost of exergy destruction can be achieved by optimizing the heat exchanger layout and using organic fluids in the lower pressure levels of a heat recovery steam generator. This research work addresses a gap in the literature by taking into account the heat exchanger layout, optimization parameters, and organic fluids while optimizing a bottoming cycle, which is of essential importance.

Research on DC Boost Type of Photovoltaic Power Generation System

For DC boosting photovoltaic power generation system, this paper mainly focuses on the theory studies, system construction and algorithm design from three aspects, such as photovoltaic array model and output characteristics, structure and principle of boost system, principle of maximum power point tracking and control method. According to the single peak of the P-U curve and the principle of maximum power point tracking, the maximum power point tracking scheme using the impedance transform function of the boost circuit and the adaptive algorithm is determined. Through the study and analysis of the control strategy of the conventional adaptive algorithm introduced the optimal gradient method, an improved adaptive algorithm that fuses the fixed voltage method, the fitting curve method and the optimal gradient method is proposed. The simulation system is established to verify the simulation. The corresponding experimental results prove the validity of the designed photovoltaic power generation system and the improved adaptive algorithm, which improve the speed and accuracy of maximum power point tracking, and solve the wrong judgment of maximum power point

Stochastic optimization of reinforced concrete elements’ design

Main goal of the paper is to present an algorithm for stochastic optimization of design of steel-reinforced concrete element’s cross section. Firstly, the deterministic problem is introduced and described, followed by the description of uncertainties involved in the process and stochastic reformulation of the problem. Afterwards, the algorithm itself is introduced. This algorithm is based on internal cycle of deterministic optimization using reduced gradient method and external cycle of stochastic optimization using regression analysis. The probability is assessed on orthogonal grid via modified bisection method. The paper concludes with presentation of the performed calculations and their results.

Spectral projected gradient method for stochastic optimization

We consider the Spectral Projected Gradient method for solving constrained optimization problems with the objective function in the form of mathematical expectation. It is assumed that the feasible set is convex, closed and easy to project on. The objective function is approximated by a sequence of different Sample Average Approximation functions with different sample sizes. The sample size update is based on two error estimates—SAA error and approximate solution error. The Spectral Projected Gradient method combined with a nonmonotone line search is used. The almost sure convergence results are achieved without imposing explicit sample growth condition. Preliminary numerical results show the efficiency of the proposed method.

On the Convergence Rate of Distributed Gradient Methods for Finite-Sum Optimization under Communication Delays

Motivated by applications in machine learning and statistics, we study distributed optimization problems over a network of processors, where the goal is to optimize a global objective composed of a sum of local functions. In these problems, due to the large scale of the data sets, the data and computation must be distributed over multiple processors resulting in the need for distributed algorithms. In this paper, we consider a popular distributed gradient-based consensus algorithm, which only requires local computation and communication. An important problem in this area is to analyze the convergence rate of such algorithms in the presence of communication delays that are inevitable in distributed systems. We prove the convergence of the gradient-based consensus algorithm in the presence of uniform, but possibly arbitrarily large, communication delays between the processors. Moreover, we obtain an upper bound on the rate of convergence of the algorithm as a function of the network size, topology, and the inter-processor communication delays.

A Global Convergence of Spectral Conjugate Gradient Method for Large Scale Optimization

A Global Convergence of Spectral Conjugate Gradient Method for Large Scale Optimization

A hybrid method of three-term conjugate gradient method and memoryless quasi-Newton method for unconstrained optimization

Memoryless quasi-Newton methods are studied for solving large-scale unconstrained optimization problems. Nakayama et al. (2017) proposed a memoryless quasi-Newton method based on the spectral-scaling Broyden family and showed that the method satisfies the sufficient descent condition and converges globally. To relax the conditions on parameters in the method, we apply the modification technique by Kou and Dai (2015) to the method of Nakayama et al., and we give a hybrid method of the three-term conjugate gradient method and the memoryless quasi-Newton method based on the spectral-scaling Broyden family. We show that our method satisfies the sufficient descent condition, and we prove that the method converges globally. Furthermore, we give a concrete choice of parameters for our method. Finally, some numerical results are given.

A Method for Diagnosis of Poloidal Field Coils’ Current Based on Inversion of Magnetic Field

Tokamak devices such as ITER, EAST, and HL-2M have been designed and constructed (or under construction) to challenge the magnetic-confined nuclear fusion technology for the sustainable energy problem of human beings. The magnet system of the tokamak devices is mainly composed with poloidal field (PF) coils, toroidal field (TF) coils, and central solenoid coils. Among them, the PF coils play a key role in adjusting the confining magnetic field for the control of the fusion plasma. In order to avoid the plasma disruption, it is necessary to know the correlation of the magnetic field in a vacuum vessel (VV) with the current in PF coils and to predict the current distribution of PF coils which can generate a required configuration of confining magnetic field. In this paper, taking the HL-2M tokamak device as an example, an inverse analysis scheme to predict and identify the perturbation of current in PF coils from the target magnetic field signals in the VV was proposed. First, based on the Biot–Savart law, a forward code to calculate the magnetic field due to PF coil currents and TF coil currents was developed. Second, an inversion code to predict and identify current distribution in PF coils from the target magnetic field was developed based on the conjugate gradient optimization method and the singular value decomposition method. The validity of the proposed inversion scheme and the corresponding numerical codes was investigated by reconstructing PF coil current distributions from several groups of perturbed magnetic fields. In addition, the effect of the measurement points and the measurement noises on the reconstruction results was also discussed. Finally, a simplified experiment on a coil system was conducted to verify the validity and efficiency of the proposed forward and inversion analysis code.

Adaptive Restart of the Optimized Gradient Method for Convex Optimization.

First-order methods with momentum such as Nesterov's fast gradient method are very useful for convex optimization problems, but can exhibit undesirable oscillations yielding slow convergence for some applications. An adaptive restarting scheme can improve the convergence rate of the fast gradient methd, when the parameter of a strongly convex cost function is unknown or when the iterates of the algorithm enter a locally well-conditioned region. Recently, we introduced an optimized gradient method, a first-order algorithm that has an inexpensive per-iteration computational cost similar to that of the fast gradient method, yet has a worst-case cost function convergence bound that is twice smaller than that of the fast gradient method and that is optimal for large-dimensional smooth convex problems. Building upon the success of accelerating the fast gradient method using adaptive restart, this paper investigates similar heuristic acceleration of the optimized gradient method. We first derive new step coefficients of the optimized gradient method for a strongly convex quadratic problem with known function parameters, yielding a convergence rate that is faster than that of the analogous version of the fast gradient method. We then provide a heuristic analysis and numerical experiments that illustrate that adaptive restart can accelerate the convergence of the optimized gradient method. Numerical results also illustrate that adaptive restart is helpful for a proximal version of the optimized gradient method for nonsmooth composite convex functions.

Inexact Proximal Gradient Methods for Non-Convex and Non-Smooth Optimization

In machine learning research, the proximal gradient methods are popular for solving various optimization problems with non-smooth regularization. Inexact proximal gradient methods are extremely important when exactly solving the proximal operator is time-consuming, or the proximal operator does not have an analytic solution. However, existing inexact proximal gradient methods only consider convex problems. The knowledge of inexact proximal gradient methods in the non-convex setting is very limited. To address this challenge, in this paper, we first propose three inexact proximal gradient algorithms, including the basic version and Nesterov’s accelerated version. After that, we provide the theoretical analysis to the basic and Nesterov’s accelerated versions. The theoretical results show that our inexact proximal gradient algorithms can have the same convergence rates as the ones of exact proximal gradient algorithms in the non-convex setting. Finally, we show the applications of our inexact proximal gradient algorithms on three representative non-convex learning problems. Empirical results confirm the superiority of our new inexact proximal gradient algorithms.

A new modified scaled conjugate gradient method for large-scale unconstrained optimization with non-convex objective function

In this paper, according to the fifth-order Taylor expansion of the objective function and the modified secant equation suggested by Li and Fukushima, a new modified secant equation is presented. Also, a new modification of the scaled memoryless BFGS preconditioned conjugate gradient algorithm is suggested which is the idea to compute the scaling parameter based on a two-point approximation of our new modified secant equation. A remarkable feature of the proposed method is that it possesses a globally convergent even without convexity assumption on the objective function. Numerical results show that the proposed new modification of scaled conjugate gradient is efficient.

Mini-batch optimized full waveform inversion with geological constrained gradient filtering

High computation cost and generating solutions without geological sense have hindered the wide application of Full Waveform Inversion (FWI). Source encoding technique is a way to dramatically reduce the cost of FWI but subject to fix-spread acquisition setup requirement and slow convergence for the suppression of cross-talk. Traditionally, gradient regularization or preconditioning is applied to mitigate the ill-posedness. An isotropic smoothing filter applied on gradients generally gives non-geological inversion results, and could also introduce artifacts. In this work, we propose to address both the efficiency and ill-posedness of FWI by a geological constrained mini-batch gradient optimization method. The mini-batch gradient descent optimization is adopted to reduce the computation time by choosing a subset of entire shots for each iteration. By jointly applying the structure-oriented smoothing to the mini-batch gradient, the inversion converges faster and gives results with more geological meaning. Stylized Marmousi model is used to show the performance of the proposed method on realistic synthetic model.

Stochastic Optimization of a Reinforced Concrete Elements Using Heuristic Algorithm

It is possible to see various applications of mathematical optimization in civil engineering (structural design, reconstruction of transportation networks etc.) Initially, deterministic approaches have been introduced to solve these problems. But despite their complexity, these approaches are insufficient to comprehend the probabilistic nature of said problems and thus provide only suboptimal solutions. Hence the effort comes to reconsider these deterministic approaches and deal with uncertainties involved in said problems in less straightforward way. The goal of the paper is to present the algorithm for stochastic optimization of design of steel-reinforced concrete cross-section. This algorithm is based on internal cycle of deterministic optimization using reduced gradient method and external cycle of stochastic optimization using regression analysis. Firstly, the deterministic problem is introduced and described. It is followed by the description of uncertainties, which are involved in the process, and stochastic reformulation of the problem. Then the algorithm itself is introduced and the paper ends with presentation of the results of performed calculations.

Multiple reduced gradient method for multiobjective optimization problems

Multiobjective optimization has a large number of real-life applications. Under this motivation, in this paper, we present a new method for solving multiobjective optimization problems with both linear constraints and bound constraints on the variables. This method extends, to the multiobjective setting, the classical reduced gradient method for scalar-valued optimization. The proposed algorithm generates a feasible descent direction by solving an appropriate quadratic subproblem, without the use of any scalarization approaches. We prove that the sequence generated by the algorithm converges to Pareto-critical points of the problem. We also present some numerical results to show the efficiency of the proposed method.

Thermodynamic efficiency improvement of combined cycle power plant's bottom cycle based on organic working fluids

This paper presents thermodynamic optimization of a bottom cycle which uses water and organic fluids as working fluids. The heat recovery steam generator is modeled with regard to different pressure levels, including the reheat of first pressure level, the configurations of heat-exchangers network where heat can be exchanged between flue gas and working fluids. Water is chosen as the working fluid of the first pressure level while an organic fluid for the second pressure level. Thermodynamic optimization of efficiency of the bottom cycle was conducted considering the variables of the heat-exchangers' inside the heat recovery steam generator HRSG and the operating parameters of working fluid of each pressure level and reheat. A genetic algorithm and a gradient optimization method were used with the thermodynamic model implanted in Matlab. It is shown that by using parallel and serial configurations of heat-exchangers and water in the first level pressure and organic working fluids in the second pressure level a better thermodynamic efficiency of the bottom cycle can be achieved.

Generalizing the Optimized Gradient Method for Smooth Convex Minimization

This paper generalizes the optimized gradient method (OGM) that achieves the optimal worst-case cost function bound of first-order methods for smooth convex minimization. Specifically, this paper studies a generalized formulation of OGM and analyzes its worst-case rates in terms of both the function value and the norm of the function gradient. This paper also develops a new algorithm called OGM-OG that is in the generalized family of OGM and that has the best known analytical worst-case bound with rate $O(1/N^{1.5})$ on the decrease of the gradient norm among fixed-step first-order methods. This paper also proves that Nesterov's fast gradient method has an $O(1/N^{1.5})$ worst-case gradient norm rate but with constant larger than OGM-OG. The proof is based on the worst-case analysis called Performance Estimation Problem.

Accelerated Residual Methods for the Iterative Solution of Systems of Equations

We present accelerated residual methods for the iterative solution of systems of equations by leveraging recent developments in accelerated gradient methods for convex optimization. The stability properties of the proposed method are analyzed for linear systems of equations by using the finite difference equation theory. Next, we introduce a residual descent restarting strategy and an adaptive computation of the acceleration parameter to enhance the robustness and efficiency of our method. Furthermore, we incorporate preconditioning techniques into the proposed method to accelerate its convergence. We demonstrate the performance of our method on systems of equations resulting from the finite element approximation of linear and nonlinear partial differential equations. In a variety of test cases, the numerical results show that the proposed method is competitive with the pseudo--time-marching method, Nesterov's method, and Newton--Krylov methods. Finally, we discuss some open issues that should be addressed in future research.

A note on the accelerated proximal gradient method for nonconvex optimization

We improve a recent accelerated proximal gradient (APG) method in [Li, Q., Zhou, Y., Liang, Y. and Varshney, P. K., Convergence analysis of proximal gradient with momentum for nonconvex optimization, in Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia, PMLR 70, 2017] for nonconvex optimization by allowing variable stepsizes. We prove the convergence of the APG method for a composite nonconvex optimization problem under the assumption that the composite objective function satisfies the Kurdyka-Łojasiewicz property.

Optimal control problem and its solution by grey wolf optimizer algorithm

The paper is devoted to a numerical method for solving the optimal control problem. The main approach to the numerical solution of the optimal control problem is the reduction of the optimal control problem to the problem of nonlinear programming and its following solution by classical gradient optimization methods. For this purpose, optimal control problem, which is a problem of searching time-dependent function, is replaced by the problem of searching of control values at discrete instants of time. An increase in the number of sampling points increases the accuracy of function approximation, but at the same time increases the dimensionality of the search space in the non-linear programming problem. In complex problems of non-linear programming with an unknown topology of the objective function, the statement that using classical gradient methods ensures finding a solution is not justified. The optimal control problem after the discretization and other modifications is often transformed to a non-linear programming problem with a non-unimodal objective function for which gradient methods are not applicable. In this paper we propose to solve the optimal control problem by evolutionary algorithms that do not use gradients and are able to find solutions of problems with nonunimodal objective function. The paper presents the modern evolutionary algorithm Grey wolf optimizer. The problem of the optimal combat turn of the aircraft is considered. In this problem the mathematical model of the control object is described by a system of seven ordinary differential equations. Also constraints on the value and rate of change of control are given. It is experimentally shown that the evolutionary algorithm Grey wolf optimizer successfully solves this optimal control problem.