quasi-Newton Optimization Method Research Articles

A RIGOROUS THREE-DIMENSIONAL MAGNETOTELLURIC INVERSION

The limited-memory quasi-Newton optimization method with simple bounds has been applied to develop a novel fully three- dimensional (3-D) magnetotelluric (MT) inversion technique. This nonlinear inversion is based on iterative minimization of a classical Tikhonov-type regularized penalty functional. But instead of the usual model space of log resistivities, the approach iterates in a model space with simple bounds imposed on the conductivities of the 3-D target. The method requires storage that is proportional to ncp×N, where the N is the number of conductivities to be recovered and ncp is the number of the correction pairs (practically, only a few). This is much less than requirements imposed by other Newton type methods (that usually require storage proportional to N ×M ,o rN ×N, where M is the number of data to be inverted). Using an adjoint method to calculate the gradients of the misfit drastically accelerates the inversion. The inversion also involves all four entries of the MTimpedance matrix. The integral equation forward modelling code x3d by Avdeev et al. (1, 2) is employed as an engine for this inversion. Convergence, performance and accuracy of the inversion are demonstrated on a 3D MTsynthetic, but realistic, example.

Crop yield estimation model for Iowa using remote sensing and surface parameters

Numerous efforts have been made to develop various indices using remote sensing data such as normalized difference vegetation index (NDVI), vegetation condition index (VCI) and temperature condition index (TCI) for mapping and monitoring of drought and assessment of vegetation health and productivity. NDVI, soil moisture, surface temperature and rainfall are valuable sources of information for the estimation and prediction of crop conditions. In the present paper, we have considered NDVI, soil moisture, surface temperature and rainfall data of Iowa state, US, for 19 years for crop yield assessment and prediction using piecewise linear regression method with breakpoint. Crop production environment consists of inherent sources of heterogeneity and their non-linear behavior. A non-linear Quasi-Newton multi-variate optimization method is utilized, which reasonably minimizes inconsistency and errors in yield prediction.Minimization of least square loss function has been carried out through iterative convergence using pre-defined empirical equation that provided acceptable lower residual values with predicted values very close to observed ones (R2=0.78) for Corn and Soybean crop (R2=0.86) for Iowa state. The crop yield prediction model discussed in the present paper will further improve in future with the use of long period dataset. Similar model can be developed for different crops of other locations.

비대칭 벽식 구조지 변위기초 내진성능평가 및 보강

편심이 있는 구조물이 지진하중을 받는 경우 비틀림의 발생으로 특정부재에 응력 및 변위가 집중되고 이는 전혀 예상치 못한 구조물의 파괴를 유발할 수 있다. 본 연구에서는 각 횡저항 부재의 한계 변위를 기반으로 하여 구조물 전체의 횡변위와 비틀림각의 관계도(D-R Relationship: Displacement-Rotation Relationship)를 작성하고 변위스펙트럼을 이용하여 내진성능평가를 수행하는 방법을 제안한다. 제안된 내진성능평가의 방법은 시간이력해석의 결과를 이용해서 검증하였다. 또한 다양한 지진수준에 대해 구조물의 다른 성능수준을 기준으로 하는 다단계 내진성능평가를 수행하였다. 최종적으로 그 결과를 기준으로 D-R 관계도를 이용한 내진보강 전략을 제시하였다. 내진보강은 각 부재의 강도/강성을 증가시키는 방법과 연성도를 증가시키는 두 가지의 방법을 사용하였다. 특히 강도/강성을 증가시키는 내진보강 전략에서는 보강의 최적화를 위하여 보강전략을 최적화 문제로 구성하고 BFGS Quasi-Newton method를 이용하여 최적보강전략을 수립하는 과정을 제시하였다. Torsional behavior of eccentric structure under seismic leading may cause the stress and/or deformation concentration, which arouse the failure of the structure in an unexpected manner. This study suggests D-R relationship which shows the overall displacement and rotation of the system based on the ultimate displacement capacity of the each lateral load resistant member. Using the suggested D-R relationship and displacement spectrum, the seismic assessment is conducted and verified in comparison with the time history analysis result. Multi-level seismic assessment Is considered which takes multiple seismic hazard levels and respective performance levels into account. Finally, based on the seismic assessment result, seismic rehabilitation process is presented. In this research, two rehabilitation methods are considered. One is done by means of stiffening/strengthening the seismic resistant members, and the other is based on the member ductility. Especially, in the first method, to optimize the rehabilitation result, the rehabilitation problem is modeled as an optimization problem, and solved using BFGS quasi-Newton optimization method.

Further results on nakagami-m parameter estimation

Accurate and efficient estimation of fading parameters are of considerable interest in the design of adaptive radios since they provide an indication of the communication link quality. In this letter, we develop a few simple estimators for the Nakagami-m, parameter by approximating the transcendental equations that arise in the computation of maximum-likelihood and generalized moment based estimators as linear of quadratic polynomials using quasinewton optimization method. Statistical properties and efficacy of the new estimators are validated via Monte Carlo simulations.

Further results on Nakagami-m parameter estimation

Accurate and efficient estimation of fading parameters are of considerable interest in the design of adaptive radios since they provide an indication of the communication link quality. In this letter, we develop a few simple estimators for the Nakagami-m, parameter by approximating the transcendental equations that arise in the computation of maximum-likelihood and generalized moment based estimators as linear of quadratic polynomials using quasinewton optimization method. Statistical properties and efficacy of the new estimators are validated via Monte Carlo simulations.

Two-stage hybrid optimization of fiber Bragg gratings for design of linear phase filters

We present a new hybrid optimization method for the synthesis of fiber Bragg gratings (FBGs) with complex characteristics. The hybrid optimization method is a two-tier search that employs a global optimization algorithm [i.e., the tabu search (TS) algorithm] and a local optimization method (i.e., the quasi-Netwon method). First the TS global optimization algorithm is used to find a "promising" FBG structure that has a spectral response as close as possible to the targeted spectral response. Then the quasi-Newton local optimization method is applied to further optimize the FBG structure obtained from the TS algorithm to arrive at a targeted spectral response. A dynamic mechanism for weighting of different requirements of the spectral response is employed to enhance the optimization efficiency. To demonstrate the effectiveness of the method, the synthesis of three linear-phase optical filters based on FBGs with different grating lengths is described.

Optimal estimation of tidal open boundary conditions using predicted tides and adjoint data assimilation technique

Lateral tidal open boundary conditions which force tides in the internal region are estimated by an adjoint data assimilation system which assimilates predicted coastal tidal elevations into a two-dimensional Princeton Ocean Model for the East Coast of the United States. Control variables are the harmonic constants (amplitude and phase) of tidal constituents (M 2, S 2, N 2, K 1, O 1) along the open boundary. The cost function is defined by the difference between predicted and model-simulated tidal elevations at coastal tide gauge locations. The limited memory Broyden–Fletcher–Goldfarb–Shanno quasi-Newton method for large-scale optimization is implemented to minimize the cost function. Identical twin experiments are performed to verify the adjoint model and to examine sensitivity of model results to the number and spatial distribution of tide gauge stations. The results from the predicted tidal elevation assimilation experiments show that the simulated tidal elevations forced by the optimal open boundary conditions are more accurate than those forced by the open boundary conditions derived from Schwiderski's global tidal model. For M 2 constituent, the maximum RMS error at tide gauge stations with data assimilation is generally 14 cm and the minimum correlation coefficient is 0.96. For the nine open coastal stations, the RMS errors are less than 5 cm . The results from the experiment in which five tidal constituents are considered together show that the RMS errors at the nine open coastal stations are less than 7 cm , and the correlation coefficients are greater than 0.99.

Global Convergence of the Broyden's Class of Quasi-Newton Methods with Nonmonotone Linesearch

In this paper, the Broyden class of quasi-Newton methods for unconstrained optimization is inves- tigated. Non-monotone linesearch procedure is introduced, which is combined with the Broyden's class. Under the convexity assumption on objective function, the global convergence of the Broyden's class is proved.

Assimilation of water level data into a coastal hydrodynamic model by an adjoint optimal technique

An adjoint data assimilation system has been developed to assimilate coastal subtidal water level data into a hydrodynamic model. In this system, a linear two-dimensional Princeton Ocean Model with an orthogonal curvilinear grid system is used as the forward model. The wind drag coefficient is used as a convenient control variable (approximately representing errors in the forecasting wind fields that are usually the primary cause of errors in model-produced water levels). The cost function is defined in terms of the water level misfits between the observations and model outputs. The limited memory Broyden–Fletcher–Goldfarb–Shanno (BFGS) quasi-Newton method for large-scale optimization is implemented to minimize the cost function. Identical twin experiments with model-generated pseudo-observations are performed and the results show that the true solution of the control variable can be recovered efficiently by assimilating pseudo-observations at limited locations into the model. The results from actual subtidal water level data assimilation experiments show that the simulated subtidal water levels with data assimilation are better than those without data assimilation even if only one control variable is used. The results from the experiment with 16 control variables demonstrate that the correlation coefficients are greater than 0.93 and the RMS errors are less than 5.3 cm at 18 coastal water level gauge stations. The nowcast/forecast experiments demonstrate that the subtidal water level forecasts are improved by water level data assimilation in the first 6 h. The average RMS error of the subtidal water level forecasts over the 18 water level gauge stations is reduced by 3 cm.

Implicit updates in multistep quasi-Newton methods

We consider multistep quasi-Newton methods for unconstrained optimization. These methods were introduced by Ford and Moghrabi [1,2], who showed how interpolating curves could be used to derive a generalization of the secant equation (the relation normally employed in the construction of quasi-Newton methods). One of the most successful of these multistep methods makes use of the current approximation to the Hessian to determine the parametrization of the interpolating curve in the variable-space and, hence, the generalized updating formula. In this paper, we investigate the use of implicit updates to the approximate Hessian, in an attempt to determine a better parametrization of the interpolation (while avoiding the computational burden of actually carrying out the update) and, thus, improve the numerical performance of such algorithms.

A nonlinear model for function-value multistep methods

We develop a framework employing scaling functions for the construction of multistep quasi-Newton methods for unconstrained optimization. These methods utilize values of the objective function. They are constructed via interpolants of the m+1 most recent iterates/gradient evaluations, and possess a free parameter which introduces an additional degree of flexibility. This permits the interpolating functions to assimilate information, in the form of function-values, which is readily available at each iteration. Motivated by previous experience [1] with the use of function-values in multistep methods, we investigate the incorporation of this information in the construction of the Hessian approximation at each iteration, in an attempt to accelerate convergence. We concentrate on a specific example from the general family of methods, corresponding to a particular choice of the scaling function, and from it derive three new algorithms. The relative numerical performance of these methods is assessed, and the most successful of them is then compared with the standard BFGS method and with an earlier algorithm utilizing function-values, also developed by the authors [1].

A Survey of Quasi-Newton Equations and Quasi-Newton Methods for Optimization

Quasi-Newton equations play a central role in quasi-Newton methods for optimization and various quasi-Newton equations are available. This paper gives a survey on these quasi-Newton equations and studies properties of quasi-Newton methods with updates satisfying different quasi-Newton equations. These include single-step quasi-Newton equations that use only gradient information and that use both gradient and function value information in one step, and multi-step quasi-Newton equations that use the gradient information in last m steps. Main properties of quasi-Newton methods with updates satisfying different quasi-Newton equations are studied. These properties include the finite termination property, invariance, heredity of positive definite updates, consistency of search directions, global convergence and local superlinear convergence properties.

Metric-Based Parameterizations for Multi-Step Unconstrained Optimization

We consider multi-step quasi-Newton methods for unconstrained optimization. These methods were introduced by Ford and Moghrabi (Appl. Math., vol. 50, pp. 305–323, 1994s Optimization Methods and Software, vol. 2, pp. 357–370, 1993), who showed how interpolating curves could be used to derive a generalization of the Secant Equation (the relation normally employed in the construction of quasi-Newton methods). One of the most successful of these multi-step methods makes use of the current approximation to the Hessian to determine the parameterization of the interpolating curve in the variable-space and, hence, the generalized updating formula. In this paper, we investigate new parameterization techniques to the approximate Hessian, in an attempt to determine a better Hessian approximation at each iteration and, thus, improve the numerical performance of such algorithms.

Powerful and flexible fuzzy algorithm for nonlinear dynamic system identification

A new powerful and flexible fuzzy algorithm for nonlinear dynamic system identification is presented. It is based on the identification of the derivative of the system state, instead of the future system state. The membership functions of the underlying static fuzzy model are two-sided Gaussian functions and the learning algorithm is a hybrid-nested routine based on least-squares, quasi-Newton and simplex optimization methods. Moreover, a simple clustering algorithm based on an additional higher level fuzzy model is proposed. The application to the identification of the Mackey-Glass chaotic time series is presented and compared with previous results in terms of maximum error and nondimensional error index. Finally, the application to a test nonlinear dynamic system is presented to show the capabilities of the clustering algorithm. The obtained results show that the proposed algorithm can find wide application in practical problems, such as in nonlinear electronic circuit design.

OPTIMIZATION-BASED SYNTHESIS OF A DEEP-DIGGING TILLAGE MECHANISM

A quasi-Newton optimization method is employed to synthesize a four-bar tillage mechanism, a device for loosening the sub-soil in farm fields. The synthesis routine uses sequential parameter transformations that map from an unconstrained search variable space to a constrained design variable space. The sequential parameter transformations ensure that only a specific mechanism sub-type is considered, that Grashof criteria are satisfied, and that all mechanism parameters satisfy specified upper and lower constraints. Objective functions quantifying the level of satisfaction of the desired task displacements are minimized. It is shown that in addition to task satisfaction, further objective function terms can be added. The addition of objective function terms to increase the minimum transmission angle and to reduce the mechanism size are found to allow the synthesis of a practical mechanism for the deep-digging application. The use of the Broyden-Fletcher-Goldfarb-Shanno (BFGS) method with Fletcher’s Line Search (FLS) algorithm has allowed for the development of an efficient optimization-based synthesis routine. The synthesis results demonstrate that the developed synthesis routine required on the order of 102 fewer iterations per start then the direct-search and Sequential Unconstrained Minimization Technique (SUMT) employed in a previous method.

X-Ray Fractography Using Synchrotron Radiation

The residual stress distributions just beneath the fatigue fracture surface were measured using synchrotron radiation of three different wavelengths having three different penetration depths. The residual stress distributions were estimated from these three diffraction data by the following process. First, a temporary residual stress distribution in the depth direction is assumed. Theoretical 2θ-sin2ψ diagrams for each wavelength are calculated by the cosψ method developed by one of the authors. The total of the differences between the theoretical and experimental values of the diffraction angle in 2θ-sin2ψ diagrams is calculated. This total value is minimized by changing the assumed stress distribution using the quasi-Newton optimization method. Finally, the optimized 2θ-sin2ψ diagrams for each diffraction and the detailed stress distribution are determined. The true surface residual stress is obtained from this stress distribution. No effect of the load ratio R (=Pmin/Pmax) on the residual stress of the fatigue fracture surface in low-carbon steels was observed when the sin2ψ method was used for stress measurement. However, it was found in this synchrotron study that the residual stress became higher with increasing R. On the basis of this, the stress intensity factor range, ΔK, can be estimated from the residual stress on the fatigue fracture surface.

Ｘ線材料強度 シンクロトロン放射光によるＸ線フラクトグラフィ 疲労破面直下の残留応力分布

The residual stress distributions just beneath the fatigue fracture surface were measured using synchrotron radiation with three different wavelengths, i.e., three different penetration depths. The residual stress distributions were estimated from three kinds of diffraction data by the following process. First, a temporary residualstress distribution in the depth direction is assumed. Theoretical 2θ-sin2Ψ diagrams for each wavelength, where each has a different penetration depth, are calculated by the cosΨ method developed by one of the authors. The sum total of the differences between the theoretical and experimental values of the diffraction angle in 2θ-sin2Ψ diagrams is calculated. This total value is minimized by changing the assumed stress distribution by the quasi-Newton optimization method. Finally, optimized 2θ-sin2Ψ diagrams for each penetration depth and detailed stress distribution are determined. The true surface residual stress is obtained from this stress distribution.No effect of load ratio R(=Pmin/Pmax) on the residual stresses of the fatigue fracture surfaces in low-carbon steels was observed when the sin2Ψ method was used for stress measurement. However, the residual stresses became higher with increasing R when these were measured by the proposed method. On the basis of this, the stress intensity factor range, ΔK, can be estimated from the residual stress on the fatigue fracture surface.

Computational experiments with scaled initial hessian approximation for the broyden family methods*

In this paper we consider two alternative choices for the factor used to scale the initial Hessian approximation, before updating by a member of the Broyden family of updates for quasi-Newton optimization methods. By extensive computational experiments carried out on a set of standard test problems from the CUTE collection, using efficient implemen-tations of the quasi-Newton method, we show that the proposed new scaling factors are better, in terms of efficiency achieved (number of iterations, number of function and gradient evaluations), than the standard choice proposed in the literature

Ab initio classical trajectories on the Born–Oppenheimer surface: Updating methods for Hessian-based integrators

For the integration of the classical equations of motion in the Born–Oppenheimer approach, each time the energy and gradient of the potential energy surface are needed, a properly converged wave function is calculated. If Hessians (second derivatives) can be calculated, significantly larger steps can be taken in the numerical integration of the equations of motion without loss of accuracy. Even larger steps can be taken with a Hessian-based predictor–corrector algorithm. Since updated Hessians are used successfully in quasi-Newton methods for geometry optimization, it should be possible to improve the performance of trajectory calculations using updated Hessians. The Murtagh–Sargent (MS) update, the Powell-symmetric–Broyden (PSB) update and Bofill’s update (a weighted combination of MS and PSB) were tested, and Bofill’s update was found to be the best. Slightly smaller step sizes were needed with Hessian updating to maintain good conservation of the energy, but this was more than compensated by the reduction in total computational cost. An overall factor of 3 in speed-up was obtained for trajectories of systems containing 4 to 6 heavy atoms computed at the HF/3-21G level.

Proximal quasi-Newton methods for nondifferentiable convex optimization

. The method is based on Rockafellar’s proximal point algorithm and a cutting-plane technique. At each step, we use an approximate proximal point pa(xk) of xk to define a vk∈∂ekf(pa(xk)) with ek≤α∥vk∥, where α is a constant. The method monitors the reduction in the value of ∥vk∥ to identify when a line search on f should be used. The quasi-Newton step is used to reduce the value of ∥vk∥. Without the differentiability of f, the method converges globally and the rate of convergence is Q-linear. Superlinear convergence is also discussed to extend the characterization result of Dennis and More. Numerical results show the good performance of the method.