Efficient Gradient-based Algorithm Research Articles

Optimizing risk budgets in portfolio selection problem: A bi-level model and an efficient gradient-based algorithm

Risk budgeting is one of the recent and successful strategies for asset portfolio selection. In this strategy, risk budgets are associated with assets, and the amount of investment is adjusted so that the contribution of each asset to the portfolio risk is proportional to its risk budget. To the best of our knowledge, no specific method has been presented in the literature to systematically determine the value of risk budgets. To fill this research gap, in this article, we consider the risk budgets as decision variables and present a bi-level programming model where the upper level decides the risk budgets and the lower level determines the risk budgeting portfolio. Three approaches are introduced to solve the model. The first is a single-level reformulation of the bi-level model, the second is a novel gradient-based algorithm, and the third is the particle swarm optimization algorithm. Moreover, the k-means clustering method is utilized to determine the assets involved in the portfolio. Computational results over real-world datasets demonstrate the significance of the bi-level model. In addition, the results confirm the proficiency of our gradient-based algorithm from both solution quality and running time.

Multihazard life-cycle cost optimization of active mass dampers in tall buildings

This article proposes an optimization-based-methodology for designing multiple active mass dampers (AMDs) in tall buildings in multihazard environments. A realistic cost of the AMD system is formulated as the objective-function, and the multihazard Life-Cycle-Cost (LCC) that represents the long-term hazard-related losses, is selected as the main performance-constraint. This allows joining winds and earthquakes in a single decision parameter. Additional constraints limit the dampers forces and strokes. To ensure stability of the actively controlled building, the LQR algorithm is nested within the formal optimization problem. In addition, for the first time in the context of AMDs, their masses are considered as design variables, emerging directly as an optimization output. As the LQR control law is not aimed at optimizing costs and LCC, the lower triangular matrices related to the LQR weighting matrices are used as additional design variables. Thus, their values converge throughout the optimization to optimize the formulated cost function while satisfying the constraints. As this strategy produces large-scale problems, an efficient gradient-based algorithm is developed, using the adjoint sensitivity analysis method. The framework is applied to several examples for a benchmark-building, providing consistent results. One example demonstrates that AMDs can offer similar performance to its passive TMDs counterpart, requiring a mass 40% lower.

A distribution-free smoothed combination method to improve discrimination accuracy in multi-category classification.

Results from multiple diagnostic tests are combined in many ways to improve the overall diagnostic accuracy. For binary classification, maximization of the empirical estimate of the area under the receiver operating characteristic curve has widely been used to produce an optimal linear combination of multiple biomarkers. However, in the presence of a large number of biomarkers, this method proves to be computationally expensive and difficult to implement since it involves maximization of a discontinuous, non-smooth function for which gradient-based methods cannot be used directly. The complexity of this problem further increases when the classification problem becomes multi-category. In this article, we develop a linear combination method that maximizes a smooth approximation of the empirical Hyper-volume Under Manifolds for the multi-category outcome. We approximate HUM by replacing the indicator function with the sigmoid function and normal cumulative distribution function. With such smooth approximations, efficient gradient-based algorithms are employed to obtain better solutions with less computing time. We show that under some regularity conditions, the proposed method yields consistent estimates of the coefficient parameters. We derive the asymptotic normality of the coefficient estimates. A simulation study is performed to study the effectiveness of our proposed method as compared to other existing methods. The method is illustrated using two real medical data sets.

Direct computation of magnetic surfaces in Boozer coordinates and coil optimization for quasisymmetry

We propose a new method to compute magnetic surfaces that are parametrized in Boozer coordinates for vacuum magnetic fields. We also propose a measure for quasisymmetry on the computed surfaces and use it to design coils that generate a magnetic field that is quasisymmetric on those surfaces. The rotational transform of the field and complexity measures for the coils are also controlled in the design problem. Using an adjoint approach, we are able to obtain analytic derivatives for this optimization problem, yielding an efficient gradient-based algorithm. Starting from an initial coil set that presents nested magnetic surfaces for a large fraction of the volume, our method converges rapidly to coil systems generating fields with excellent quasisymmetry and low particle losses. In particular for low complexity coils, we are able to significantly improve the performance compared with coils obtained from the standard two-stage approach, e.g. reduce losses of fusion-produced alpha particles born at half-radius from $17.7\,\%$ to $6.6\,\%$ . We also demonstrate 16-coil configurations with alpha loss ${<}1\,\%$ and neoclassical transport magnitude $\epsilon _{\text {eff}}^{3/2}$ less than approximately $5\times 10^{-9}$ .

A Gradient-Based Approach for Solving the Stochastic Optimal Power Flow Problem with Wind Power Generation

Although wind power generation improves decarbonization of the electricity sector, its increasing penetration poses new challenges for power systems planning, operation and control. In this paper, we propose a solution approach for Stochastic Optimal Power Flow (SOPF) models under uncertainty in wind power generation. Two complicating issues are handled: i) difficulties imposed by probability density functions used to formulate wind power costs and their derivatives; ii) the non-differentiability of the cost function for thermal units. Due to such issues, SOPF models cannot be solved by gradient-based approaches and have been solved by meta-heuristics only. We obtain exact analytical expressions for the first and second order derivatives of wind power costs and propose a technique for handling non-differentiability in thermal costs. The equivalent SOPF model that results from such recasting is a differentiable NLP problem which can be solved by efficient gradient-based algorithms. Finally, we propose a modified log-barrier primal-dual interior/exterior-point method for solving the equivalent SOPF model which, differently from meta-heuristic approaches, is able to calculate important dual variables such as energy prices. Our approach, which is applied to the IEEE 30-, 57- 118- and 300-bus systems, strongly outperforms a meta-heuristic approach in terms of computation times and optimality.

Equation-oriented methods for optimizing Rankine cycles using radial inflow turbine

This paper presents methods for optimizing the design and the performance of Rankine cycles using radial inflow turbines. Both methods follow a novel equation-oriented approach and involve a single mathematical problem that is solved by an efficient gradient-based algorithm. The capabilities of the proposed methods were demonstrated through a case study for power generation from the batch-wise casting process at a representative ferroalloy plant. More specifically, the proposed methods were used to design and analyze three Rankine cycles with CO2 as the working fluid. The design optimization method converged in most cases to essentially the same solution regardless of the start values of the independent variables. The performance optimization method demonstrated that the control approaches with variable rotational turbine speed improved the turbine off-design efficiency over the control approaches with a constant rotational speed. Moreover, the control approaches with variable inlet guide vanes improved the thermodynamic performance of the cycle by facilitating operation at a higher pressure than the control approaches with a fixed geometry turbine. Considering the flexibility, robustness and the computational cost of the proposed methods, they can be regarded as a powerful tool for the preliminary design and performance prediction of Rankine cycles.

Physics-informed Neural Networks-based Model Predictive Control for Multi-link Manipulators

We discuss nonlinear model predictive control (MPC) for multi-body dynamics via physics-informed machine learning methods. In more detail, we use a physics-informed neural networks (PINNs)-based MPC to solve a tracking problem for a complex mechanical system, a multi-link manipulator. PINNs are a promising tool to approximate (partial) differential equations but are not suited for control tasks in their original form since they are not designed to handle variable control actions or variable initial values. We thus follow the strategy of Antonelo et al. (arXiv:2104.02556, 2021) by enhancing PINNs with adding control actions and initial conditions as additional network inputs. Subsequently, the high-dimensional input space is reduced via a sampling strategy and a zero-hold assumption. This strategy enables the controller design based on a PINN as an approximation of the underlying system dynamics. The additional benefit is that the sensitivities are easily computed via automatic differentiation, thus leading to efficient gradient-based algorithms for the underlying optimal control problem.

Bi‐tuned semi‐active TMDs: Multi‐hazard design for tall buildings using gradient‐based optimization

Current research on tall buildings design has shown the need for multi-hazard approaches in numerous practical cases. To improve the performance under hazard-related horizontal loads, supplemental damping devices emerge as an efficient solution. For example, passive tuned massive dampers (TMDs) have been used in tall buildings for decades. However, since earthquakes and winds mobilize the structures in different ways, the multi-hazard design of TMDs remains a difficult task. Thus, adaptive devices are an encouraging solution for this purpose. This research proposes an efficient use of existing semi-active TMDs that were proposed for other purposes through tuning them exclusively to two sets of frequency and damping ratio: a set for seismic loads and a set for wind loads. These bi-tuned STMDs would behave as passive dampers during any hazard event, but with an appropriate tuning. Because mechanical properties switch only at the beginning and at the end of seismic events, the control system required would be simple and reliable. An optimization-based methodology is proposed to design multiple BSTMDs. The total added mass is minimized, while the building life-cycle cost (LCC) is selected as a performance constraint. An efficient gradient-based algorithm, originally developed for passive TMDs, is adapted for BSTMDs. Two problem formulations are proposed, to allow distinct design alternatives. The devices design is illustrated by four case studies. The examples show that BSTMDs may provide the same performance of their passive counterparts, using less than half of their total added mass.

Layout optimization of continuum structures embedded with movable components and holes simultaneously

In this paper, the layout optimization problem of continuum structures embedded with movable components and holes simultaneously is solved for the first time. We propose a new methodology for the embedding problem under SIMP-based computational framework, where the positions and orientations of embedded components and holes and the topology of connecting structure are optimized concurrently to maximize the overall stiffness. To this end, the parameterized topology description function, combining the Kreisselmeier-Steinhauser (KS) function, is used to construct the geometric shapes of embedded components and holes. The material density defining the topology of connecting structure and the geometric parameters used to describe the location and orientation of embedded components and holes are considered as design variables of the optimization problem. To unite these two seemingly different representations into a single computational framework, we first project the embedded components and holes into two density fields on a fixed grid using a smoothed Heaviside function, then introduce a new SIMP-motivated material interpolation scheme invoked at the finite element level for material parameterization. The material parameterization scheme supports full analytical sensitivities, which can greatly improve the accuracy and efficiency of sensitivity analysis, and make it suitable for use with efficient gradient-based algorithms. Finally, the effectiveness of the proposed method is illustrated by several numerical examples.

Efficient Gradient-Based Algorithm with Numerical Derivatives for Expedited Optimization of Multi-Parameter Miniaturized Impedance Matching Transformers

Efficient Gradient-Based Algorithm with Numerical Derivatives for Expedited Optimization of Multi-Parameter Miniaturized Impedance Matching Transformers

Designing PAR-constrained periodic/aperiodic sequence via the gradient-based method

Periodic/aperiodic sequence with good correlation and stopband properties has received widespread attention and been widely applied in radar and communication systems. To meet the hardware requirement and maximize the transmitter efficiency, the unimodular or peak-to-average ratio (PAR) constraint is always required in sequence design. In this paper, we consider the problem of designing PAR-constrained periodic/aperiodic sequence with good properties. After establishing the corresponding criterions for both correlation and stopband properties, the unified PAR-constrained problem is formulated and then transformed into an unconstrained minimization problem via sequence synthesis. To solve the problem, an efficient gradient-based algorithm is proposed to minimize the objective function directly. As the main steps can be implemented by fast Fourier transform (FFT) operations and Hadamard product, the whole algorithm is computationally efficient. In addition, the proposed algorithm can be applied to design both the periodic and aperiodic sequences by choosing proper parameters. Numerical experiments show that the proposed algorithm has better performance than the state-of-the-art algorithms in terms of the sequence quality and the running time.

Designing Waveform Sets with Good Correlation and Stopband Properties for MIMO Radar via the Gradient-Based Method.

Waveform sets with good correlation and/or stopband properties have received extensive attention and been widely used in multiple-input multiple-output (MIMO) radar. In this paper, we aim at designing unimodular waveform sets with good correlation and stopband properties. To formulate the problem, we construct two criteria to measure the correlation and stopband properties and then establish an unconstrained problem in the frequency domain. After deducing the phase gradient and the step size, an efficient gradient-based algorithm with monotonicity is proposed to minimize the objective function directly. For the design problem without considering the correlation weights, we develop a simplified algorithm, which only requires a few fast Fourier transform (FFT) operations and is more efficient. Because both of the algorithms can be implemented via the FFT operations and the Hadamard product, they are computationally efficient and can be used to design waveform sets with a large waveform number and waveform length. Numerical experiments show that the proposed algorithms can provide better performance than the state-of-the-art algorithms in terms of the computational complexity.

Approximation by piecewise polynomials on Voronoi tessellation

We propose a novel method to approximate a function on 2D domain by piecewise polynomials. The Voronoi tessellation is used as a partition of the domain, on which the best fitting polynomials in L2 metric are constructed. Our method optimizes the domain partition and the fitting polynomials simultaneously by minimizing an objective function indicating the approximation quality. We also provide the explicit formula of the gradient of the objective function, which makes an efficient gradient-based algorithm workable for the function minimization. We conduct several experiments to demonstrate the efficacy of our new approach for generating piecewise polynomial approximations of analytic functions and color images.

Joint optimization of number of wells, well locations and controls using a gradient-based algorithm

This paper presents a detailed algorithm for solving the general well-placement optimization problem in which the number of wells, their locations and rates are simultaneously optimized with an efficient gradient-based algorithm. The proposed well-placement optimization algorithm begins by placing a large number of wells in the reservoir, where, the well rates are the optimization variables. During iterations of the algorithm, most of the wells are eliminated by setting their rates to zero. The remaining wells and their controls determine the optimal number of wells, their optimum locations and rates. The well-placement algorithm consists of two optimization stages. In the initialization stage, the appropriate total reservoir production rate (or the total injection rate) for the set of to-be-optimized producers (or injectors) is estimated by maximizing the net-present-value for the specified operational life of the reservoir. In the second stage, a modified net-present-value functional which also considers the drilling cost of the wells is maximized subject to the a total rate constraint determined in the initialization stage. Both stages of the algorithm use gradient projection to enforce the linear and bound constraints, where the required gradients are computed with the adjoint method. The bottomhole pressure constraints on the wells are enforced using a practical approach. The applicability and robustness of our well-placement algorithm is discussed through several example problems.

Aerodynamic Optimization Algorithm with Integrated Geometry Parameterization and Mesh Movement

An efficient gradient-based algorithm for aerodynamic shape optimization is presented. The algorithm consists of several components, including a novel integrated geometry parameterization and mesh movement, a parallel Newton―Krylov flow solver, and an adjoint-based gradient evaluation. To integrate geometry parameterization and mesh movement, generalized B-spline volumes are used to parameterize both the surface and volume mesh. The volume mesh of B-spline control points mimics a coarse mesh; a linear elasticity mesh-movement algorithm is applied directly to this coarse mesh and the fine mesh is regenerated algebraically. Using this approach, mesh-movement time is reduced by two to three orders of magnitude relative to a node-based movement. The mesh-adjoint system also becomes smaller and is thus amenable to complex-step derivative approximations. When solving the flow-adjoint equations using restarted Krylov-subspace methods, a nested-subspace strategy is shown to be more robust than truncating the entire subspace. Optimization is accomplished using a sequential-quadratic-programming algorithm. The effectiveness of the complete algorithm is demonstrated using a lift-constrained induced-drag minimization that involves large changes in geometry.

Dynamic Motion Planning for the Design of Robotic Gait Rehabilitation

In this paper we examine a method to control the stepping motion of a paralyzed person suspended over a treadmill using a robot attached to the pelvis. A leg swing motion is created by moving the pelvis without contact with the legs. The problem is formulated as an optimal control problem for an underactuated articulated chain. The optimal control problem is converted into a discrete parameter optimization and an efficient gradient-based algorithm is used to solve it. Motion capture data from an unimpaired human subject is compared to the simulation results from the dynamic motion optimization. Our results suggest that it is feasible to drive repetitive stepping on a treadmill by a paralyzed person by assisting in torso movement alone. The optimized, pelvic motion strategies are comparable to "hip-hiking" gait strategies used by people with lower limb prostheses or hemiparesis. The resulting motions can be found at the web site http://ww.eng.uci.edu/-chwang/project/stepper/stepper.html.

Application of an improved subpixel registration algorithm on digital speckle correlation measurement

Digital speckle correlation method (DSCM) has been widely used in experimental mechanics to obtain the surface deformation fields. One of the challenges in practical applications is how to obtain the high accuracy with far less computation complexity. To determine the subpixel registration of the DSCM, a high efficient gradient-based algorithm is developed in this paper. The principle is described and four different modes of the algorithm are given. Based on computer-simulated images, the optimal mode of the algorithm is verified through the comparison of computation time, optimal subset-region size and sensitivity of the four modes. The influences of speckle-granule size and speckle-granule density on accuracy are studied and a quantitative estimation of the optimal speckle-granule size range is obtained. As the applications of this method, the practical deformation measurements with the rigid body translation and rotation as well as an experiment on biomechanics are presented to certify the feasibility and the validity of the algorithm.

Iterative and sequential algorithms for multisensor signal enhancement

In problems of enhancing a desired signal in the presence of noise, multiple sensor measurements will typically have components from both the signal and the noise sources. When the systems that couple the signal and the noise to the sensors are unknown, the problem becomes one of joint signal estimation and system identification. The authors specifically consider the two-sensor signal enhancement problem in which the desired signal is modeled as a Gaussian autoregressive (AR) process, the noise is modeled as a white Gaussian process, and the coupling systems are modeled as linear time-invariant finite impulse response (FIR) filters. The main approach consists of modeling the observed signals as outputs of a stochastic dynamic linear system, and the authors apply the estimate-maximize (EM) algorithm for jointly estimating the desired signal, the coupling systems, and the unknown signal and noise spectral parameters. The resulting algorithm can be viewed as the time-domain version of the frequency-domain approach of Feder et al. (1989), where instead of the noncausal frequency-domain Wiener filter, the Kalman smoother is used. This approach leads naturally to a sequential/adaptive algorithm by replacing the Kalman smoother with the Kalman filter, and in place of successive iterations on each data block, the algorithm proceeds sequentially through the data with exponential weighting applied to allow adaption to nonstationary changes in the structure of the data. A computationally efficient implementation of the algorithm is developed. An expression for the log-likelihood gradient based on the Kalman smoother/filter output is also developed and used to incorporate efficient gradient-based algorithms in the estimation process.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Maximum likelihood and lower bounds in system identification with non-Gaussian inputs

We consider the problem of estimating the parameters of an unknown discrete linear system driven by a sequence of independent identically distributed (i.i.d.) random variables whose probability density function (PDF) may be non-Gaussian. We assume a general system structure that may contain causal and noncausal poles and zeros. The parameters characterizing the input PDF may also be unknown. We derive an asymptotic expression for the Cramer-Rao lower bound, and show that it is the highest (worst) in the Gaussian case, indicating that the estimation accuracy can only be improved when the input PDF is non-Gaussian. It is further shown that the asymptotic error variance in estimating the system parameters is unaffected by lack of knowledge of the PDF parameters, and vice verse. Computationally efficient gradient-based algorithms for finding the maximum likelihood estimate of the unknown system and PDF parameters, which incorporate backward filtering for the identification of non-causal parameters, are presented. The dual problem of blind deconvolution/equalization is considered, and asymptotically attainable lower bounds on the equalization performance are derived. These bounds imply that it is preferable to work with compact equalizer structures characterized by a small number of parameters as the attainable performance depend only on the total number of equalizer parameters.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

An Efficient Gradient-Based Algorithm for On-Line Training of Recurrent Network Trajectories

A novel variant of the familiar backpropagation-through-time approach to training recurrent networks is described. This algorithm is intended to be used on arbitrary recurrent networks that run continually without ever being reset to an initial state, and it is specifically designed for computationally efficient computer implementation. This algorithm can be viewed as a cross between epochwise backpropagation through time, which is not appropriate for continually running networks, and the widely used on-line gradient approximation technique of truncated backpropagation through time.