Steepest Descent Algorithm Research Articles

A Quasi-Newton Trust-Region Method for Well Location Optimization Under Uncertainty

Summary Subsurface development involves well-placement decisions considering the highly uncertain understanding of the reservoir in the subsurface. The simultaneous optimization of a large number of well locations is a challenging problem. Traditional gradient-based methods can be adapted for well location optimization (WLO) when these problems are converted into real-valued representations and equipped with protocols to handle noisy objective functions. However, their application to large-scale scenarios often remains impractical. This impracticality arises because computing gradients of the objective function can be prohibitively expensive in realistic settings, particularly without using the adjoint method. In this paper, we explore the application of a novel quasi-Newton trust-region (TR) method that employs the stochastic simplex approximate gradient (StoSAG). We have implemented the Broyden-Fletcher-Goldfarb-Shanno (BFGS) quasi-Newton updating algorithm together with line-search (LS) and TR optimization strategies. The StoSAG-based optimization methods have been applied to a realistic synthetic reservoir featuring 26 wells considering two distinct cases: Each includes 20 realizations of porosity and permeability. The first case exhibits mild heterogeneity, while the second exhibits significant heterogeneity with a large correlation length. We have conducted a series of runs to evaluate the performance of these algorithms in addition to comparisons to the finite-difference (FD) and particle-swarm-optimization (PSO) algorithms. We introduce a novel approach to enhance the accuracy of StoSAG gradients by proposing modified StoSAG formulations. These formulations are tailored to exploit the structure of the objective function and to capture the relationships between its components and the individual optimization parameters. This approach involves using a correction matrix W informed by problem-specific knowledge. The entries of W vary from 0 to 1 and are proportional to the interference effects the neighboring wells have on each other. We determine those entries (or weights) based on the radii of investigation around the wells and the distance between the well pairs. Results indicate that the steepest-descent (SD) algorithm coupled with StoSAG has superior performance to PSO and FD. Although the objective function is prone to numerical noise and not continuously differentiable with respect to well locations, StoSAG overcomes this challenge because it acts as a smooth approximation. Comparative tests further confirm that the TR-BFGS is more effective than the LS-BFGS. Moreover, we show that using the proposed gradient correction procedure results in a significant acceleration in convergence, indicating an enhancement in the StoSAG gradient approximation quality. This enhancement allows the TR-BFGS algorithm to achieve considerably higher performance than SD, illustrating that the accuracy of the BFGS approximation benefits from improved gradient quality.

A NEW CONVERGENCE ANALYSIS FOR MINIMUM NORM SOLUTION OF SPLIT SYSTEM OF NONSMOOTH MINIMIZATION PROBLEMS

This work presents a novel self-adaptive steepest-descent type algorithm for solving the split system of minimization problem (SSMP) related to convex nonsmooth functions. The algorithm includes a self-adaptive step size mechanism, which uses a step size that does not need prior information about the operator norm. Under certain weakened assumptions of parameters, a strong convergence theorem is established and proved for the algorithm. Specifically, the sequence generated by this new algorithm strongly converges towards the minimum norm element of the SSPM. To assess the implementation of our algorithm, a numerical example is provided. According to the numerical results, our algorithm showcases effectiveness and simplicity in its implementation. Furthermore, the primary numerical experiment results indicate that our proposed algorithm surpasses some existing results in the literature in terms of CPU time and iteration count. Our result represents an extension and enhancement of recent findings in this area.

Ant-colony optimization-based multi-input multi-output (ACO-MIMO) equalization for low-complexity mode-division multiplexing (MDM) processing

In mode-division multiplexing (MDM) systems, the computational complexity of the multi-input multi-output (MIMO) equalization module is a critical obstacle to practical development. The step size μ and the number of taps K are key parameters in the equalization algorithm, influencing the performance of finite impulse response (FIR) equalizers, including convergence speed and output signal quality. To alleviate the computational burden of locating the optimal μ-K combination, we propose two ant colony optimization (ACO) -based MIMO equalization schemes: the fixed ACO-MIMO and the random ACO-MIMO, corresponding to two optimization strategies. These schemes expedite the initialization process of both parameters. Subsequently, we conduct experiments to evaluate their performance in a 3-mode recirculating-loop transmission system. Our findings demonstrate that, compared to conventional schemes, such as genetic algorithm (GA) and steepest descent algorithm (SDA), the proposed ACO-MIMO schemes significantly reduce the number of calls to the equalization algorithm for locating optimal μ-K combination by up to 42.74% and 80.63%, reducing the complexity of the whole MIMO equalization for MDM systems. And the resulting hit-rate Phit for the optimal μ-K combination reaches up to 99.34%. Moreover, the ACO-MIMO schemes exhibit stable performance across different data collected from various round-trips, confirming the robust operation for the long-haul MDM transmission. Finally, we investigate the performance disparity between the two proposed ACO-MIMO schemes through bit-error-rate (BER) distribution, concluding that under a large dataset with various BER distributions, the performance of both schemes is essentially equivalent.

Minimization of peak stresses with the shape derivative.

This article is concerned with the minimization of peak stresses occurring in linear elasticity. We propose to minimize the maximal von Mises stress of the elastic body. This leads to a non-smooth shape functional. We derive the shape derivative and associate it with the Clarke sub-differential. Using a steepest descent algorithm, we present numerical simulations. We compare our results to the usual [Formula: see text]-norm regularization and show that our algorithm performs better in the presented tests.This article is part of the theme issue 'Non-smooth variational problems with applications in mechanics'.

Optimality and KKT conditions for interval valued optimization problems on Hadamard manifolds

Recently, a new type of optimization problems, the so-called interval optimization problems on Hadamard manifolds, is introduced by the authors in Nguyen et al. [Interval optimization problems on Hadamard manifolds. J Nonlinear Convex Anal. 2023;24(11):2489–2511]. In this follow-up, we further offer the algorithmic bricks for these problems. More specifically, we characterize the optimality and KKT conditions for the interval valued optimization problems on Hadamard manifolds. For unconstrained problems, the existence of efficient points and the steepest descent algorithm are investigated. To the contrast, the KKT conditions and exact penalty approach are explored in the ones involving inequality constraints. These results pave the foundations for the solvability of interval valued optimization problems on Hadamard manifolds.

Multi-objective shape optimization of TESLA-like cavities: Addressing stochastic Maxwell's eigenproblem constraints

In this paper, we present a robust formulation to address nonlinear equality-constrained and inequality-constrained multi-objective optimization problems for multi-cell accelerating cavities with TESLA-like geometry under consideration of uncertain parameters. We propose to explore a robust approach that employs the polynomial chaos expansion (PCE) to model the uncertainty. For the uncertainty quantification (UQ), the resulting random-dependent eigenvalue problem is solved using the stochastic collocation method (SCM) combined with PCE. As a result, the shape optimization of the studied multi-cell cavity is defined in terms of statistical moments, which serve as the cost functionals in the multi-objective (MO) steepest descent algorithm. Two versions of this method are developed and implemented. Finally, the optimization results and their implications for the operation of a multi-cell cavity are discussed.

Simulation of range code tracking loop for multipath mitigation in NavIC receiver

Abstract The Operational Navigation with Indian Constellation (NavIC) comprises seven satellites in the orbit, including three in geostationary orbit (GEO) and four in geosynchronous orbit (GSO). NavIC provides both Standard Positioning Service and Restricted Service, using L5 (1176.45 MHz) and S1 (2492.028 MHz) frequencies, with coverage extending 1500 km around the mainland of India. In an urban canyon, multipath interference severely reduces the precision and reliability of NavIC positioning. Many current multipath mitigation techniques often exhibit high computational requirements or reliance on external assistance. In this paper, a ranging code tracking loop is proposed that can sustain either a late or early branch in contrast to the Narrow-Spacing (NS) correlation technique for mitigating multipath for NavIC receiver. The design of proposed code tracking loop is based on steepest descent algorithm. The findings demonstrate that, in terms of calculation time and code multipath mitigation, the suggested technique performs better than both Multipath Estimated Delay Locked Loop (MEDL) and NS correlation. The proposed method produces less than 0.016 chips for the tracking error Standard Deviation (STD). In addition, the recommended method takes 24 % less computation time.

Cost Function Unrolling in Unsupervised Optical Flow.

Steepest descent algorithms, which are commonly used in deep learning, use the gradient as the descent direction, either as-is or after a direction shift using preconditioning. In many scenarios calculating the gradient is numerically hard due to complex or non-differentiable cost functions, specifically next to singular points. This has been commonly overcome by increased DNN model sizes and complexity. In this work we propose a novel mechanism we refer to as Cost Unrolling, for improving the ability of a given DNN model to solve a complex cost function, without modifying its architecture or increasing computational complexity. We focus on the derivation of the Total Variation (TV) smoothness constraint commonly used in unsupervised cost functions. We introduce an iterative differentiable alternative to the TV smoothness constraint, which is demonstrated to produce more stable gradients during training, enable faster convergence and improve the predictions of a given DNN model. We test our method in several tasks, including image denoising and unsupervised optical flow. Replacing the TV smoothness constraint with our loss during DNN training, we report improved results in all tested scenarios. Specifically, our method improves flows predicted at occluded regions, a crucial task by itself, resulting in sharper motion boundaries.

Parallel-swath-array analysis of passive source surface waves based on beamforming

The detection of mined-out coal area remains a pivotal subject in geophysical prospecting and presents an urgent issue requiring resolution. A refined and effective approach to tackle this challenge involves conducting dispersion analysis on passive surface waves, which are derived from ambient noises. While classical passive surface wave methods, such as the Spatial Autocorrelation Method (SPAC), rely on spatial and time domain stationary assumptions, these can be difficult to fulfill. This is especially true when using ambient noises with frequencies >1 Hz, leading to a growing disparity between basic assumptions and actual scenario. To reduce this discrepancy, it's crucial to account for the non-stationary features of noise sources. In this study, we introduce a pioneering approach for parallel-swath-array analysis of passive source surface waves based on beamforming, making full use of non-stationary ambient noises. The parallel-swath array allows for efficient data collection and high-density spatial observation. The method's core concept involves: 1) The collection of noise data by a 2-D array arranged in three parallel lines; 2) The use of a plane-wave basis function to directly scan the original noise records in a time-segment mode within the time domain. These are conducive to unveil the azimuths and time-variant characteristics of the noise sources, resulting in accurate surface wave dispersion information. To ascertain the feasibility and effectiveness of this innovative method, we simulated passive surface wave records for a geological model, accommodating both time-invariant and time-variant distribution of noise sources. The results demonstrate that the method successfully reveals the time-invariant or time-variant characteristics of noise sources and the associated surface wave dispersion information. Simultaneously, we employ the preprocessed steepest-descent (PSD) algorithm to invert the dispersion curves, and the low-velocity layer in the numerical tests can be clearly revealed. In conclusion, we deployed this method in a coal mine situated in Northern China, which encompasses abandoned mined-out areas. This procedure effectively disclosed the intricate azimuthal properties of the noise sources, yielding high-resolution dispersion images. The inversion outcomes depicted a 2-D S-wave velocity section plunging to a depth of 60 m. By interpreting the low-velocity features of the mined-out area, we can accurately delineate its boundaries. The results of the interpretation align coherently with the findings from the drilling data, to verify the feasibility and effectiveness of parallel-swath-array analysis of passive source surface waves based on beamforming in detecting coal mined-out area under non-stationary ambient noises.

Design of novel deformable mirror with large displacement

SummaryA stabilized zoom system with deformable mirrors (DMs) was designed for continuous optical zoom. In order to adjust the focal length and expand the field angle, the characteristic surface shapes of the DMs composed of the low‐order and high‐order Zernike polynomials were provided. The maximum peak‐to‐valley (PV) value at the center of the surface shape is 80 μm. This work presents a piezoelectrically‐actuated deformable mirror (PADM) with large displacement for a stabilized zoom system. The COMSOL simulation was conducted to obtain the desired design by matching and optimizing structural parameters. The maximum displacement of PADM was more than 80 μm. The fitting results by the steepest descent algorithm (SD) showed that the RMS values of the residual surface shapes were 1 of 7 to 1 of 4 of their PV values, and the PV values of the residual surface shapes were less than 1 of 10 of the PV values of the target surface shapes.

Interval Type-2 Fuzzy Relation Matrix Model via Semitensor Product

A new modeling idea of interval type-2 fuzzy logic system (IT2 FLS) is proposed on the basis of matrix semi-tensor product (STP) fuzzy representation technique, to construct an interval type-2 fuzzy relationship matrix (IT2 FRM) model. The model naturally inherits the advantages of STP theory, which enables the fuzzy logic reasoning process to be translated into algebraic form for computation. In most cases, the fuzzy rules in the IT2 FRM are generally unknown. To address the above situation, the parameters of this model can be trained with observed data pairs. Firstly, the fuzzy rules are generated by the space division method, and the initial parameters of the model are determined. Secondly, the parameters in the model are trained using the steepest descent algorithm. Finally, the IT2 FRM model is validated by simulation, and the results demonstrate that the IT2 FLS can be designed using the STP method.

Improved Position Observer Using Adaptive Training Control-Based Filter for Interior Permanent Magnet Synchronous Motor Drives

Rotor position is the key information to achieve superior performance for sensorless control of interior permanent magnet synchronous motor (IPMSM). Nevertheless, the back electromotive force (EMF) model-based position estimation suffers from severe contamination from the fifth and seventh harmonics resulting from inverter nonlinearity and flux spatial harmonics. Therefore, an adaptive training control-based adaptive filter combined with a sliding-mode observer (SMO) is presented for harmonics rejection in the estimated back-EMF, and thus improving the rotor position estimation performance. This method, based on the steepest descent algorithm, is capable of self-adjusting harmonic coefficients to obtain the fundamental component online under various frequency conditions adaptively. Additionally, the proposed method has simpler structure and less calculation burden since its reference signal is self-generated without external injection compared to the conventional method. The effectiveness is verified by experiments at a 1.5-kW IPMSM drive platform.

A novel hybrid optimization model to determine optimum water resources for water supply of residential areas

In this study, a new hybrid model was improved by combining heuristic and numerical optimization algorithms to decide on optimum water resources based on their costs in the water supply. The purpose of the hybrid model is to reach a best result in the shortest time by simultaneously searching global and local minimums. Therefore, the steepest descent (SD) algorithm (numerical optimization method) was embedded in the classical modified clonal selection algorithm (the classical modified Clonalg) (one of artificial immune systems, heuristic optimization technique). This hybridization allows the SD algorithm to search local minimums while the classical modified Clonalg is searching a global minimum. The hybrid optimization model was applied to the cost objective function depending on distances and piezometric head differences between the water resources and destination. A scenario consists of five hypothetical water resources and one residential area/settlement. Herein, the aim is to satisfy the water demand of the residential area with a minimum cost from the water resources. The cost objective function was also minimized by the regular model (the classical model) according to the scenario, and their results were compared. Both models were run ten times for testing their stabilities. According to the results, the hybrid model is better than the regular model in terms of run time and stability. The hybrid model found a minimum cost for the water supply in a shorter time (in half) in comparison with the regular model in all runs.

The DESC stellarator code suite. Part 1. Quick and accurate equilibria computations

Three-dimensional equilibrium codes are vital for stellarator design and operation, and high-accuracy equilibria are also necessary for stability studies. This paper details comparisons of two three-dimensional equilibrium codes: VMEC, which uses a steepest-descent algorithm to reach a minimum-energy plasma state, and DESC, which minimizes the magnetohydrodynamic (MHD) force error in real space directly. Accuracy as measured by satisfaction of MHD force balance is presented for each code, along with the computation time. It is shown that DESC is able to achieve more accurate solutions, especially near axis. The importance of higher-accuracy equilibria is shown in DESC's better agreement of stability metrics with asymptotic formulae. DESC's global Fourier–Zernike basis also yields solutions with analytic derivatives explicitly everywhere in the plasma volume, provides improved accuracy in the radial direction versus conventional finite differences and allows for exponential convergence. Further, DESC can compute a solution with the same accuracy as VMEC in order-of-magnitude less time.

A comparison between steepest descent and non-linear conjugate gradient algorithms for binding energy minimization of organic molecules

The main intention of optimization is to bring about the “best” of any model by prioritizing the needs along with a given set of constraints. There is a wide range of optimization problems, among which, unfortunately, the problems that are formulated from nature are not convex in nature. Solving non-convex optimization problems are quite trickier than the conventional method of derivatives. One such problem is Computing the minimum value of binding free energy of various molecules. Minimization of free energy of a molecule is highly significant in the field of Molecular mechanics which is the foundation of computational biology. For any molecule, free energy refers to the amount of energy needed to separate an individual particle from a system of particles or to disseminate all particles in a system. The significance of free energy is that it can be used to compute the lowest energy conformation, which corresponds to the least Steric energy. Hence, this paper aims at computing the minimum value of binding free energy of various organic molecules in isolated conditions using the steepest descent algorithm and conjugate gradient algorithm and comparing them.

A tomographic reconstruction algorithm for cross-sectional imaging of IMRT beams from six projections

Objective. Patient-specific Quality Assurance (QA) measurements are of key importance in radiotherapy for safe and efficient treatment delivery and allow early detection of clinically relevant errors. Such QA processes remain challenging to implement for complex Intensity Modulated Radiation Therapy (IMRT) radiotherapy fields delivered using a multileaf collimator (MLC) which often feature small open segments and raise QA issues similar to those encountered in small field dosimetry. Recently, detectors based on long scintillating fibers have been proposed to measure a few parallel projections of the irradiation field with good performance for small field dosimetry. The purpose of this work is to develop and validate a novel approach to reconstruct MLC-shaped small irradiation fields from six projections. Approach. The proposed field reconstruction method uses a limited number of geometric parameters to model the irradiation field. These parameters are iteratively estimated with a steepest descent algorithm. The reconstruction method was first validated on simulated data. Real data were measured with a water-equivalent slab phantom equipped with a detector made of 6 scintillating-fiber ribbons placed at 1 m from the source. A radiochromic film was used to acquire a reference measurement of a first dose distribution in the slab phantom at the same source-to-detector distance and the treatment planning system (TPS) provided the reference for another dose distribution. In addition, simulated errors introduced on the delivered dose, field location and field shape were used to evaluate the ability of the proposed method to efficiently identify a deviation between the planned and delivered treatments. Main results. For a first small IMRT segment, 3%/3 mm, 2%/2 mm and 2%/1 mm gamma analysis conducted between the reconstructed dose distribution and the dose measured with radiochromic film exhibited pass rates of 100%, 99.9% and 95.7%, respectively. For a second and smaller IMRT segment, the same gamma analysis performed between the reconstructed dose distribution and the reference provided by the TPS showed pass rates of 100%, 99.4% and 92.6% for the 3%/3 mm, 2%/2 mm and 2%/1 mm gamma criteria, respectively. Gamma analysis of the simulated treatment delivery errors showed the ability of the reconstruction algorithm to detect a 3% deviation between the planned and delivered doses, as well as shifts lower than 7 mm and 3 mm when considering an individual leaf and a whole field shift, respectively. Significance. The proposed method allows accurate tomographic reconstruction of IMRT segments by processing projections measured with six scintillating-fiber ribbons and is suitable for water-equivalent real-time small IMRT segments QA.

Accelerated Conjugate Gradient for Second-Order Blind Signal Separation

This paper proposes a new adaptive algorithm for the second-order blind signal separation (BSS) problem with convolutive mixtures by utilising a combination of an accelerated gradient and a conjugate gradient method. For each iteration of the adaptive algorithm, the search point and the search direction are obtained based on the current and the previous iterations. The algorithm efficiently calculates the step size for the accelerated conjugate gradient algorithm in each iteration. Simulation results show that the proposed accelerated conjugate gradient algorithm with optimal step size converges faster than the accelerated descent algorithm and the steepest descent algorithm with optimal step size while having lower computational complexity. In particular, the number of iterations required for convergence of the accelerated conjugate gradient algorithm is significantly lower than the accelerated descent algorithm and the steepest descent algorithm. In addition, the proposed system achieves improvement in terms of the signal to interference ratio and signal to noise ratio for the dominant speech outputs.

A Hybrid Steepest-Descent Algorithm for Convex Minimization Over the Fixed Point Set of Multivalued Mappings

In the setting of Hilbert spaces, we show that a hybrid steepest-descent algorithm converges strongly to a solution of a convex minimization problem over the fixed point set of a finite family of multivalued demicontractive mappings. We also provide numerical results concerning the viability of the proposed algorithm with possible applications.

Output tracking control of non‐minimum phase systems using on‐line output redefinition approach based on barrier function method

This paper considers the output tracking problem of non-minimum phase non-linear systems using the inverse dynamics control which is impossible in conventional methods due to unstable zero dynamics. The proposed method is based on the output redefinition approach defined as a parametric function. The tracking error is decomposed into two terms. The first one named redefined output tracking error is the difference between the redefined output and the reference signal, and the second one is the distance between the redefined output and the system output. For the first term, due to minimum phase property of the system with the redefined output, an inverse dynamics control is used to guarantee the convergence of the redefined output tracking error to zero. To minimize the second term, the parameters of the redefined output are set by solving an optimization problem whose constraints include minimum phase conditions. To solve the optimization problem which satisfies all constraints, the barrier function method as an interior point method is adopted. The optimization parameters are set using steepest descent algorithm. Due to on-line updating the redefined output parameters, the related zero dynamics is time varying. Therefore, sufficient conditions for exponential stability of the resulting time-varying zero dynamics are obtained.

New search direction of steepest descent method for solving large linear systems

The steepest descent (SD) method is well-known as the simplest method in optimization. In this paper, we propose a new SD search direction for solving system of linear equations Ax = b. We also prove that the proposed SD method with exact line search satisfies descent condition and possesses global convergence properties. This proposed method is motivated by previous work on the SD method by Zubai’ah-Mustafa-Rivaie-Ismail (ZMRI)[2]. Numerical comparisons with a classical SD algorithm and ZMRI algorithm show that this algorithm is very effective depending on the number of iterations (NOI) and CPU time.