Descent Method Research Articles

Distributed Sparse Precision Matrix Estimation via Alternating Block-Based Gradient Descent

Precision matrices can efficiently exhibit the correlation between variables and they have received much attention in recent years. When one encounters large datasets stored in different locations and when data sharing is not allowed, the implementation of high-dimensional precision matrix estimation can be numerically challenging or even infeasible. In this work, we studied distributed sparse precision matrix estimation via an alternating block-based gradient descent method. We obtained a global model by aggregating each machine’s information via a communication-efficient surrogate penalized likelihood. The procedure chooses the block coordinates using the local gradient, to guide the global gradient updates, which can efficiently accelerate precision estimation and lessen communication loads on sensors. The proposed method can efficiently achieve the correct selection of non-zero elements of a sparse precision matrix. Under mild conditions, we show that the proposed estimator achieved a near-oracle convergence rate, as if the estimation had been conducted with a consolidated dataset on a single computer. The promising performance of the method was supported by both simulated and real data examples.

Feature-Based Inventory Control with Censored Demand

Problem definition: We study stochastic periodic-review inventory systems with lost sales, where the decision maker has no access to the true demand distribution a priori and can only observe historical sales data (referred to as censored demand) and feature information about the demand. In an inventory system, excess demand is unobservable because of inventory constraints, and sales data alone cannot fully recover the true demand. Meanwhile, feature information about the demand is abundant to assist inventory decisions. We incorporate features for inventory systems with censored demand. Methodology/results: We propose two feature-based inventory algorithms called the feature-based adaptive inventory algorithm and the dynamic shrinkage algorithm. Both algorithms are based on the stochastic gradient descent method. We measure the performance of the proposed algorithms through the average expected regret in finite periods: that is, the difference between the cost of our algorithms and that of a clairvoyant optimal policy with access to information, which is acting optimally. We show that the average expected cost incurred under both algorithms converges to the clairvoyant optimal cost at the rate of [Formula: see text] for the perishable inventory case and [Formula: see text] for the nonperishable inventory case. The feature-based adaptive inventory algorithm results in high volatility in the stochastic gradients, which hampers the initial performance of regret. The dynamic shrinkage algorithm uses a shrinkage parameter to adjust the gradients, which significantly improves the initial performance. Managerial implications: This paper considers feature information. The idea of dynamic shrinkage for the stochastic gradient descent method builds on a fundamental insight known as the bias-variance trade-off. Our research shows the importance of incorporating the bias-variance in a dynamic environment for inventory systems with feature information. Funding: W. T. Huh acknowledges support from the NSERC Discovery Grants [Grant RGPIN 2020-04213] and the Canada Research Chair Program. The work of Y. Rong was supported by the National Natural Science Foundation of China [Grants 72025201, 72331006, and 72221001]. Supplemental Material: The online appendix is available at https://doi.org/10.1287/msom.2021.0135 .

First-Order Algorithms Without Lipschitz Gradient: A Sequential Local Optimization Approach

Most first-order methods rely on the global Lipschitz continuity of the objective gradient, which fails to hold in many problems. This paper develops a sequential local optimization (SLO) framework for first-order algorithms to optimize problems without Lipschitz gradient. Operating on the assumption that the gradient is locally Lipschitz continuous over any compact set, SLO develops a careful scheme to control the distance between successive iterates. The proposed framework can easily adapt to the existing first-order methods, such as projected gradient descent (PGD), truncated gradient descent (TGD), and a parameter-free variant of Armijo linesearch. We show that SLO requires [Formula: see text] gradient evaluations to find an ϵ-stationary point, where Y is certain compact set with [Formula: see text] radius, and [Formula: see text] denotes the Lipschitz constant of the i-th order derivatives in Y. It is worth noting that our analysis provides the first nonasymptotic convergence rate for the (slight variant of) Armijo linesearch algorithm without globally Lipschitz continuous gradient or convexity. As a generic framework, we also show that SLO can incorporate more complicated subroutines, such as a variant of the accelerated gradient descent (AGD) method that can harness the problem’s second-order smoothness without Hessian computation, which achieves an improved [Formula: see text] complexity. Funding: J. Zhang is supported by the MOE AcRF [Grant A-0009530-04-00], from Singapore Ministry of Education. M. Hong is supported by NSF [Grants CIF-1910385 and EPCN-2311007]. Supplemental Material: The online appendix is available at https://doi.org/10.1287/ijoo.2021.0029 .

Long-time asymptotics for the integrable nonlocal Lakshmanan–Porsezian–Daniel equation with decaying initial value data

In this work, we study the Cauchy problem of integrable nonlocal Lakshmanan-Porsezian-Daniel equation with rapid attenuation of initial data. The basic Riemann–Hilbert problem of integrable nonlocal Lakshmanan-Porsezian-Daniel equation is constructed from Lax pair. Using Deift-Zhou nonlinear steepest descent method, the explicit long-time asymptotic formula of integrable nonlocal Lakshmanan-Porsezian-Daniel equation is derived, which is different from the local model. Besides, compared to the nonlocal nonlinear Schrödinger equation, since the increase of real stationary phase points, the long-time asymptotic formula for nonlocal Lakshmanan-Porsezian-Daniel equation becomes more complex.

Fast Multiobjective Gradient Methods with Nesterov Acceleration via Inertial Gradient-Like Systems

We derive efficient algorithms to compute weakly Pareto optimal solutions for smooth, convex and unconstrained multiobjective optimization problems in general Hilbert spaces. To this end, we define a novel inertial gradient-like dynamical system in the multiobjective setting, which trajectories converge weakly to Pareto optimal solutions. Discretization of this system yields an inertial multiobjective algorithm which generates sequences that converge weakly to Pareto optimal solutions. We employ Nesterov acceleration to define an algorithm with an improved convergence rate compared to the plain multiobjective steepest descent method (Algorithm 1). A further improvement in terms of efficiency is achieved by avoiding the solution of a quadratic subproblem to compute a common step direction for all objective functions, which is usually required in first-order methods. Using a different discretization of our inertial gradient-like dynamical system, we obtain an accelerated multiobjective gradient method that does not require the solution of a subproblem in each step (Algorithm 2). While this algorithm does not converge in general, it yields good results on test problems while being faster than standard steepest descent.

An Automatic Frequency Control System for Transmission Machinery Based on Back-Propagation Neural Network Algorithm in the Internet of Things Environment

Abstract In order to reduce the error of automatic frequency control of transmission machinery under different disturbances, an automatic frequency control system of transmission machinery based on back-propagation (BP) neural network algorithm is designed. The gradient descent method and Newton method are combined to optimize the BP neural network. Aiming at the periodicity and trend of the actual mechanical vibration frequency time series, the first order backward difference is processed by the difference method, and the autoregressive sequence is derived to obtain the control model. The experimental results show that the system can realize the automatic control of transmission mechanical frequency with small error under different types of interference, and the control effect is ideal.

Coordinate descent methods beyond smoothness and separability

Coordinate descent methods beyond smoothness and separability

A randomized sparse Kaczmarz solver for sparse signal recovery via minimax‐concave penalty

The randomized sparse Kaczmarz (RSK) method is an algorithm used to calculate sparse solutions for the basis pursuit problem. In this paper, we propose an algorithm framework for computing sparse solutions of linear systems, which includes the sparse Kaczmarz and sparse block Kaczmarz algorithms. In order to overcome the limitations of the penalty, we design an effective and new randomized sparse Kaczmarz algorithm (RSK‐MCP) based on the non‐convex minimax‐concave penalty (MCP) in sparse signal reconstruction. Additionally, we prove that the RSK‐MCP algorithm is equivalent to the randomized coordinate descent method for the corresponding dual problem. Based on this result, we demonstrate that the RSK‐MCP algorithm exhibits linear convergence, meaning it converges to a sparse solution of the MCP model when the regularization of MCP is a strongly convex function. Numerical experiments indicate that the RSK‐MCP algorithm outperforms RSK‐L1 in terms of both efficiency and accuracy.

Mainlobe deceptive jammer suppression with DEPC-MIMO radar with joint transmit–receive design

This paper investigates the problem of mainlobe deceptive jammer suppression via joint transmit–receive design in a multiple-input multiple-output (MIMO) radar. At the design stage, a discrete element-pulse coding (DEPC) phase is implemented in both the slow-time pulses and transmit array antennas. After decoding and compensation of the delayed pulses, the true and false targets can be identified in the joint transmit–receive (JTR) spatial frequency domain. Furthermore, to eliminate the false targets, a data-dependent beamforming technique is adopted, where an optimization problem is constructed to maximize the output signal-to-interference-plus-noise ratio (SINR) by optimizing the DEPC coefficients and the weight vector of the receive filter in an alternating way. Specifically, two methods, including the Single Jammer Suppression (SJS)-Alternating Direction Method of Multipliers (ADMM) and Multiple Jammers Suppression (MJS)-Gradient Descent (GD) methods are developed for the cases of a single jammer and multiple jammers, respectively. A series of numerical results are utilized to verify the effectiveness of the proposed methods for mainlobe deceptive jammer suppression.

Simulations of multiple breeding strategy scenarios in common bean for assessing genomic selection accuracy and model updating.

The aim of this study was to evaluate the accuracy of the ridge regression best linear unbiased prediction model across different traits, parent population sizes, and breeding strategies when estimating breeding values in common bean (Phaseolus vulgaris). Genomic selection was implemented to make selections within a breeding cycle and compared across five different breeding strategies (single seed descent, mass selection, pedigree method, modified pedigree method, and bulk breeding) following 10 breeding cycles. The model was trained on a simulated population of recombinant inbreds genotyped for 1010 single nucleotide polymorphism markers including 38 known quantitative trait loci identified in the literature. These QTL included 11 for seed yield, eight for white mold disease incidence, and 19 for days to flowering. Simulation results revealed that realized accuracies fluctuate depending on the factors investigated: trait genetic architecture, breeding strategy, and the number of initial parents used to begin the first breeding cycle. Trait architecture and breeding strategy appeared to have a larger impact on accuracy than the initial number of parents. Generally, maximum accuracies (in terms of the correlation between true and estimated breeding value) were consistently achieved under a mass selection strategy, pedigree method, and single seed descent method depending on the simulation parameters being tested. This study also investigated model updating, which involves retraining the prediction model with a new set of genotypes and phenotypes that have a closer relation to the population being tested. While it has been repeatedly shown that model updating generally improves prediction accuracy, it benefited some breeding strategies more than others. For low heritability traits (e.g., yield), conventional phenotype-based selection methods showed consistent rates of genetic gain, but genetic gain under genomic selection reached a plateau after fewer cycles. This plateauing is likely a cause of faster fixation of alleles and a diminishing of genetic variance when selections are made based on estimated breeding value as opposed to phenotype.

Analysis of Octane Retention Prediction Model for Catalytic Cracked Gasoline Based on Ridge Regression Model and Gradient Descent Optimization

On the basis of the given material, in order to increase the RON retention of the catalytic cracking unit, the prediction model of gasoline octane retention and the best operation variable inversion model were established based on the Ridge regression model and Gradient descent method. First, based on the Ridge regression model, the leave-one method is used to obtain the relative importance of the operational variables, and select the most important variables, so as to reduce the characteristic dimension of the model; Then, the RON retention prediction model is trained based on the Ridge regression model; Finally, based on the trained Ridge regression model and its weight parameters, the optimal operating variables were optimized separately using the gradient when the operation variable has a range or no range of value. The experimental results show that when 146 are selected from 361 operating variables, the model loss value stabilizes; when α is 0.6, the test set R2 is 0.9882, test set MSE is 0.0193, and the comprehensive performance is better than the random forest, support vector machine model; When the operation variable has two categories of value range and no value range, 2,000 times, the best inversion value of the operation variable makes the RON retention prediction value of the test sample similar to the expected value, and the MAE drops to 2.89999�10-3 and 7.62939�10-6, respectively. In conclusion, the RON retention prediction model proposed in this study has good results, and the best operating variable can be reversed, based on the given material parameters, making the optimal RON retention quantity.

Optimal Control of Temporal Networks With Variable Input and Node-Source Connection.

Many networked systems built upon real-life physical or social interactions have time-varying connections among individual units, where the temporal changes in connectivity and/or interaction strength lead to complicated dynamics. The temporal network model was proposed in the form of controlled linear dynamical systems acting in an ordered sequence of time intervals. One of the core challenges in network science is the control of networks and the optimization of the control strategy. However, most canonical frameworks for solving optimal control problems were established for static networks featuring constant topology. New theories and techniques are yet to be developed for the temporal networks, with an important case being that the input and the source-node connection are both variables. In this work, by formulating a quadratic energy cost without solving the Riccati differential equation, we show that the control effort can be reduced substantially by improving either the system trajectories or the input matrices. The two approaches are further combined in a coordinate descent framework, integrating linearly constrained quadratic programming, and a projected gradient descent method. Taken together, the results underline the potential of temporal networks as energy-efficient control systems and present strategies to improve the control input. Moreover, the proposed algorithms can serve as a starting point for future engineering of real-world temporal networks.

DC Cascaded Energy Storage System Based on DC Collector With Gradient Descent Method

With the continuous development of distributed energy, the energy storage system (ESS) is indispensable in improving power quality. Aiming at the application of large-capacity storage battery access to medium voltage dc power grid, a dc cascaded energy storage system based on the dc collector is proposed, and the characteristic, topology and control are presented in detail. In this scheme, the low-voltage storage batteries are accessed to medium voltage dc bus directly with dc collectors, which not only has higher efficiency, but also improves power density. Especially, the diversity of state of charge (SOC) is only reflected in the duty cycle of submodules (SMs), rather than the voltage deviation. Further, a variable angle carrier phase-shifted modulation with gradient descent method is discussed to eliminate the dominant current harmonics. By calculating the harmonic amplitude and solving the Jacobian matrix, the gradient that reduces harmonic current is obtained, and the optimal combination of the carrier phases is deduced by iterative calculation. Meanwhile, the method can be extended in the dc collector with any number of SMs to eliminate the dominant harmonics. Finally, the experimental prototype is built and the benefit of proposed solution is verified.

Automated Restarting Fast Proximal Gradient Descent Method for Single-View Cone-Beam X-ray Luminescence Computed Tomography Based on Depth Compensation.

Single-view cone-beam X-ray luminescence computed tomography (CB-XLCT) has recently gained attention as a highly promising imaging technique that allows for the efficient and rapid three-dimensional visualization of nanophosphor (NP) distributions in small animals. However, the reconstruction performance is hindered by the ill-posed nature of the inverse problem and the effects of depth variation as only a single view is acquired. To tackle this issue, we present a methodology that integrates an automated restarting strategy with depth compensation to achieve reconstruction. The present study employs a fast proximal gradient descent (FPGD) method, incorporating L0 norm regularization, to achieve efficient reconstruction with accelerated convergence. The proposed approach offers the benefit of retrieving neighboring multitarget distributions without the need for CT priors. Additionally, the automated restarting strategy ensures reliable reconstructions without the need for manual intervention. Numerical simulations and physical phantom experiments were conducted using a custom CB-XLCT system to demonstrate the accuracy of the proposed method in resolving adjacent NPs. The results showed that this method had the lowest relative error compared to other few-view techniques. This study signifies a significant progression in the development of practical single-view CB-XLCT for high-resolution 3-D biomedical imaging.

A Collocation Numerical Method for Highly Oscillatory Algebraic Singular Volterra Integral Equations

The highly oscillatory algebraic singular Volterra integral equations cannot be solved directly. A collocation numerical method is proposed to overcome the difficulty created by the highly oscillatory algebraic singular kernel. This paper is composed primarily of two methods—the piecewise constant collocation method and the piecewise linear collocation method—in which uniformly distributed nodes serve as collocation points. For the efficient computation of highly oscillatory and algebraic singular integrals, the steepest descent method as well as the Gauss–Laguerre and generalized Gauss–Laguerre quadrature rules are employed. Consequently, the resulting linear system is solved for the unknown function approximated by the Lagrange interpolation polynomial. Detailed theoretical analysis is carried out and numerical experiments showing high accuracy are also presented to confirm our analysis.

Numerical evaluation of oscillatory integrals via automated steepest descent contour deformation

Steepest descent methods combining complex contour deformation with numerical quadrature provide an efficient and accurate approach for the evaluation of highly oscillatory integrals. However, unless the phase function governing the oscillation is particularly simple, their application requires a significant amount of a priori analysis and expert user input, to determine the appropriate contour deformation, and to deal with the non-uniformity in the accuracy of standard quadrature techniques associated with the coalescence of stationary points (saddle points) with each other, or with the endpoints of the original integration contour. In this paper we present a novel algorithm for the numerical evaluation of oscillatory integrals with general polynomial phase functions, which automates the contour deformation process and avoids the difficulties typically encountered with coalescing stationary points and endpoints. The inputs to the algorithm are simply the phase and amplitude functions, the endpoints and orientation of the original integration contour, and a small number of numerical parameters. By a series of numerical experiments we demonstrate that the algorithm is accurate and efficient over a large range of frequencies, even for examples with a large number of coalescing stationary points and with endpoints at infinity. As a particular application, we use our algorithm to evaluate cuspoid canonical integrals from scattering theory. A Matlab implementation of the algorithm is made available and is called PathFinder.

A novel stepsize for gradient descent method

We propose a novel adaptive stepsize for the gradient descent scheme to solve unconstrained nonlinear optimization problems. With the convex and smooth objective satisfying locally Lipschitz gradient we obtain the complexity O(1k) of f(xk)−f⁎ at most. By using the idea of the new stepsize, we propose another new algorithm based on the projected gradient for solving a class of nonconvex optimization problems over a closed convex set. The computational experiments show the efficiency of the new method.

Inverse design of a grating metasurface for enhancing spontaneous emission through hyperbolic metamaterials

This work is concerned with inverse design of the grating metasurface over hyperbolic metamaterials (HMMs) in order to enhance spontaneous emission (SE). We formulate the design problem as a PDE-constrained optimization problem and employ the gradient descent method to solve the underlying optimization problem. The adjoint-state method is applied to compute the gradient of the objective function efficiently. Computational results show that the SE efficiency of the optical structure with the optimized metasurface increases by 600% in the near field compared to the bare HMM layer. In particular, an optimized double-slot metasurface obtained by this design method enhances the SE intensity by a factor of over 100 in the observation region.

Cochlear implant positioning: development and validation of an automatic method using computed tomography image analysis.

An in-house designed semi-automatic algorithm was developed to analyse a 3D model of the skull. The feasibility of drilling the recess was determined by a gradient descent method to search for the thickest part of the temporal bone. Feasibility was determined by the residual bone thickness which was calculated after a simulated drilling of the recess at the thickest position. An initial validation of the algorithm was performed by measuring the accuracy of the algorithm on five 3D models with known thickest locations for the recess. The accuracy was determined by a part comparison between the known position and algorithm provided position. In four of the five validation models a standard deviation for accuracy below the predetermined cut-off value of 4.2 mm was achieved between the actual thickest position and the position determined by the algorithm. Furthermore, the residual thickness calculated by the algorithm showed a high agreement (max. 0.02 mm difference) with the actual thickness. With the developed algorithm, a semi-automatic method was created to analyse the temporal bone thickness within a specified region of interest on the skull. Thereby, providing indications for surgical feasibility, potential risks for anatomical structures and impact on procedure time of cochlear implantation. This method could be a valuable research tool to objectively assess feasibility of drilling a recess in patients with thin temporal bones preoperatively.

Vehicle pitch dynamics control using in-wheel motors

Implementation of in-wheel motors brings numerous advantages: decreased overall vehicle weight and manufacture cost, available accurate torque control for better road handling and powertrain high energy efficiency. Even so, when the motor is placed inside the wheel, the unsprung mass of the vehicle increases, leading to a less comfortable ride. This paper puts forward and studies a control system that takes advantage of the vertical component of the driving force provided by an in-wheel motor to enhance ride comfort and handling dynamics. The proportional integral (PI) controller was proposed to minimize the pitch acceleration response of the vehicle. Controller’s gains were optimized using the gradient descent method with sequential quadratic programing algorithm. A co-simulation approach and a dynamic model of the vehicle were presented. Data regarding the vehicle’s measurements was taken from a real car with in-wheel motors. The dynamic model of the car was validated based on the results derived during the road test experiment. Assessing the controller’s effectiveness, a co-simulation approach between the dynamic model and the control system was established. An acceleration test in a straight line showed a 25% increase in ride comfort as measured by the RMS pitch angular vibrations of the sprung mass. The control is attained with no increase in the tire load and the vehicle’s longitudinal dynamics.