Perform Gradient Descent Research Articles

Improved protein complex prediction with AlphaFold-multimer by denoising the MSA profile.

Structure prediction of protein complexes has improved significantly with AlphaFold2 and AlphaFold-multimer (AFM), but only 60% of dimers are accurately predicted. Here, we learn a bias to the MSA representation that improves the predictions by performing gradient descent through the AFM network. We demonstrate the performance on seven difficult targets from CASP15 and increase the average MMscore to 0.76 compared to 0.63 with AFM. We evaluate the procedure on 487 protein complexes where AFM fails and obtain an increased success rate (MMscore>0.75) of 33% on these difficult targets. Our protocol, AFProfile, provides a way to direct predictions towards a defined target function guided by the MSA. We expect gradient descent over the MSA to be useful for different tasks.

Frequency Propagation: Multimechanism Learning in Nonlinear Physical Networks.

We introduce frequency propagation, a learning algorithm for nonlinear physical networks. In a resistive electrical circuit with variable resistors, an activation current is applied at a set of input nodes at one frequency and an error current is applied at a set of output nodes at another frequency. The voltage response of the circuit to these boundary currents is the superposition of an activation signal and an error signal whose coefficients can be read in different frequencies of the frequency domain. Each conductance is updated proportionally to the product of the two coefficients. The learning rule is local and proved to perform gradient descent on a loss function. We argue that frequency propagation is an instance of a multimechanism learning strategy for physical networks, be it resistive, elastic, or flow networks. Multimechanism learning strategies incorporate at least two physical quantities, potentially governed by independent physical mechanisms, to act as activation and error signals in the training process. Locally available information about these two signals is then used to update the trainable parameters to perform gradient descent. We demonstrate how earlier work implementing learning via chemical signaling in flow networks (Anisetti, Scellier, etal., 2023) also falls under the rubric of multimechanism learning.

Actor-critic objective penalty function method: an adaptive strategy for trajectory tracking in autonomous driving

Trajectory tracking is a key technology for controlling the autonomous vehicles effectively and stably to track the reference trajectory. How to handle the various constraints in trajectory tracking is very challenging. The recently proposed generalized exterior point method (GEP) shows high computational efficiency and closed-loop performance in solving the constrained trajectory tracking problem. However, the neural networks used in the GEP may suffer from the ill-conditioning issue during model training, which result in a slow or even non-converging training convergence process and the control output of the policy network being suboptimal or even severely constraint-violating. To effectively deal with the large-scale nonlinear state-wise constraints and avoid the ill-conditioning issue, we propose a model-based reinforcement learning (RL) method called the actor-critic objective penalty function method (ACOPFM) for trajectory tracking in autonomous driving. We adopt an integrated decision and control (IDC)-based planning and control scheme to transform the trajectory tracking problem into MPC-based nonlinear programming problems and embed the objective penalty function method into an actor-critic solution framework. The nonlinear programming problem is transformed into an unconstrained optimization problem and employed as a loss function for model updating of the policy network, and the ill-conditioning issue is avoided by alternately performing gradient descent and adaptively adjusting the penalty parameter. The convergence of ACOPFM is proved. The simulation results demonstrate that the ACOPFM converges to the optimal control strategy fast and steadily, and perform well under the multi-lane test scenario.

Mirror Descent of Hopfield Model

Abstract Mirror descent is an elegant optimization technique that leverages a dual space of parametric models to perform gradient descent. While originally developed for convex optimization, it has increasingly been applied in the field of machine learning. In this study, we propose a novel approach for using mirror descent to initialize the parameters of neural networks. Specifically, we demonstrate that by using the Hopfield model as a prototype for neural networks, mirror descent can effectively train the model with significantly improved performance compared to traditional gradient descent methods that rely on random parameter initialization. Our findings highlight the potential of mirror descent as a promising initialization technique for enhancing the optimization of machine learning models.

A mixed-signal oscillatory neural network for scalable analog computations in phase domain

Digital electronics based on von Neumann’s architecture is reaching its limits to solve large-scale problems essentially due to the memory fetching. Instead, recent efforts to bring the memory near the computation have enabled highly parallel computations at low energy costs. Oscillatory neural network (ONN) is one example of in-memory analog computing paradigm consisting of coupled oscillating neurons. When implemented in hardware, ONNs naturally perform gradient descent of an energy landscape which makes them particularly suited for solving optimization problems. Although the ONN computational capability and its link with the Ising model are known for decades, implementing a large-scale ONN remains difficult. Beyond the oscillators’ variations, there are still design challenges such as having compact, programmable synapses and a modular architecture for solving large problem instances. In this paper, we propose a mixed-signal architecture named Saturated Kuramoto ONN (SKONN) that leverages both analog and digital domains for efficient ONN hardware implementation. SKONN computes in the analog phase domain while propagating the information digitally to facilitate scaling up the ONN size. SKONN’s separation between computation and propagation enhances the robustness and enables a feed-forward phase propagation that is showcased for the first time. Moreover, the SKONN architecture leads to unique binarizing dynamics that are particularly suitable for solving NP-hard combinatorial optimization problems such as finding the weighted Max-cut of a graph. We find that SKONN’s accuracy is as good as the Goemans–Williamson 0.878-approximation algorithm for Max-cut; whereas SKONN’s computation time only grows logarithmically. We report on Weighted Max-cut experiments using a 9-neuron SKONN proof-of-concept on a printed circuit board (PCB). Finally, we present a low-power 16-neuron SKONN integrated circuit and illustrate SKONN’s feed-forward ability while computing the XOR function.

Accelerating Inverse Learning via Intelligent Localization with Exploratory Sampling

In the scope of "AI for Science", solving inverse problems is a longstanding challenge in materials and drug discovery, where the goal is to determine the hidden structures given a set of desirable properties. Deep generative models are recently proposed to solve inverse problems, but these are currently struggling in expensive forward operators, precisely localizing the exact solutions and fully exploring the parameter spaces without missing solutions. In this work, we propose a novel approach (called iPage) to accelerate the inverse learning process by leveraging probabilistic inference from deep invertible models and deterministic optimization via fast gradient descent. Given a target property, the learned invertible model provides a posterior over the parameter space; we identify these posterior samples as an intelligent prior initialization which enables us to narrow down the search space. We then perform gradient descent to calibrate the inverse solutions within a local region. Meanwhile, a space-filling sampling is imposed on the latent space to better explore and capture all possible solutions. We evaluate our approach on three benchmark tasks and create two datasets of real-world applications from quantum chemistry and additive manufacturing and find our method achieves superior performance compared to several state-of-the-art baseline methods. The iPage code is available at https://github.com/jxzhangjhu/MatDesINNe.

Learning by non-interfering feedback chemical signaling in physical networks

Both non-neural and neural biological systems can learn. So rather than focusing on purely brain-like learning, efforts are underway to study learning in physical systems. Such efforts include equilibrium propagation (EP) and coupled learning (CL), which require storage of two different states---the free state and the perturbed state---during the learning process to retain information about gradients. Here, we propose a learning algorithm rooted in chemical signaling that does not require storage of two different states. Rather, the output error information is encoded in a chemical signal that diffuses into the network in a similar way as the activation/feedforward signal. The steady-state feedback chemical concentration, along with the activation signal, stores the required gradient information locally. We apply our algorithm using a physical, linear flow network and test it using the Iris data set with 93% accuracy. We also prove that our algorithm performs gradient descent. Finally, in addition to comparing our algorithm directly with EP and CL, we address the biological plausibility of the algorithm.

Information-based seismic amplitude inversion

Amplitude variation with offset (AVO) inversion, particularly for more than two model parameters, is a highly ill-posed problem and, hence, regularization is indispensable. For common regularization such as norm-regularization, learning can be forged ahead via gradient descent. Alternatively, learning can be achieved by performing gradient descent on the objective and projects the result back into the space of the regularizer; this allows the use of atypical regularization functions. Here, we propose a regularized inverse problem to mitigate the Ill-posedness of the amplitude inversion. The regularization is added to measure the difference in information between the a priori probability density function and the predicted probability density of the inverted parameters, and therefore enhances the consistency between the inversion results and the a priori geological information. Information theory provides a collection of contrast functions which quantify the divergence from one probability distribution to another, such as the relative entropy. The a priori density is approximated by a Gaussian mixture model, obtained from well logs and rock physics data. The mixture model is a density estimator, providing the statistical properties of the model parameters of interest. The likelihood of the data and the divergence are combined in an augmented Lagrangian scheme, to obtain a stable solution that best generate the recorded seismic data and satisfy the geological constraints conveyed by the a priori probability density function. The proposed inversion scheme is applied to the anisotropy AVO inversion, for estimating the elastic and seismic anisotropy parameters of shale formations. Compared to the unconstrained minimization, the P- and S-wave velocity, and ε are better recovered, moreover, density and Thomsen's δ are well-constrained.

A Hybrid Iterative Thresholding Method for Solving Sparsity-Regularized Linear Inverse Problems and Its Application in Seismic Sparse-Spike Deconvolution

Sparsity-regularized linear inverse problem has been served as the base in many disciplines, such as remote sensing imaging, image processing and analysis, seismic deconvolution, compressed sensing, medical imaging, and so forth. Iterative hard thresholding algorithm (IHTA) and iterative soft thresholding algorithm (ISTA) are two frequently used methods to solve sparsity-regularized linear inverse problems. They are also the basic unit of other more complex methods. IHTA and ISTA are derived under steepest descent method. I.e., iteratively perform gradient descent and thresholding shrinkage. Steepest descent method is a first order algorithm, which is a powerful way to solve optimization due to its relatively simple implementation. However, a known issue of first order method is possible poor convergence rate. Fast iterative thresholding-like algorithms have been proposed to overcome this issue in the existing literatures. In history, another alternative way is second order algorithms or quasi-second order algorithms. In this paper, we include a quasi-Newton’s method, i.e., DFP (Davidon-Fletcher-Powell) formulations in the framework of iterative thresholding-like algorithms to replace gradient descent to form a hybrid method to further increase convergence rate. The proposed method has been performed on two numerical examples and a real-life application in sparse-spike seismic deconvolution. The numerical examples and real-life application showed that it provides an effective alternative method to solve sparsity-regularized linear inverse problems.

The Complexity of Gradient Descent: CLS = PPAD ∩ PLS

We study search problems that can be solved by performing Gradient Descent on a bounded convex polytopal domain and show that this class is equal to the intersection of two well-known classes: PPAD and PLS. As our main underlying technical contribution, we show that computing a Karush-Kuhn-Tucker (KKT) point of a continuously differentiable function over the domain [0,1] 2 is PPAD ∩ PLS-complete. This is the first non-artificial problem to be shown complete for this class. Our results also imply that the class CLS (Continuous Local Search) – which was defined by Daskalakis and Papadimitriou as a more “natural” counterpart to PPAD ∩ PLS and contains many interesting problems – is itself equal to PPAD ∩ PLS.

Problems and Prospects for the Application of Neural Networks for the Sphere of Education

The purpose of the study is to examine the theory and practice of using neural networks in education, to develop the concept of a neural network adaptive learning environment, and to implement a neural network model of one of the subsystems of this environment (using a model example of creating an adaptive educational trajectory).Materials and methods. The study includes a review of bibliographic sources on the application of neural networks in the field of education. It also includes modeling the structure of a neural network adaptive learning environment. The PyTorch library was used to programmatically implement the neural network.Results. The prospects for the use of neural networks in the field of education are considered, including various tasks of recognition, diagnostics, classification, clustering, forecasting, optimization, etc. A structural model of an adaptive learning NeuroSmart environment is created. This environment includes a number of subsystems for solving the following tasks: biometric identification of the student; determining starting level; choosing an adaptive learning path; determining the current level of competency development; automated verification of students’ work using recognition technology; analysis of the final result of training; monitoring information security incidents in an electronic course, etc. In order to study the possibilities and problems of applying neural network models to the tasks of constructing student adaptive learning trajectories, a model example of a neural network was created. This example illustrates the possibility of using a neural network to select further nodes of the educational trajectory based on the available data on the current learning parameters in an electronic course. To implement the neural network, the PyTorch deep learning library and Pandas library modules were used. SGD, Adam, Rprop were used as an optimizer to perform gradient descent steps. For each of the optimizers, a computer study of the stability of the neural network was carried out by varying the following parameters: the learning rate coefficient, the number of neurons in the hidden layer, and the number of training epochs.Conclusion. It can be assumed that the next stage in the evolution of the use of neural network technologies in the field of education will be their integration into complex multi-component Smart systems capable of automatically supporting student learning at all stages of the implementation of their personal educational trajectory. Obviously, the practical implementation of complex neural network systems of this level is a very difficult task and can be solved so far only at the level of individual subsystems. There are a number of problems associated with computer simulation of the educational environment based on neural network models: the question of the optimal structure of the neural network (for example, the number of neurons and layers in the network) has not been studied enough, there are no clear criteria for the optimality of the adaptive educational trajectory. Nevertheless, it should be noted that the task of developing new forms and technologies of personalized e-learning is in great demand, which makes modeling based on neural networks especially relevant.

Optimal control of PDEs using physics-informed neural networks

Physics-informed neural networks (PINNs) have recently become a popular method for solving forward and inverse problems governed by partial differential equations (PDEs). By incorporating the residual of the PDE into the loss function of a neural network-based surrogate model for the unknown state, PINNs can seamlessly blend measurement data with physical constraints. Here, we extend this framework to PDE-constrained optimal control problems, for which the governing PDE is fully known and the goal is to find a control variable that minimizes a desired cost objective. We provide a set of guidelines for obtaining a good optimal control solution; first by selecting an appropriate PINN architecture and training parameters based on a forward problem, second by choosing the best value for a critical scalar weight in the loss function using a simple but effective two-step line search strategy. We then validate the performance of the PINN framework by comparing it to adjoint-based nonlinear optimal control, which performs gradient descent on the discretized control variable while satisfying the discretized PDE. This comparison is carried out on several distributed control examples based on the Laplace, Burgers, Kuramoto-Sivashinsky, and Navier-Stokes equations. Finally, we discuss the advantages and caveats of using the PINN and adjoint-based approaches for solving optimal control problems constrained by nonlinear PDEs.

Optimal Control of Pdes Using Physics-Informed Neural Networks

Physics-informed neural networks (PINNs) have recently become a popular method for solving forward and inverse problems governed by partial differential equations (PDEs). By incorporating the residual of the PDE into the loss function of a neural network-based surrogate model for the unknown state, PINNs can seamlessly blend measurement data with physical constraints. Here, we extend this framework to PDE-constrained optimal control problems, for which the governing PDE is fully known and the goal is to find a control variable that minimizes a desired cost objective. We provide a set of guidelines for obtaining a good optimal control solution; first by ensuring that the PDE remains well satisfied during the training process, second by assessing rigorously the quality of the computed optimal control. We then validate the performance of the PINN framework by comparing it to adjoint-based nonlinear optimal control, which performs gradient descent on the discretized control variable while satisfying the discretized PDE. This comparison is carried out on several distributed control examples based on the Laplace, Burgers, Kuramoto-Sivashinsky, and Navier-Stokes equations. Finally, we discuss the advantages and caveats of using the PINN and adjoint-based approaches for solving optimal control problems constrained by nonlinear PDEs.

Memorized sparse backpropagation

Neural network learning is usually time-consuming since backpropagation needs to compute full gradients and backpropagate them across multiple layers. Despite its success of existing works in accelerating propagation through sparseness, the relevant theoretical characteristics remain under-researched and empirical studies found that they suffer from the loss of information contained in unpropagated gradients. To tackle these problems, this paper presents a unified sparse backpropagation framework and provides a detailed analysis of its theoretical characteristics. Analysis reveals that when applied to a multilayer perceptron, our framework essentially performs gradient descent using an estimated gradient similar enough to the true gradient, resulting in convergence in probability under certain conditions. Furthermore, a simple yet effective algorithm named memorized sparse backpropagation (MSBP) is proposed to remedy the problem of information loss by storing unpropagated gradients in memory for learning in the next steps. Experimental results demonstrate that the proposed MSBP is effective to alleviate the information loss in traditional sparse backpropagation while achieving comparable acceleration.

Quantum gradient descent for linear systems and least squares

Quantum machine learning and optimization are exciting new areas that have been brought forward by the breakthrough quantum algorithm of Harrow, Hassidim and Lloyd for solving systems of linear equations. The utility of {classical} linear system solvers extends beyond linear algebra as they can be leveraged to solve optimization problems using iterative methods like gradient descent. In this work, we provide the first quantum method for performing gradient descent when the gradient is an affine function. Performing $\tau$ steps of the gradient descent requires time $O(\tau C_S)$ for weighted least squares problems, where $C_S$ is the cost of performing one step of the gradient descent quantumly, which at times can be considerably smaller than the classical cost. We illustrate our method by providing two applications: first, for solving positive semidefinite linear systems, and, second, for performing stochastic gradient descent for the weighted least squares problem with reduced quantum memory requirements. We also provide a quantum linear system solver in the QRAM data structure model that provides significant savings in cost for large families of matrices.

Gradient Descent with Identity Initialization Efficiently Learns Positive-Definite Linear Transformations by Deep Residual Networks.

We analyze algorithms for approximating a function mapping to using deep linear neural networks, that is, that learn a function parameterized by matrices and defined by . We focus on algorithms that learn through gradient descent on the population quadratic loss in the case that the distribution over the inputs is isotropic. We provide polynomial bounds on the number of iterations for gradient descent to approximate the least-squares matrix , in the case where the initial hypothesis has excess loss bounded by a small enough constant. We also show that gradient descent fails to converge for whose distance from the identity is a larger constant, and we show that some forms of regularization toward the identity in each layer do not help. If is symmetric positive definite, we show that an algorithm that initializes learns an -approximation of using a number of updates polynomial in , the condition number of , and . In contrast, we show that if the least-squares matrix is symmetric and has a negative eigenvalue, then all members of a class of algorithms that perform gradient descent with identity initialization, and optionally regularize toward the identity in each layer, fail to converge. We analyze an algorithm for the case that satisfies for all but may not be symmetric. This algorithm uses two regularizers: one that maintains the invariant for all and the other that "balances" so that they have the same singular values.

Reference-shaping adaptive control by using gradient descent optimizers.

This study presents a model reference adaptive control scheme based on reference-shaping approach. The proposed adaptive control structure includes two optimizer processes that perform gradient descent optimization. The first process is the control optimizer that generates appropriate control signal for tracking of the controlled system output to a reference model output. The second process is the adaptation optimizer that performs for estimation of a time-varying adaptation gain, and it contributes to improvement of control signal generation. Numerical update equations derived for adaptation gain and control signal perform gradient descent optimization in order to decrease the model mismatch errors. To reduce noise sensitivity of the system, a dead zone rule is applied to the adaptation process. Simulation examples show the performance of the proposed Reference-Shaping Adaptive Control (RSAC) method for several test scenarios. An experimental study demonstrates application of method for rotor control.

Computer aided restoration of handwritten character strokes

This work suggests a new variational approach to the task of computer aided segmentation and restoration of incomplete characters, residing in a highly noisy document image. We model character strokes as the movement of a pen with a varying radius. Following this model, in order to fit the digital image, a cubic spline representation is being utilized to perform gradient descent steps, while maintaining interpolation at some initial (manually sampled) points. The proposed algorithm was used in the process of restoring approximately 1000 ancient Hebrew characters (dating to ca. 8th–7th century BCE), some of which are presented herein and show that the algorithm yields plausible results when applied on deteriorated documents.

Optimal Control of Conditional Value-at-Risk in Continuous Time

We consider continuous-time stochastic optimal control problems featuring conditional value-at-risk (CVaR) in the objective. The major difficulty in these problems arises from time inconsistency, which prevents us from directly using dynamic programming. To resolve this challenge, we convert to an equivalent bilevel optimization problem in which the inner optimization problem is standard stochastic control. Furthermore, we provide conditions under which the outer objective function is convex and differentiable. We compute the outer objective's value via a Hamilton--Jacobi--Bellman equation and its gradient via the viscosity solution of a linear parabolic equation, which allows us to perform gradient descent. The significance of this result is that we provide an efficient dynamic-programming-based algorithm for optimal control of CVaR without lifting the state space. To broaden the applicability of the proposed algorithm, we propose convergent approximation schemes in cases where our key assumptions do not hold and characterize relevant suboptimality bounds. In addition, we extend our method to a more general class of risk metrics, which includes mean variance and median deviation. We also demonstrate a concrete application to portfolio optimization under CVaR constraints. Our results contribute an efficient framework for solving time-inconsistent CVaR-based sequential optimization.

Concurrent Monte Carlo transport and fluence optimization with fluence adjusting scalable transport Monte Carlo.

The future of radiation therapy will require advanced inverse planning solutions to support single-arc, multiple-arc, and "4π" delivery modes, which present unique challenges in finding an optimal treatment plan over a vast search space, while still preserving dosimetric accuracy. The successful clinical implementation of such methods would benefit from Monte Carlo (MC) based dose calculation methods, which can offer improvements in dosimetric accuracy when compared to deterministic methods. The standard method for MC based treatment planning optimization leverages the accuracy of the MC dose calculation and efficiency of well-developed optimization methods, by precalculating the fluence to dose relationship within a patient with MC methods and subsequently optimizing the fluence weights. However, the sequential nature of this implementation is computationally time consuming and memory intensive. Methods to reduce the overhead of the MC precalculation have been explored in the past, demonstrating promising reductions of computational time overhead, but with limited impact on the memory overhead due to the sequential nature of the dose calculation and fluence optimization. The authors propose an entirely new form of "concurrent" Monte Carlo treat plan optimization: a platform which optimizes the fluence during the dose calculation, reduces wasted computation time being spent on beamlets that weakly contribute to the final dose distribution, and requires only a low memory footprint to function. In this initial investigation, the authors explore the key theoretical and practical considerations of optimizing fluence in such a manner. The authors present a novel derivation and implementation of a gradient descent algorithm that allows for optimization during MC particle transport, based on highly stochastic information generated through particle transport of very few histories. A gradient rescaling and renormalization algorithm, and the concept of momentum from stochastic gradient descent were used to address obstacles unique to performing gradient descent fluence optimization during MC particle transport. The authors have applied their method to two simple geometrical phantoms, and one clinical patient geometry to examine the capability of this platform to generate conformal plans as well as assess its computational scaling and efficiency, respectively. The authors obtain a reduction of at least 50% in total histories transported in their investigation compared to a theoretical unweighted beamlet calculation and subsequent fluence optimization method, and observe a roughly fixed optimization time overhead consisting of ∼10% of the total computation time in all cases. Finally, the authors demonstrate a negligible increase in memory overhead of ∼7-8 MB to allow for optimization of a clinical patient geometry surrounded by 36 beams using their platform. This study demonstrates a fluence optimization approach, which could significantly improve the development of next generation radiation therapy solutions while incurring minimal additional computational overhead.