Abstract

We initiate the study of quantum algorithms for escaping from saddle points with provable guarantee. Given a functionf:Rn→R, our quantum algorithm outputs anϵ-approximate second-order stationary point usingO~(log2⁡(n)/ϵ1.75)queries to the quantum evaluation oracle (i.e., the zeroth-order oracle). Compared to the classical state-of-the-art algorithm by Jin et al. withO~(log6⁡(n)/ϵ1.75)queries to the gradient oracle (i.e., the first-order oracle), our quantum algorithm is polynomially better in terms oflog⁡nand matches its complexity in terms of1/ϵ. Technically, our main contribution is the idea of replacing the classical perturbations in gradient descent methods by simulating quantum wave equations, which constitutes the improvement in the quantum query complexity withlog⁡nfactors for escaping from saddle points. We also show how to use a quantum gradient computation algorithm due to Jordan to replace the classical gradient queries by quantum evaluation queries with the same complexity. Finally, we also perform numerical experiments that support our theoretical findings.

Highlights

  • Nonconvex optimization is a central research topic in optimization theory, mainly because the loss functions in many machine learning models are typically nonconvex

  • Our quantum algorithm is built upon perturbed gradient descent (PGD) and perturbed accelerated gradient descent (PAGD) and shares their simplicity of being single-loop, but we propose two main modifications

  • Can we give quantum-inspired classical algorithms for escaping from saddle points? Our work suggests that compared to uniform perturbation, there exist physics-motivated methods to better exploit the randomness in gradient descent

Read more

Summary

Introduction

Nonconvex optimization is a central research topic in optimization theory, mainly because the loss functions in many machine learning models (including neural networks) are typically nonconvex. Achieved to be almost dimension-free with complexity O(log4(n)/ 2), and the state-of-theart result takes O(log6(n)/ 1.75) queries [48] These results suffer from a significant overhead in terms of log n, and it has been an open question to keep both the merits of using only the first-order oracle as well as being close to dimension-free [49]. We explore quantum algorithms for escaping from saddle points. This is a mutual generalization of both classical and quantum algorithms for optimization:. Quantum tunneling is a phenomenon in quantum mechanics where the wave function of a quantum particle can tunnel through a potential barrier and appear on the other side with significant probability This very much resembles escaping from poor landscapes in nonconvex optimization. Quantum algorithms motivated by quantum tunneling will be essentially different from those motivated by the Grover search [42], and will demonstrate significant novelty if the quantum speedup compared to the classical counterparts is more than quadratic

Contributions
Related Work
Open Questions
Organization
Escape from Saddle Points by Quantum Simulation
Quantum Simulation of the Schrödinger Equation
Quantum Query Complexity of Simulating the Schrödinger Equation
Perturbed Gradient Descent with Quantum Simulation
Effectiveness of the Perturbation by Quantum Simulation
Proof of Our Quantum Speedup
Perturbed Accelerated Gradient Descent with Quantum Simulation
Gradient Descent by the Quantum Evaluation Oracle
Error Bounds of Gradient Computation Steps
Escaping from Saddle Points with Quantum Simulation and Gradient Computation
Numerical Experiments
Dispersion of the Wave Packet
Quantum Simulation on Non-quadratic Potential Fields
Comparison Between PGD and Algorithm 2
Dimension Dependence
Schrödinger Equation with a Quadratic Potential
Bounding the deviation from perfect Gaussian in quantum evolution
Variance of Gaussian Wave Packets
Findings
Existing Lemmas
Full Text
Published version (Free)

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call