Abstract
We initiate the study of quantum algorithms for escaping from saddle points with provable guarantee. Given a functionf:Rn→R, our quantum algorithm outputs anϵ-approximate second-order stationary point usingO~(log2(n)/ϵ1.75)queries to the quantum evaluation oracle (i.e., the zeroth-order oracle). Compared to the classical state-of-the-art algorithm by Jin et al. withO~(log6(n)/ϵ1.75)queries to the gradient oracle (i.e., the first-order oracle), our quantum algorithm is polynomially better in terms oflognand matches its complexity in terms of1/ϵ. Technically, our main contribution is the idea of replacing the classical perturbations in gradient descent methods by simulating quantum wave equations, which constitutes the improvement in the quantum query complexity withlognfactors for escaping from saddle points. We also show how to use a quantum gradient computation algorithm due to Jordan to replace the classical gradient queries by quantum evaluation queries with the same complexity. Finally, we also perform numerical experiments that support our theoretical findings.
Highlights
Nonconvex optimization is a central research topic in optimization theory, mainly because the loss functions in many machine learning models are typically nonconvex
Our quantum algorithm is built upon perturbed gradient descent (PGD) and perturbed accelerated gradient descent (PAGD) and shares their simplicity of being single-loop, but we propose two main modifications
Can we give quantum-inspired classical algorithms for escaping from saddle points? Our work suggests that compared to uniform perturbation, there exist physics-motivated methods to better exploit the randomness in gradient descent
Summary
Nonconvex optimization is a central research topic in optimization theory, mainly because the loss functions in many machine learning models (including neural networks) are typically nonconvex. Achieved to be almost dimension-free with complexity O(log4(n)/ 2), and the state-of-theart result takes O(log6(n)/ 1.75) queries [48] These results suffer from a significant overhead in terms of log n, and it has been an open question to keep both the merits of using only the first-order oracle as well as being close to dimension-free [49]. We explore quantum algorithms for escaping from saddle points. This is a mutual generalization of both classical and quantum algorithms for optimization:. Quantum tunneling is a phenomenon in quantum mechanics where the wave function of a quantum particle can tunnel through a potential barrier and appear on the other side with significant probability This very much resembles escaping from poor landscapes in nonconvex optimization. Quantum algorithms motivated by quantum tunneling will be essentially different from those motivated by the Grover search [42], and will demonstrate significant novelty if the quantum speedup compared to the classical counterparts is more than quadratic
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have