Abstract
Quantum hardware and quantum-inspired algorithms are becoming increasingly popular for combinatorial optimization. However, these algorithms may require careful hyperparameter tuning for each problem instance. We use a reinforcement learning agent in conjunction with a quantum-inspired algorithm to solve the Ising energy minimization problem, which is equivalent to the Maximum Cut problem. The agent controls the algorithm by tuning one of its parameters with the goal of improving recently seen solutions. We propose a new Rescaled Ranked Reward (R3) method that enables a stable single-player version of self-play training and helps the agent escape local optima. The training on any problem instance can be accelerated by applying transfer learning from an agent trained on randomly generated problems. Our approach allows sampling high quality solutions to the Ising problem with high probability and outperforms both baseline heuristics and a black-box hyperparameter optimization approach.
Highlights
Many important real-world combinatorial problems can be mapped to the Ising model, ranging from portfolio optimization (Venturelli and Kondratyev 2019, Marzec 2016) to protein folding (Perdomo-Ortiz et al 2012)
Ablation study We study the effect of the three main components of our approach: transfer learning from random problems, Rescaled Ranked Rewards (R3) scheme, and feature-wise linear modulation (FiLM) of the actor network with the problem features
In this work we proposed an reinforcement learning (RL)-based approach to tuning the regularization function of SimCIM, a quantum-inspired algorithm, to robustly solve the Ising problem
Summary
Many important real-world combinatorial problems can be mapped to the Ising model, ranging from portfolio optimization (Venturelli and Kondratyev 2019, Marzec 2016) to protein folding (Perdomo-Ortiz et al 2012). The Ising problem consists in finding binary strings that minimize the energy It is a quadratic unconstrained optimization task over the discrete {±1}n domain and equivalent to the Max-Cut problem from graph theory. The symmetric matrix J defines this pairwise interaction This problem is NP-hard (Barahona 1982) into two subsets, sauncdhitsheaqtutihvealseunmt tCo(tJh,ex)M=ax14-(cxuTtJpxr−ob∑lemij Joij)f dividing a set of n nodes of of edge weights connecting a weighted graph these subsets is maximized. In this interpretation, the problem matrix J is the adjacency matrix of the graph, and binary variables x denote the choice of the subset for each node. The optimization objective C(J, x) is called the cut value (higher is better); in this paper we use it to evaluate our algorithm and compare it to benchmarks
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
