Hybrid Ising Machine and Reinforcement Learning Approach to Folding Lattice Proteins

Abstract

The hydrophobic polarity (HP) protein folding model is a simplified computation model utilized to study the folding of proteins into their three-dimensional structures. Understanding how proteins fold and misfold is fundamental to designing effective, safe drugs, allowing for targeted therapies and better patient outcomes. The amino acids of a protein can be categorized as either hydrophobic or polar, where the hydrophobic beads tend to cluster together in a properly folded protein to minimize exposure to the surrounding environment. Determining the lowest energy configurations of the HP model is a non-deterministic polynomial (NP) hard problem. One method is formulating the HP model as a quadratic unconstrained binary optimization (QUBO) problem which can be mapped to an Ising machine. In the Ising model, the lowest energy state of the system represents the correct lattice configuration of the folded protein. The Ising machine technique utilized of simulated annealing allows for future application of the work on an integrated silicon photonic chip. Alternatively, reinforcement learning designed with an energy-based reward function can optimize the HP model by adapting to the environment without the consequence of remaining in local minima like that of the Ising machine. A hybrid pipeline combining the Ising machine and reinforcement learning sequentially was designed to determine the two-dimensional lattice configurations of various protein chain lengths and sequences, combining the parallel optimization of the Ising machine on an integrated photonic platform and the adaptability of the reinforcement learning algorithm. Results found that this hybrid model improved the accuracy and ability of determining the lowest energy state in contrast to the Ising machine alone as well as improved the efficiency by reducing the number of iterations required when compared solely to reinforcement learning. Future work focuses on implementing a parallel pipeline approach as well as adapting to a three-dimensional model.