Deep multi-agent fusion Q-Network for graph generation

Abstract

Deep Generative models for graph generation have made important advances in medicine, and particularly in drug research. Using graph representations of the molecules, these models proved efficient for predicting new molecular drug candidates with chemically valid structures. The main objective of the molecular graph generation task is to provide new molecular structures with the required chemical properties. Using a Reinforcement Learning (RL) agent to conduct the generation process while defining the reward function as the target chemical property, enables to generate new molecular drug candidates. However, sampling molecular candidates from the huge chemical space can be challenging, if only one single RL agent is deployed. Thus, we propose the Deep Fusion Q-Network (DFQN), a multi-agent RL-based Deep Neural Network for graph generation. We disclose that DFQN allows a better exploration of the environment, since multiple RL agents are exploited to generate candidate graph structures. Using a heterogeneous graph representation and an attention based fusion network, DFQN promotes coordination and communication between agents to jointly generate new graph structures with the required target properties. We investigate the optimal number of RL-agents using different degrees of communication in order to generate new graph structures under different property constraints. Moreover, we propose to conduct an adversarial training to incorporate prior knowledge of realistic drug molecules in the newly generated molecular graphs. We propose a new approach to check the chemical validity of the designed molecules in each iteration of the generation process. We evaluate the performance of our model for generating new molecules with different chemical properties and achieve remarkable results compared to state-of-the-art models for molecular graph generation.