Stacked Graph Transformer for HIV Molecular Prediction

Abstract

A Neural Network(NN) is a tool that is modeled after the human brain's neuron system and is used to learn from data and produce the required results. NNs have shown themselves to be highly effective in a variety of domains so far. Graph-structured data or non-euclidean data (non-grid/non-text) growth has, however, constrained the applications of traditional Neural Networks. Graph Neural Networks (GNNs) have recently entered the game, which can efficiently learn representations from Graph Structure. GNNs come in a variety of flavors and quickly gained popularity. Various approaches from other ML domains have been included in GNNs in an effort to improve their performance. One such approach is the Transformer architecture that has become well-known and famous in the NLP field thanks to its admirable attention mechanism. Recently, academics have attempted to use an amalgamation of Graph Neural Networks with Transformers on a variety of problem statements. In our work, for instance, we have tried to incorporate the same philosophy into the molecular field that has had fewer contributions with Transformers. We have narrowed our focus on HIV inhibitor classifications for this study. With a few more details covered further, this paper suggests a novel way to use a stacked transformer method that makes use of multiple transformer blocks to alleviate the limitations of traditional methodologies and achieve great performance. The proposed methods produce results that are comparable or slightly better in some cases to those achieved by current approaches.