Prediction Of Molecular Properties Research Articles

Evaluating uncertainty-based active learning for accelerating the generalization of molecular property prediction

Deep learning models have proven to be a powerful tool for the prediction of molecular properties for applications including drug design and the development of energy storage materials. However, in order to learn accurate and robust structure–property mappings, these models require large amounts of data which can be a challenge to collect given the time and resource-intensive nature of experimental material characterization efforts. Additionally, such models fail to generalize to new types of molecular structures that were not included in the model training data. The acceleration of material development through uncertainty-guided experimental design has the promise to significantly reduce the data requirements and enable faster generalization to new types of materials. To evaluate the potential of such approaches for electrolyte design applications, we perform comprehensive evaluation of existing uncertainty quantification methods on the prediction of two relevant molecular properties - aqueous solubility and redox potential. We develop novel evaluation methods to probe the utility of the uncertainty estimates for both in-domain and out-of-domain data sets. Finally, we leverage selected uncertainty estimation methods for active learning to evaluate their capacity to support experimental design.

Insights into deep learning framework for molecular property prediction based on different tokenization algorithms

With the rapid development of deep learning, research on quantitative structure–property relationships based on deep learning has received widespread attention. The deep learning architecture combining Bidirectional Encoder Representation from Transformers (BERT) and Feedforward Neural Networks (FNN) is proposed to compare the performance of different tokenization algorithms. And t-distributed stochastic neighbor embedding reveals valuable information about the mechanism of structure–property relationships. Additionally, a deep learning framework, BERT-Convolutional Neural Network (CNN)-FNN, is developed based on the optimal tokenization algorithm to accurately predict the σ-profile and VCOSMO. The molecular structures are vectorized with the BERT model capturing local and global features of the entire molecule. And the CNN model enhances the latent representation associated with molecular properties, while the FNN model establishes the correlation. The deep learning frameworks predict σ-profile and VCOSMO properties with R2 greater than 0.9703, making it a promising intelligent tool for guiding solvent design and screening.

MolCAP: Molecular Chemical reActivity Pretraining and prompted-finetuning enhanced molecular representation learning

Molecular representation learning (MRL) is a fundamental task for drug discovery. However, previous deep-learning (DL) methods focus excessively on learning robust inner-molecular representations by mask-dominated pretraining frameworks, neglecting abundant chemical reactivity molecular relationships that have been demonstrated as the determining factor for various molecular property prediction tasks. Here, we present MolCAP to promote MRL, a graph-pretraining Transformer based on chemical reactivity (IMR) knowledge with prompted finetuning. Results show that MolCAP outperforms comparative methods based on traditional molecular pretraining frameworks, in 13 publicly available molecular datasets across a diversity of biomedical tasks. Prompted by MolCAP, even basic graph neural networks are capable of achieving surprising performance that outperforms previous models, indicating the promising prospect of applying reactivity information to MRL. In addition, manually designed molecular templets are potential to uncover the dataset bias. All in all, we expect our MolCAP to gain more chemical meaningful insights for the entire process of drug discovery.

T-MGCL: Molecule Graph Contrastive Learning Based on Transformer for Molecular Property Prediction.

In recent years, machine learning has gained increasing traction in the study of molecules, enabling researchers to tackle challenging tasks including molecular property prediction and drug design.Consequently, there remains an open challenge to develop a neural network architecture that can make use of extensive amounts of unlabeled data for training while still providing competitive results in various molecular property prediction tasks. To address this challenge, we propose a Molecule Graph Contrastive Learning approach based on the Transformer framework (T-MGCL). Our approach involves expanding numerous unsupervised molecular graphs and using a contrast estimator to ensure consistency among various graph expansions of the same molecule. Transformer framework is employed to consider the distance between atoms and molecular graph attributes, thereby accounting for structural information that may be overlooked by traditional graph neural networks. Our experimental results demonstrate that the T-MGCL model outperforms other models in several molecular property prediction tasks. Additionally, we observe that the attention weight learned by T-MGCL can be easily explained from a chemical perspective.

Adaptive Transfer of Graph Neural Networks for Few-Shot Molecular Property Prediction.

Few-Shot Molecular Property Prediction (FSMPP) is an improtant task on drug discovery, which aims to learn transferable knowledge from base property prediction tasks with sufficient data for predicting novel properties with few labeled molecules. Its key challenge is how to alleviate the data scarcity issue of novel properties. Pretrained Graph Neural Network (GNN) based FSMPP methods effectively address the challenge by pre-training a GNN from large-scale self-supervised tasks and then finetuning it on base property prediction tasks to perform novel property prediction. However, in this paper, we find that the GNN finetuning step is not always effective, which even degrades the performance of pretrained GNN on some novel properties. This is because these molecule-property relationships among molecules change across different properties, which results in the finetuned GNN overfits to base properties and harms the transferability performance of pretrained GNN on novel properties. To address this issue, in this paper, we propose a novel Adaptive Transfer framework of GNN for FSMPP, called ATGNN, which transfers the knowledge of pretrained and finetuned GNNs in a task-adaptive manner to adapt novel properties. Specifically, we first regard the pretrained and finetuned GNNs as model priors of target-property GNN. Then, a task-adaptive weight prediction network is designed to leverage these priors to predict target GNN weights for novel properties. Finally, we combine our ATGNN framework with existing FSMPP methods for FSMPP. Extensive experiments on four real-world datasets, i.e., Tox21, SIDER, MUV, and ToxCast, show the effectiveness of our ATGNN framework.

Explainable AI in drug discovery: self-interpretable graph neural network for molecular property prediction using concept whitening

Molecular property prediction is a fundamental task in the field of drug discovery. Several works use graph neural networks to leverage molecular graph representations. Although they have been successfully applied in a variety of applications, their decision process is not transparent. In this work, we adapt concept whitening to graph neural networks. This approach is an explainability method used to build an inherently interpretable model, which allows identifying the concepts and consequently the structural parts of the molecules that are relevant for the output predictions. We test popular models on several benchmark datasets from MoleculeNet. Starting from previous work, we identify the most significant molecular properties to be used as concepts to perform classification. We show that the addition of concept whitening layers brings an improvement in both classification performance and interpretability. Finally, we provide several structural and conceptual explanations for the predictions.

Prediction of Human Pharmacokinetics From Chemical Structure: Combining Mechanistic Modeling with Machine Learning

Pharmacokinetics (PK) is the result of a complex interplay between compound properties and physiology, and a detailed characterization of a molecule's PK during preclinical research is key to understanding the relationship between applied dose, exposure, and pharmacological effect. Predictions of human PK based on the chemical structure of a compound are highly desirable to avoid advancing compounds with unfavorable properties early on and to reduce animal testing, but data to train such models are scarce. To address this problem, we combine well-established Physiologically Based Pharmacokinetic models with Deep Learning models for molecular property prediction into a hybrid model to predict PK parameters for small molecules directly from chemical structure. Our model predicts exposure after oral and intravenous administration with fold change errors of 1.87 and 1.86, respectively, in healthy subjects and 2.32 and 2.23, respectively, in patients with various diseases. Unlike pure Deep Learning models, the hybrid model can predict endpoints on which it was not trained. We validate this extrapolation capability by predicting full concentration-time profiles for compounds with published PK data. Our model enables early selection and prioritization of the most promising drug candidates, which can lead to a reduction in animal testing during drug discovery and development.

A pre-trained multi-representation fusion network for molecular property prediction

In the field of machine learning and cheminformatics, the prediction of molecular properties holds significant importance. Molecules can be represented in various formats, including 1D SMILES string, 2D graph, and 3D conformation. Numerous models have been proposed for different representations to accomplish molecular property prediction. However, most recent works have focused on one or two representations or combining embedding vectors from different perspectives in an unsophisticated manner. To address this issue, we present PremuNet, a novel pre-trained multi-representation fusion network for molecular property prediction. PremuNet can extract comprehensive molecular information from multiple views and combine them interactively through pre-training and fine-tuning. The framework of PremuNet consists of two branches: a Transformer-GNN branch that extracts SMILES and graph information, and a Fusion Net branch that extracts topology and geometry information, called PremuNet-L and PremuNet-H respectively. We employ masked self-supervised methods to enable the model to learn information fusion and achieve enhanced performance in downstream tasks. The proposed model has been evaluated on eight molecular property prediction tasks, including five classification and three regression tasks, and attained state-of-the-art performance in most cases. Additionally, we conduct the ablation studies to demonstrate the effect of each view and the branch combination approaches.

On Degree-Based Topological Indices of Kagome Graphene, and Carbon Kagome Nanotubes and Nanotori

Approaches based on graph descriptors have been used in cheminformatics and bioinformatics for molecular property prediction. In this article, we apply partition technique and simple counting schemes to describe the structure of a two-dimensional (2D) kagome graphene and then derive the closed form of various degree-based topological indices (graph descriptor) of 2D kagome graphene, and its tubular and toroidal forms.

Molecular geometric deep learning

Molecular representation learning plays an important role in molecular property prediction. Existing molecular property prediction models rely on the de facto standard of covalent-bond-based molecular graphs for representing molecular topology at the atomic level and totally ignore the non-covalent interactions within the molecule. In this study, we propose a molecular geometric deep learning model to predict the properties of molecules that aims to comprehensively consider the information of covalent and non-covalent interactions of molecules. The essential idea is to incorporate a more general molecular representation into geometric deep learning (GDL) models. We systematically test molecular GDL (Mol-GDL) on fourteen commonly used benchmark datasets. The results show that Mol-GDL can achieve a better performance than state-of-the-art (SOTA) methods. Extensive tests have demonstrated the important role of non-covalent interactions in molecular property prediction and the effectiveness of Mol-GDL models.

Automated identification of isofragmented reactions and application in correcting molecular property models

Machine learning techniques are increasingly being employed to predict molecular properties. Such models are often trained on large computationally derived datasets, and are only as accurate as the underlying data. We exploit the well-known error cancelling effect of isodesmic and homodesmotic reactions to develop a multi-fidelity data-driven molecular property prediction method. First, we propose an optimization-based scheme to quickly and automatically identify all isofragmented reactions for a target molecule u, i.e., balanced reactions involving u and one or more molecules from a given set of molecules (M), which conserve a predefined set of fragments of arbitrary size. Second, we show that such isofragmented reactions can be leveraged to improve the predictive accuracy of a data-driven model by infusing a small high-accuracy dataset comprising molecules in M. We applied this method with a high-accuracy subset of the NIST thermochemistry database and a simple additive data-driven model trained on a QM9 subset. Our results show that the heats of formation using our method were ∼4.4 kcal/mol more accurate on average than the data-driven model.

A systematic study of key elements underlying molecular property prediction

Artificial intelligence (AI) has been widely applied in drug discovery with a major task as molecular property prediction. Despite booming techniques in molecular representation learning, key elements underlying molecular property prediction remain largely unexplored, which impedes further advancements in this field. Herein, we conduct an extensive evaluation of representative models using various representations on the MoleculeNet datasets, a suite of opioids-related datasets and two additional activity datasets from the literature. To investigate the predictive power in low-data and high-data space, a series of descriptors datasets of varying sizes are also assembled to evaluate the models. In total, we have trained 62,820 models, including 50,220 models on fixed representations, 4200 models on SMILES sequences and 8400 models on molecular graphs. Based on extensive experimentation and rigorous comparison, we show that representation learning models exhibit limited performance in molecular property prediction in most datasets. Besides, multiple key elements underlying molecular property prediction can affect the evaluation results. Furthermore, we show that activity cliffs can significantly impact model prediction. Finally, we explore into potential causes why representation learning models can fail and show that dataset size is essential for representation learning models to excel.

Molecular properties prediction, anticancer and anti-inflammatory activities of some pyrimido[1,2-b]pyridazin-2-one derivatives.

The anticancer and anti-inflammatory activities of a novel series of eleven pyrimido[1,2-b]pyridazin-2-one analogues substituted at position 7 were assessed in the current study. The physicochemical characteristics were studied using MolSoft software. The antiproliferative activity was investigated by MTT cell viability assay, and cell cycle analysis elucidated the antiproliferative mechanism of action. Western blot analysis examined the expression levels of key pro-apoptotic (Bax, p53) and pro-survival (Bcl-2) proteins. The anti-inflammatory activity was assessed by measuring the production levels of nitric oxide in RAW264.7 cells, and the expression levels of COX-2 enzyme in LPS-activated THP-1 cells. In addition, the gene expression of various pro-inflammatory cytokines (IL-6, IL-8, IL-1β, TNF-α) and chemokines (CCL2, CXCL1, CXCL2, CXCL3) was assessed by RT-qPCR. Compound 1 bearing a chlorine substituent displayed the highest cytotoxic activity against HCT-116 and MCF-7 cancer cells where IC50 values of 49.35 ± 2.685 and 69.32 ± 3.186 µM, respectively, were achieved. Compound 1 increased the expression of pro-apoptotic proteins p53 and Bax while reducing the expression of pro-survival protein Bcl-2. Cell cycle analysis revealed that compound 1 arrested cell cycle at the G0/G1 phase. Anti-inflammatory assessments revealed that compound 1 displayed the strongest inhibitory activity on NO production with IC50 of 29.94 ± 2.24 µM, and down-regulated the expression of COX-2. Compound 1 also induced a statistically significant decrease in the gene expression of various cytokines and chemokines. These findings showed that the pyrimidine derivative 1 displayed potent anti-inflammatory and anticancer properties in vitro, and can be selected as a lead compound for further investigation.

GeoT: A Geometry-Aware Transformer for Reliable Molecular Property Prediction and Chemically Interpretable Representation Learning.

In recent years, molecular representation learning has emerged as a key area of focus in various chemical tasks. However, many existing models fail to fully consider the geometric information on molecular structures, resulting in less intuitive representations. Moreover, the widely used message passing mechanism is limited to providing the interpretation of experimental results from a chemical perspective. To address these challenges, we introduce a novel transformer-based framework for molecular representation learning, named the geometry-aware transformer (GeoT). The GeoT learns molecular graph structures through attention-based mechanisms specifically designed to offer reliable interpretability as well as molecular property prediction. Consequently, the GeoT can generate attention maps of the interatomic relationships associated with training objectives. In addition, the GeoT demonstrates performance comparable to that of MPNN-based models while achieving reduced computational complexity. Our comprehensive experiments, including an empirical simulation, reveal that the GeoT effectively learns chemical insights into molecular structures, bridging the gap between artificial intelligence and molecular sciences.

Evidential Meta-model for Molecular Property Prediction.

The usefulness of supervised molecular property prediction (MPP) is well-recognized in many applications. However, the insufficiency and the imbalance of labeled data make the learning problem difficult. Moreover, the reliability of the predictions is also a huddle in the deployment of MPP models in safety-critical fields. We propose the Evidential Meta-model for Molecular Property Prediction (EM3P2) method that returns uncertainty estimates along with its predictions. Our EM3P2 trains an evidential graph isomorphism network classifier using multi-task molecular property datasets under the model-agnostic meta-learning (MAML) framework while addressing the problem of data imbalance. Our results showed better prediction performances compared to existing meta-MPP models. Furthermore, we showed that the uncertainty estimates returned by our EM3P2 can be used to reject uncertain predictions for applications that require higher confidence. Source code available for download at https://github.com/Ajou-DILab/EM3P2. Supplementary Materials are available at Bioinformatics.

Recent Applications of Machine Learning in Molecular Property and Chemical Reaction Outcome Predictions.

Burgeoning developments in machine learning (ML) and its rapidly growing adaptations in chemistry are noteworthy. Motivated by the successful deployments of ML in the realm of molecular property prediction (MPP) and chemical reaction prediction (CRP), herein we highlight some of its most recent applications in predictive chemistry. We present a nonmathematical and concise overview of the progression of ML implementations, ranging from an ensemble-based random forest model to advanced graph neural network algorithms. Similarly, the prospects of various feature engineering and feature learning approaches that work in conjunction with ML models are described. Highly accurate predictions reported in MPP tasks (e.g., lipophilicity, solubility, distribution coefficient), using methods such as D-MPNN, MolCLR, SMILES-BERT, and MolBERT, offer promising avenues in molecular design and drug discovery. Whereas MPP pertains to a given molecule, ML applications in chemical reactions present a different level of challenge, primarily arising from the simultaneous involvement of multiple molecules and their diverse roles in a reaction setting. The reported RMSEs in MPP tasks range from 0.287 to 2.20, while those for yield predictions are well over 4.9 in the lower end, reaching thresholds of >10.0 in several examples. Our Review concludes with a set of persisting challenges in dealing with reaction data sets and an overall optimistic outlook on benefits of ML-driven workflows for various MPP as well as CRP tasks.

Chemical structure-aware molecular image representation learning.

Current methods of molecular image-based drug discovery face two major challenges: (1) work effectively in absence of labels, and (2) capture chemical structure from implicitly encoded images. Given that chemical structures are explicitly encoded by molecular graphs (such as nitrogen, benzene rings and double bonds), we leverage self-supervised contrastive learning to transfer chemical knowledge from graphs to images. Specifically, we propose a novel Contrastive Graph-Image Pre-training (CGIP) framework for molecular representation learning, which learns explicit information in graphs and implicit information in images from large-scale unlabeled molecules via carefully designed intra- and inter-modal contrastive learning. We evaluate the performance of CGIP on multiple experimental settings (molecular property prediction, cross-modal retrieval and distribution similarity), and the results show that CGIP can achieve state-of-the-art performance on all 12 benchmark datasets and demonstrate that CGIP transfers chemical knowledge in graphs to molecular images, enabling image encoder to perceive chemical structures in images. We hope this simple and effective framework will inspire people to think about the value of image for molecular representation learning.

FG-BERT: a generalized and self-supervised functional group-based molecular representation learning framework for properties prediction.

Artificial intelligence-based molecular property prediction plays a key role in molecular design such as bioactive molecules and functional materials. In this study, we propose a self-supervised pretraining deep learning (DL) framework, called functional group bidirectional encoder representations from transformers (FG-BERT), pertained based on ~1.45 million unlabeled drug-like molecules, to learn meaningful representation of molecules from function groups. The pretrained FG-BERT framework can be fine-tuned to predict molecular properties. Compared to state-of-the-art (SOTA) machine learning and DL methods, we demonstrate the high performance of FG-BERT in evaluating molecular properties in tasks involving physical chemistry, biophysics and physiology across 44 benchmark datasets. In addition, FG-BERT utilizes attention mechanisms to focus on FG features that are critical to the target properties, thereby providing excellent interpretability for downstream training tasks. Collectively, FG-BERT does not require any artificially crafted features as input and has excellent interpretability, providing an out-of-the-box framework for developing SOTA models for a variety of molecule (especially for drug) discovery tasks.

Extended study on atomic featurization in graph neural networks for molecular property prediction

Graph neural networks have recently become a standard method for analyzing chemical compounds. In the field of molecular property prediction, the emphasis is now on designing new model architectures, and the importance of atom featurization is oftentimes belittled. When contrasting two graph neural networks, the use of different representations possibly leads to incorrect attribution of the results solely to the network architecture. To better understand this issue, we compare multiple atom representations by evaluating them on the prediction of free energy, solubility, and metabolic stability using graph convolutional networks. We discover that the choice of atom representation has a significant impact on model performance and that the optimal subset of features is task-specific. Additional experiments involving more sophisticated architectures, including graph transformers, support these findings. Moreover, we demonstrate that some commonly used atom features, such as the number of neighbors or the number of hydrogens, can be easily predicted using only information about bonds and atom type, yet their explicit inclusion in the representation has a positive impact on model performance. Finally, we explain the predictions of the best-performing models to better understand how they utilize the available atomic features.

Few-shot learning with transformers via graph embeddings for molecular property prediction

Few-shot learning with transformers via graph embeddings for molecular property prediction