Applicability Domain Of Model Research Articles

Prediction of acute toxicity for Chlorella vulgaris caused by tire wear particle-derived compounds using quantitative structure-activity relationship models

Tire wear particles (TWPs) enter aquatic ecosystems through various pathways, such as rainwater and urban runoff. Additives in TWPs can harm aquatic organisms in these ecosystems. Therefore, it is essential to investigate their toxicity to aquatic organisms. In our study, we initially recorded the median effective concentrations of 21 TWP-derived compounds on Chlorella vulgaris growth, ranging from 0.04 to 8.60 mg/L. Subsequently, through an extensive review of the literature, we incorporated 112 compounds with specific toxicity endpoints to construct the QSAR model using genetic algorithm and multiple linear regression techniques, followed by the construction of the consensus model and the quantitative read-across structure-activity relationship (q-RASAR) model. Meanwhile, we employed rigorous internal and external validation measures to assess the performance of the model. The results indicated that the developed q-RASAR model exhibited strong adaptation, robustness, and reliable prediction, with q-RASAR indicators of Q2LOO = 0.7673, R2tr = 0.8079, R2test = 0.8610, Q2Fn = 0.8285−0.8614, and CCCtest = 0.9222. Based on an external dataset containing 128 emerging TWP-derived compounds, the model's applicability domain coverage was 90.6 %. The q-RASAR model predicted that the structure of diphenylamine was associated with higher toxicity, possibly liked to the SpMax2_Bhm and LogBCF descriptors. The established model reliably provides prediction and fills a critical data gap. These findings highlight the potential risks posed by emerging TWP-derived compounds to aquatic organisms.

Expanding Predictive Capacities in Toxicology: Insights from Hackathon-Enhanced Data and Model Aggregation.

In the realm of predictive toxicology for small molecules, the applicability domain of QSAR models is often limited by the coverage of the chemical space in the training set. Consequently, classical models fail to provide reliable predictions for wide classes of molecules. However, the emergence of innovative data collection methods such as intensive hackathons have promise to quickly expand the available chemical space for model construction. Combined with algorithmic refinement methods, these tools can address the challenges of toxicity prediction, enhancing both the robustness and applicability of the corresponding models. This study aimed to investigate the roles of gradient boosting and strategic data aggregation in enhancing the predictivity ability of models for the toxicity of small organic molecules. We focused on evaluating the impact of incorporating fragment features and expanding the chemical space, facilitated by a comprehensive dataset procured in an open hackathon. We used gradient boosting techniques, accounting for critical features such as the structural fragments or functional groups often associated with manifestations of toxicity.

Modeling of ionic liquids viscosity via advanced white-box machine learning.

Ionic liquids (ILs) are more widely used within the industry than ever before, and accurate models of their physicochemical characteristics are becoming increasingly important during the process optimization. It is especially challenging to simulate the viscosity of ILs since there is no widely agreed explanation of how viscosity is determined in liquids. In this research, genetic programming (GP) and group method of data handling (GMDH) models were used as white-box machine learning approaches to predict the viscosity of pure ILs. These methods were developed based on a large open literature database of 2813 experimental viscosity values from 45 various ILs at different pressures (0.06-298.9MPa) and temperatures (253.15-573K). The models were developed based on five, six, and seven inputs, and it was found that all the models with seven inputs provided more accurate results, while the models with five and six inputs had acceptable accuracy and simpler formulas. Based on GMDH and GP proposed approaches, the suggested GMDHmodel with seven inputs gave the most exact results with an average absolute relative deviation (AARD) of 8.14% and a coefficient of determination (R2) of 0.98. The proposed techniques were compared with theoretical and empirical models available in the literature, and it was displayed that the GMDH model with seven inputs strongly outperforms the existing approaches. The leverage statistical analysis revealed that most of the experimental data were located within the applicability domains of both GMDH and GP models and were of high quality. Trend analysis also illustrated that the GMDH and GP models could follow the expected trends of viscosity with variations in pressure and temperature. In addition, the relevancy factor portrayed that the temperature had the greatest impact on the ILs viscosity. The findings of this study illustrated that the proposed models represented strong alternatives to time-consuming and costly experimental methods of ILs viscosity measurement.

An integrated approach to predict activators of NRF2 - the transcription factor for oxidative stress response

A variety of environmental and physiological conditions can cause oxidative stress that damage cellular components such as DNA, proteins and lipids. Oxidative stress is implicated in many human diseases including cancer, cardiovascular diseases, neurological diseases, inflammatory diseases, and aging. The nuclear factor erythroid 2–related factor 2 (NRF2) is a transcriptional factor that plays a key role in the cellular antioxidant defense system as it regulates transcription of antioxidant proteins and detoxifying enzymes. There is an urgent need to identify novel compounds that activate NRF2 and enhance antioxidant defense. We collected data from the high-throughput screening of NRF2 activators and identified molecular fragments (structural alerts) associated with the activation of NRF2. We also developed ten classification models using different types of molecular descriptors and machine learning techniques. Two approaches were used to establish the applicability domain of developed models: the structure-based approach and the distance to model approach. The best performing model that used message passing neural network (MPNN) technique showed accuracy of 87 % for the test set of chemicals within the distance to model of 0.3. The integrative approach using a combination of generated structural alerts and MPNN model was used to screen approved drugs collected in the DrugBank to identify potential NRF2 activators. Out of 2393 screened chemicals 138 compounds were predicted as NRF2 activators by both approaches. Analysis of these compounds showed that some drugs were already known activators of NRF2 while others are potentially novel activators.

Going Beyond XAI: A Systematic Survey for Explanation-Guided Learning

As the societal impact of Deep Neural Networks (DNNs) grows, the goals for advancing DNNs become more complex and diverse, ranging from improving a conventional model accuracy metric to infusing advanced human virtues such as fairness, accountability, transparency, and unbiasedness. Recently, techniques in Explainable Artificial Intelligence (XAI) have been attracting considerable attention and have tremendously helped Machine Learning (ML) engineers in understand AI models. However, at the same time, we started to witness the emerging need beyond XAI among AI communities; based on the insights learned from XAI, how can we better empower ML engineers in steering their DNNs so that the model’s reasonableness and performance can be improved as intended? This article provides a timely and extensive literature overview of the field Explanation-Guided Learning (EGL), a domain of techniques that steer the DNNs’ reasoning process by adding regularization, supervision, or intervention on model explanations. In doing so, we first provide a formal definition of EGL and its general learning paradigm. Second, an overview of the key factors for EGL evaluation, as well as summarization and categorization of existing evaluation procedures and metrics for EGL are provided. Finally, the current and potential future application areas and directions of EGL are discussed, and an extensive experimental study is presented aiming at providing comprehensive comparative studies among existing EGL models in various popular application domains, such as Computer Vision and Natural Language Processing domains. Additional resources related to event prediction are included in the article website: https://kugaoyang.github.io/EGL/

Improving predictions of compound amenability for liquid chromatography-mass spectrometry to enhance non-targeted analysis.

Mass-spectrometry-based non-targeted analysis (NTA), in which mass spectrometric signals are assigned chemical identities based on a systematic collation of evidence, is a growing area of interest for toxicological risk assessment. Successful NTA results in better identification of potentially hazardous pollutants within the environment, facilitating the development of targeted analytical strategies to best characterize risks to human and ecological health. A supporting component of the NTA process involves assessing whether suspected chemicals are amenable to the mass spectrometric method, which is necessary in order to assign an observed signal to the chemical structure. Prior work from this group involved the development of a random forest model for predicting the amenability of 5517 unique chemical structures to liquid chromatography-mass spectrometry (LC-MS). This work improves the interpretability of the group's prior model of the same endpoint, as well as integrating 1348 more data points across negative and positive ionization modes. We enhance interpretability by feature engineering, a machine learning practice that reduces the input dimensionality while attempting to preserve performance statistics. We emphasize the importance of interpretable machine learning models within the context of building confidence in NTA identification. The novel data were curated by the labeling of compounds as amenable or unamenable by expert curators, resulting in an enhanced set of chemical compounds to expand the applicability domain of the prior model. The balanced accuracy benchmark of the newly developed model is comparable to performance previously reported (mean CV BA is 0.84 vs. 0.82 in positive mode, and 0.85 vs. 0.82 in negative mode), while on a novel external set, derived from this work's data, the Matthews correlation coefficients (MCC) for the novel models are 0.66 and 0.68 for positive and negative mode, respectively. Our group's prior published models scored MCC of 0.55 and 0.54 on the same external sets. This demonstrates appreciable improvement over the chemical space captured by the expanded dataset. This work forms part of our ongoing efforts to develop models with higher interpretability and higher performance to support NTA efforts.

Active Crowdsourcing for Multilabel Annotation.

Multilabel annotation is a critical step to generate training sets when learning classification models in various application domains, but asking domain experts to provide labels is usually time-consuming and expensive, which cannot meet the current requirement of the fast evolution of the models in the big data era. Although crowdsourcing provides a fast solution to acquire labels for multilabel learning, it faces the risk of high data acquisition cost and low label quality. This article proposes a novel one-coin label-dependent active crowdsourcing (OCLDAC) method to iteratively query noisy labels from crowd workers and learn multilabel classification models. In each iteration of active learning, integrated labels of instances are first inferred by a novel one-coin label-dependent model, which utilizes a mixture of multiple independent Bernoulli distributions to explore and exploit correlations among the labels to increase the accuracy of truth inference. Then, instances, labels, and workers are selected according to the novel strategies that incorporate the distribution of noisy labels, the prediction probability of learning models, label correlations, and the reliability of crowd workers. Simulations on eight multilabel datasets and evaluation on one real-world crowdsourcing dataset consistently show that the proposed OCLDAC significantly outperforms the state-of-the-art methods and their variants.

Evaluation and Optimization Methods for Applicability Domain Methods and Their Hyperparameters, Considering the Prediction Performance of Machine Learning Models.

In molecular, material, and process design and control, the applicability domain (AD) of a mathematical model y = f(x) between properties, activities, and features x is constructed. As there are multiple AD methods, each with its own set of hyperparameters, it is necessary to select an appropriate AD method and hyperparameters for each data set and mathematical model. However, there is no method for optimizing the AD model. This study proposes a method for evaluating and optimizing the AD model for each data set and a mathematical model. Using the predictions of double cross-validation with all samples, the relationship between coverage and root-mean-squared error (RMSE) was calculated for all combinations of AD methods and their hyperparameters, and the area under the coverage and RMSE curve (AUCR) was calculated. The AD model with the lowest AUCR value was selected as the optimal fit for the mathematical model. The proposed method was validated using eight data sets, including molecules, materials, and spectra, demonstrating that the proposed method could generate optimal AD models for all data sets. The Python code for the proposed method is available at https://github.com/hkaneko1985/dcekit.

Expanding the Applicability Domain of Machine Learning Model for Advancements in Electrochemical Material Discovery

AbstractMachine learning has gained considerable attention in the material science domain and helped discover advanced materials for electrochemical applications. Numerous studies have demonstrated its potential to reduce the resources required for material screening. However, a significant proportion of these studies have adopted a supervised learning approach, which entails the laborious task of constructing random training databases and does not always ensure the model‘s reliability while screening unseen materials. Herein, we evaluate the limitations of supervised machine learning from the perspective of the applicability domain. The applicability domain of a model is the region in chemical space where the structure‐property relationship is covered by the training set so that the model can give reliable predictions. We review methods that have been developed to overcome such limitations, such as the active learning framework and self‐supervised learning.

Challenges in software model reuse: cross application domain vs. cross modeling paradigm

Challenges in software model reuse: cross application domain vs. cross modeling paradigm

On the Development of Descriptor-Based Machine Learning Models for Thermodynamic Properties: Part 1—From Data Collection to Model Construction: Understanding of the Methods and Their Effects

In the present work, a multi-angle approach is adopted to develop two ML-QSPR models for the prediction of the enthalpy of formation and the entropy of molecules, in their ideal gas state. The molecules were represented by high-dimensional vectors of structural and physico-chemical characteristics (i.e., descriptors). In this sense, an overview is provided of the possible methods that can be employed at each step of the ML-QSPR procedure (i.e., data preprocessing, dimensionality reduction and model construction) and an attempt is made to increase the understanding of the effects related to a given choice or method on the model performance, interpretability and applicability domain. At the same time, the well-known OECD principles for the validation of (Q)SAR models are also considered and addressed. The employed data set is a good representation of two common problems in ML-QSPR modeling, namely the high-dimensional descriptor-based representation and the high chemical diversity of the molecules. This diversity effectively impacts the subsequent applicability of the developed models to a new molecule. The data set complexity is addressed through customized data preprocessing techniques and genetic algorithms. The former improves the data quality while limiting the loss of information, while the latter allows for the automatic identification of the most important descriptors, in accordance with a physical interpretation. The best performances are obtained with Lasso linear models (MAE test = 25.2 kJ/mol for the enthalpy and 17.9 J/mol/K for the entropy). Finally, the overall developed procedure is also tested on various enthalpy and entropy related data sets from the literature to check its applicability to other problems and competing performances are obtained, highlighting that different methods and molecular representations can lead to good performances.

Developing a SARS-CoV-2 main protease binding prediction random forest model for drug repurposing for COVID-19 treatment.

The coronavirus disease 2019 (COVID-19) global pandemic resulted in millions of people becoming infected with the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) virus and close to seven million deaths worldwide. It is essential to further explore and design effective COVID-19 treatment drugs that target the main protease of SARS-CoV-2, a major target for COVID-19 drugs. In this study, machine learning was applied for predicting the SARS-CoV-2 main protease binding of Food and Drug Administration (FDA)-approved drugs to assist in the identification of potential repurposing candidates for COVID-19 treatment. Ligands bound to the SARS-CoV-2 main protease in the Protein Data Bank and compounds experimentally tested in SARS-CoV-2 main protease binding assays in the literature were curated. These chemicals were divided into training (516 chemicals) and testing (360 chemicals) data sets. To identify SARS-CoV-2 main protease binders as potential candidates for repurposing to treat COVID-19, 1188 FDA-approved drugs from the Liver Toxicity Knowledge Base were obtained. A random forest algorithm was used for constructing predictive models based on molecular descriptors calculated using Mold2 software. Model performance was evaluated using 100 iterations of fivefold cross-validations which resulted in 78.8% balanced accuracy. The random forest model that was constructed from the whole training dataset was used to predict SARS-CoV-2 main protease binding on the testing set and the FDA-approved drugs. Model applicability domain and prediction confidence on drugs predicted as the main protease binders discovered 10 FDA-approved drugs as potential candidates for repurposing to treat COVID-19. Our results demonstrate that machine learning is an efficient method for drug repurposing and, thus, may accelerate drug development targeting SARS-CoV-2.

In silico bioavailability triggers applied to direct and indirect thyroid hormone disruptors

Among endocrine disruption, interference with the thyroid hormone (TH) regulation is of increasing concern. Respective compounds encode through their structural features both the potential for TH disruption, and the bioavailability mitigating the toxicological effect. The aim of this study is to provide a substructure-based screening-level QSAR (quantitative structure-activity relationship) that discriminates bioavailable TH disruptors from not bioavailable counterparts, covering both direct and indirect (retinoid- and AhR-mediated) modes of action. The QSAR has been derived from literature data for 1642 compounds, and takes into account Lipinski's rule-of-five and the brain/blood partition coefficient Kbrain/blood. For its validation, an external test set of 145 substances has been used. For 1787 compounds meeting the model application domain, the model yields only one false negative. The discussion addresses the mechanistic meaning of the bioavailability triggers molecular weight, H-bond donor and acceptor, hydrophobicity (log Kow), and the physicochemical properties underlying log Kbrain/blood. The model may serve as bioavailability-screening step within a decision support system for the predictive assessment of chemicals regarding their potential to disrupt thyroid hormone function in a direct or indirect manner.

Interpretable machine learning for the identification of estrogen receptor agonists, antagonists, and binders

An abnormal hormonal activity or exposure to endocrine-disrupting chemicals (EDCs) can cause endocrine system malfunction. Among the many interactions EDCs can affect is the disruption of estrogen signalling, which can lead to adverse health effects such as cancer, osteoporosis, neurodegenerative diseases, cardiovascular disease, insulin resistance, and obesity. Knowing which chemical can act as an EDC is a significant advantage and a practical necessity. New Approach Methodologies (NAM) computational models offer a quick and cost-effective solution for preliminary hazard assessment of chemicals without animal testing. Therefore, a machine learning approach was used to investigate the relationships between estrogen receptor (ER) activity and chemical structure to identify chemicals that can interact with ER. For this purpose, the consolidated in vitro assay data from ToxCast/Tox21 projects was used for developing Random Forest classification models for ER binding, agonists, and antagonists. The overall classification prediction accuracy reaches up to 82%, depending on whether the model predicted agonists, antagonists, or compounds that bind to the active site. Given the imbalance in endocrine disruption data, the derived models are good candidates for deprioritising chemicals and reducing animal testing. The interpretation of theoretical molecular descriptors of the models was consistent with the molecular interactions known in the ligand binding pocket. The estimated class probabilities enabled the analysis of the applicability domain of the developed models and the assessment of the predictions’ reliability, followed by the guidelines for interpreting prediction results. The models are openly accessible and useable at QsarDB.org (http://dx.doi.org/10.15152/QDB.259) according to the FAIR (Findable, Accessible, Interoperable, Reusable) principles.

Forecasting China’s total renewable energy capacity using a novel dynamic fractional order discrete grey model

Accurately forecasting renewable energy capacity is crucial for informing national energy policies and promoting sustainable socio-economic growth. This study proposes a novel dynamic fractional-order discrete grey model (DFDGM (1,1)) for forecasting China's total renewable energy capacity. The model constructs a dynamic fractional time delay polynomial, which enhances the adaptability of the model to different sample sizes. The time response function of model is directly derived from the discrete equation, preventing errors caused by the transition from discreteness to continuity. Furthermore, the proposed model represents the unified form of six grey models, effectively expanding the model's application domain and scope. The method introduces a double error approach to facilitate optimal parameter selection, successfully addressing overfitting and randomness challenges in parameter selection. Empirical studies show that the proposed model outperforms eleven other methods, with fitting and forecasting errors of 0.39% and 1.13%, respectively. Finally, the proposed model forecasts the future development trend of China’s total renewable energy capacity. In general, this research not only improves the precision of predictions but also progresses the dynamic theory in prediction studies.

Count-Based Morgan Fingerprint: A More Efficient and Interpretable Molecular Representation in Developing Machine Learning-Based Predictive Regression Models for Water Contaminants' Activities and Properties.

In this study, we introduce the count-based Morgan fingerprint (C-MF) to represent chemical structures of contaminants and develop machine learning (ML)-based predictive models for their activities and properties. Compared with the binary Morgan fingerprint (B-MF), C-MF not only qualifies the presence or absence of an atom group but also quantifies its counts in a molecule. We employ six different ML algorithms (ridge regression, SVM, KNN, RF, XGBoost, and CatBoost) to develop models on 10 contaminant-related data sets based on C-MF and B-MF to compare them in terms of the model's predictive performance, interpretation, and applicability domain (AD). Our results show that C-MF outperforms B-MF in nine of 10 data sets in terms of model predictive performance. The advantage of C-MF over B-MF is dependent on the ML algorithm, and the performance enhancements are proportional to the difference in the chemical diversity of data sets calculated by B-MF and C-MF. Model interpretation results show that the C-MF-based model can elucidate the effect of atom group counts on the target and have a wider range of SHAP values. AD analysis shows that C-MF-based models have an AD similar to that of B-MF-based ones. Finally, we developed a "ContaminaNET" platform to deploy these C-MF-based models for free use.

Determination of acidity constants of pyridines, imidazoles, and oximes by capillary electrophoresis.

The acidity constant in the form of pKa is one of the most important physicochemical quantities. There are prediction tools available for calculating the pKa , but they only deliver precise calculated values for a relatively small set of chemicals. For complex structures with multiple functional groups in particular, the error in the predicted pKa is high due to the application domain of the corresponding models. Thus, we aim to enlarge the dataset of experimentally determined pKa values using capillary electrophoresis. We, therefore, selected various pyridines, imidazoles, and oximes to determine the pKa values using the internal standard approach and the classical method. Especially oximes were not investigated in the past, and predictions for them include larger errors. Thus, our experimentally determined values could contribute to an improved understanding of various functional groups impacting the pKa values and serve as additional datasets to develop improved pKa prediction tools.

The VEGA Tool to Check the Applicability Domain Gives Greater Confidence in the Prediction of In Silico Models.

A sound assessment of in silico models and their applicability domain can support the use of new approach methodologies (NAMs) in chemical risk assessment and requires increasing the users' confidence in this approach. Several approaches have been proposed to evaluate the applicability domain of such models, but their prediction power still needs a thorough assessment. In this context, the VEGA tool capable of assessing the applicability domain of in silico models is examined for a range of toxicological endpoints. The VEGA tool evaluates chemical structures and other features related to the predicted endpoints and is efficient in measuring applicability domain, enabling the user to identify less accurate predictions. This is demonstrated with many models addressing different endpoints, towards toxicity of relevance to human health, ecotoxicological endpoints, environmental fate, physicochemical and toxicokinetic properties, for both regression models and classifiers.

Merging bioactivity predictions from cell morphology and chemical fingerprint models using similarity to training data

The applicability domain of machine learning models trained on structural fingerprints for the prediction of biological endpoints is often limited by the lack of diversity of chemical space of the training data. In this work, we developed similarity-based merger models which combined the outputs of individual models trained on cell morphology (based on Cell Painting) and chemical structure (based on chemical fingerprints) and the structural and morphological similarities of the compounds in the test dataset to compounds in the training dataset. We applied these similarity-based merger models using logistic regression models on the predictions and similarities as features and predicted assay hit calls of 177 assays from ChEMBL, PubChem and the Broad Institute (where the required Cell Painting annotations were available). We found that the similarity-based merger models outperformed other models with an additional 20% assays (79 out of 177 assays) with an AUC > 0.70 compared with 65 out of 177 assays using structural models and 50 out of 177 assays using Cell Painting models. Our results demonstrated that similarity-based merger models combining structure and cell morphology models can more accurately predict a wide range of biological assay outcomes and further expanded the applicability domain by better extrapolating to new structural and morphology spaces.Graphical

Combatting over-specialization bias in growing chemical databases

BackgroundPredicting in advance the behavior of new chemical compounds can support the design process of new products by directing the research toward the most promising candidates and ruling out others. Such predictive models can be data-driven using Machine Learning or based on researchers’ experience and depend on the collection of past results. In either case: models (or researchers) can only make reliable assumptions about compounds that are similar to what they have seen before. Therefore, consequent usage of these predictive models shapes the dataset and causes a continuous specialization shrinking the applicability domain of all trained models on this dataset in the future, and increasingly harming model-based exploration of the space.Proposed solutionIn this paper, we propose cancels (CounterActiNg Compound spEciaLization biaS), a technique that helps to break the dataset specialization spiral. Aiming for a smooth distribution of the compounds in the dataset, we identify areas in the space that fall short and suggest additional experiments that help bridge the gap. Thereby, we generally improve the dataset quality in an entirely unsupervised manner and create awareness of potential flaws in the data. cancels does not aim to cover the entire compound space and hence retains a desirable degree of specialization to a specified research domain.ResultsAn extensive set of experiments on the use-case of biodegradation pathway prediction not only reveals that the bias spiral can indeed be observed but also that cancels produces meaningful results. Additionally, we demonstrate that mitigating the observed bias is crucial as it cannot only intervene with the continuous specialization process, but also significantly improves a predictor’s performance while reducing the number of required experiments. Overall, we believe that cancels can support researchers in their experimentation process to not only better understand their data and potential flaws, but also to grow the dataset in a sustainable way. All code is available under github.com/KatDost/Cancels.