Shedding light on “Black Box” machine learning models for predicting the reactivity of HO[rad] radicals toward organic compounds

Abstract

Developing quantitative structure-activity relationships (QSARs) is an important approach to predicting the reactivity of HO radicals toward newly emerged organic compounds. As compared with molecular descriptors-based and the group contribution method-based QSARs, a combined molecular fingerprint-machine learning (ML) method can more quickly and accurately develop such models for a growing number of contaminants. However, it is yet unknown whether this method makes predictions by choosing meaningful structural features rather than spurious ones, which is vital for trusting the models. In this study, we developed QSAR models for the logkHO values of 1089 organic compounds in the aqueous phase by two ML algorithms—deep neural networks (DNN) and eXtreme Gradient Boosting (XGBoost), and interpreted the built models by the SHapley Additive exPlanations (SHAP) method. The results showed that for the contribution of a given structural feature to logkHO for different compounds, DNN and XGBoost treated it as a fixed and variable value, respectively. We then developed an ensemble model combining the DNN with XGBoost, which achieved satisfactory predictive performance for all three datasets: Training dataset: R-square (R2) 0.89–0.91, root-mean-squared-error (RMSE) 0.21–0.23, and mean absolute error (MAE) 0.15–0.17; Validation dataset: R2 0.63–0.78, RMSE 0.29–0.32, and MAE 0.21–0.25; and Test dataset: R2 0.60–0.71, RMSE 0.30–0.35, and MAE 0.23–0.25. The SHAP method was further used to unveil that this ensemble model made predictions on logkHO based on a correct ‘understanding’ of the impact of electron-withdrawing and -donating groups and of the reactive sites in the compounds that can be attacked by HO. This study offered some much-needed mechanistic insights into a ML-assisted environmental task, which are important for evaluating the trustworthiness of the ML-based models, further improving the models for specific applications, and leveraging the implicit knowledge the models carry.