Many-body Tensor Representation Research Articles

Utilizing machine learning and molecular dynamics for enhanced drug delivery in nanoparticle systems

Materials data science and machine learning (ML) are pivotal in advancing cancer treatment strategies beyond traditional methods like chemotherapy. Nanotherapeutics, which merge nanotechnology with targeted drug delivery, exemplify this advancement by offering improved precision and reduced side effects in cancer therapy. The development of these nanotherapeutic agents depends critically on understanding nanoparticle (NP) properties and their biological interactions, often analyzed through molecular dynamics (MD) simulations. This study enhances these analyses by integrating ML with MD simulations, significantly improving both prediction accuracy and computational efficiency. We introduce a comprehensive three-stage methodology for predicting the solvent-accessible surface area (SASA) of NPs, which is crucial for their therapeutic efficacy. The process involves training an ML model to forecast the many-body tensor representation (MBTR) for future time steps, applying data augmentation to increase dataset realism, and refining the SASA predictor with both augmented and original data. Results demonstrate that our methodology can predict SASA values 299 time steps ahead with a 40-fold speed improvement and a 25% accuracy increase over existing methods. Importantly, it provides a 300-fold increase in computational speed compared to traditional simulation techniques, offering substantial cost and time savings for nanotherapeutic research and development.

Information bottleneck in peptide conformation determination by x-ray absorption spectroscopy

We apply a recently developed technique utilizing machine learning for statistical analysis of computational nitrogen K-edge spectra of aqueous triglycine. This method, the emulator-based component analysis, identifies spectrally relevant structural degrees of freedom from a data set filtering irrelevant ones out. Thus tremendous reduction in the dimensionality of the ill-posed nonlinear inverse problem of spectrum interpretation is achieved. Structural and spectral variation across the sampled phase space is notable. Using these data, we train a neural network to predict the intensities of spectral regions of interest from the structure. These regions are defined by the temperature-difference profile of the simulated spectra, and the analysis yields a structural interpretation for their behavior. Even though the utilized local many-body tensor representation implicitly encodes the secondary structure of the peptide, our approach proves that this information is irrecoverable from the spectra. A hard x-ray Raman scattering experiment confirms the overall sensibility of the simulated spectra, but the predicted temperature-dependent effects therein remain beyond the achieved statistical confidence level.

Structural descriptors and information extraction from X-ray emission spectra: aqueous sulfuric acid.

Machine learning can reveal new insights into X-ray spectroscopy of liquids when the local atomistic environment is presented to the model in a suitable way. Many unique structural descriptor families have been developed for this purpose. We benchmark the performance of six different descriptor families using a computational data set of 24 200 sulfur Kβ X-ray emission spectra of aqueous sulfuric acid simulated at six different concentrations. We train a feed-forward neural network to predict the spectra from the corresponding descriptor vectors and find that the local many-body tensor representation, smooth overlap of atomic positions and atom-centered symmetry functions excel in this comparison. We found a similar hierarchy when applying the emulator-based component analysis to identify and separate the spectrally relevant structural characteristics from the irrelevant ones. In this case, the spectra were dominantly dependent on the concentration of the system, whereas adding the second most significant degree of freedom in the decomposition allowed for distinction of the protonation state of the acid molecule.

QMLMaterial─A Quantum Machine Learning Software for Material Design and Discovery.

Structural elucidation of chemical compounds is challenging experimentally, and theoretical chemistry methods have added important insight into molecules, nanoparticles, alloys, and materials geometries and properties. However, finding the optimum structures is a bottleneck due to the huge search space, and global search algorithms have been used successfully for this purpose. In this work, we present the quantum machine learning software/agent for materials design and discovery (QMLMaterial), intended for automatic structural determination in silico for several chemical systems: atomic clusters, atomic clusters and the spin multiplicity together, doping in clusters or solids, vacancies in clusters or solids, adsorption of molecules or adsorbents on surfaces, and finally atomic clusters on solid surfaces/materials or encapsulated in porous materials. QMLMaterial is an artificial intelligence (AI) software based on the active learning method, which uses machine learning regression algorithms and their uncertainties for decision making on the next unexplored structures to be computed, increasing the probability of finding the global minimum with few calculations as more data is obtained. The software has different acquisition functions for decision making (e.g., expected improvement and lower confidence bound). Also, the Gaussian process is available in the AI framework for regression, where the uncertainty is obtained analytically from Bayesian statistics. For the artificial neural network and support vector regressor algorithms, the uncertainty can be obtained by K-fold cross-validation or nonparametric bootstrap resampling methods. The software is interfaced with several quantum chemistry codes and atomic descriptors, such as the many-body tensor representation. QMLMaterial's capabilities are highlighted in the current work by its applications in the following systems: Na20, Mo6C3 (where the spin multiplicity was considered), H2O@CeNi3O5, Mg8@graphene, Na3Mg3@CNT (carbon nanotube).

Updates to the DScribe library: New descriptors and derivatives.

We present an update of the DScribe package, a Python library for atomistic descriptors. The update extends DScribe's descriptor selection with the Valle-Oganov materials fingerprint and provides descriptor derivatives to enable more advanced machine learning tasks, such as force prediction and structure optimization. For all descriptors, numeric derivatives are now available in DScribe. For the many-body tensor representation (MBTR) and the Smooth Overlap of Atomic Positions (SOAP), we have also implemented analytic derivatives. We demonstrate the effectiveness of the descriptor derivatives for machine learning models of Cu clusters and perovskite alloys.

Unified representation of molecules and crystals for machine learning

Accurate simulations of atomistic systems from first principles are limited by computational cost. In high-throughput settings, machine learning can reduce these costs significantly by accurately interpolating between reference calculations. For this, kernel learning approaches crucially require a representation that accommodates arbitrary atomistic systems. We introduce a many-body tensor representation that is invariant to translations, rotations, and nuclear permutations of same elements, unique, differentiable, can represent molecules and crystals, and is fast to compute. Empirical evidence for competitive energy and force prediction errors is presented for changes in molecular structure, crystal chemistry, and molecular dynamics using kernel regression and symmetric gradient-domain machine learning as models. Applicability is demonstrated for phase diagrams of Pt-group/transition-metal binary systems.

Machine-learning-based prediction of first-principles XANES spectra for amorphous materials

In this paper, a machine-learning-based method is proposed for predicting the x-ray absorption near-edge structure (XANES) for local configurations specific to amorphous materials. A combination of molecular dynamics and first-principles XANES simulations was adopted. The XANES spectrum was assumed to be accurately represented by linear regression of the local atomic descriptors. A comprehensive prediction of Si $K$-edge XANES spectra was performed based on an atom-centered symmetry function, smooth overlap of atomic positions, local many-body tensor representation, and spectral neighbor analysis potential. Furthermore, prediction accuracy was improved by compression of XANES spectral data and efficient sampling of training data.

Machine Learning Predicts the X-ray Photoelectron Spectroscopy of the Solid Electrolyte Interface of Lithium Metal Battery.

X-ray photoelectron spectroscopy (XPS) is a powerful surface analysis technique widely applied in characterizing the solid electrolyte interphase (SEI) of lithium metal batteries. However, experiment XPS measurements alone fail to provide atomic structures from a deeply buried SEI, leaving vital details missing. By combining hybrid ab initio and reactive molecular dynamics (HAIR) and machine learning (ML) models, we present an artificial intelligence ab initio (AI-ai) framework to predict the XPS of a SEI. A localized high-concentration electrolyte with a Li metal anode is simulated with a HAIR scheme for ∼3 ns. Taking the local many-body tensor representation as a descriptor, four ML models are utilized to predict the core level shifts. Overall, extreme gradient boosting exhibits the highest accuracy and lowest variance (with errors ≤ 0.05 eV). Such an AI-ai model enables the XPS predictions of ten thousand frames with marginal cost.

Predicting gas–particle partitioning coefficients of atmospheric molecules with machine learning

Abstract. The formation, properties, and lifetime of secondary organic aerosols in the atmosphere are largely determined by gas–particle partitioning coefficients of the participating organic vapours. Since these coefficients are often difficult to measure and to compute, we developed a machine learning model to predict them given molecular structure as input. Our data-driven approach is based on the dataset by Wang et al. (2017), who computed the partitioning coefficients and saturation vapour pressures of 3414 atmospheric oxidation products from the Master Chemical Mechanism using the COSMOtherm programme. We trained a kernel ridge regression (KRR) machine learning model on the saturation vapour pressure (Psat) and on two equilibrium partitioning coefficients: between a water-insoluble organic matter phase and the gas phase (KWIOM/G) and between an infinitely dilute solution with pure water and the gas phase (KW/G). For the input representation of the atomic structure of each organic molecule to the machine, we tested different descriptors. We find that the many-body tensor representation (MBTR) works best for our application, but the topological fingerprint (TopFP) approach is almost as good and computationally cheaper to evaluate. Our best machine learning model (KRR with a Gaussian kernel + MBTR) predicts Psat and KWIOM/G to within 0.3 logarithmic units and KW/G to within 0.4 logarithmic units of the original COSMOtherm calculations. This is equal to or better than the typical accuracy of COSMOtherm predictions compared to experimental data (where available). We then applied our machine learning model to a dataset of 35 383 molecules that we generated based on a carbon-10 backbone functionalized with zero to six carboxyl, carbonyl, or hydroxyl groups to evaluate its performance for polyfunctional compounds with potentially low Psat. The resulting saturation vapour pressure and partitioning coefficient distributions were physico-chemically reasonable, for example, in terms of the average effects of the addition of single functional groups. The volatility predictions for the most highly oxidized compounds were in qualitative agreement with experimentally inferred volatilities of, for example, α-pinene oxidation products with as yet unknown structures but similar elemental compositions.

Systematic Investigation of Error Distribution in Machine Learning Algorithms Applied to the Quantum-Chemistry QM9 Data Set Using the Bias and Variance Decomposition.

Most machine learning applications in quantum-chemistry (QC) data sets rely on a single statistical error parameter such as the mean square error (MSE) to evaluate their performance. However, this approach has limitations or can even yield incorrect interpretations. Here, we report a systematic investigation of the two components of the MSE, i.e., the bias and variance, using the QM9 data set. To this end, we experiment with three descriptors, namely (i) symmetry functions (SF, with two-body and three-body functions), (ii) many-body tensor representation (MBTR, with two- and three-body terms), and (iii) smooth overlap of atomic positions (SOAP), to evaluate the prediction process's performance using different numbers of molecules in training samples and the effect of bias and variance on the final MSE. Overall, low sample sizes are related to higher MSE. Moreover, the bias component strongly influences the larger MSEs. Furthermore, there is little agreement among molecules with higher errors (outliers) across different descriptors. However, there is a high prevalence among the outliers intersection set and the convex hull volume of geometric coordinates (VCH). According to the obtained results with the distribution of MSE (and its components bias and variance) and the appearance of outliers, it is suggested to use ensembles of models with a low bias to minimize the MSE, more specifically when using a small number of molecules in the training set.

Deep Learning Total Energies and Orbital Energies of Large Organic Molecules Using Hybridization of Molecular Fingerprints.

The ability to predict material properties without the need for resource-consuming experimental efforts can immensely accelerate material and drug discovery. Although ab initio methods can be reliable and accurate in making such predictions, they are computationally too expensive on a large scale. The recent advancements in artificial intelligence and machine learning as well as the availability of large quantum mechanics derived datasets enable us to train models on these datasets as a benchmark and to make fast predictions on much larger datasets. The success of these machine learning models highly depends on the machine-readable fingerprints of the molecules that capture their chemical properties as well as topological information. In this work, we propose a common deep learning-based framework to combine different types of molecular fingerprints to enhance prediction accuracy. A graph neural network (GNN), many-body tensor representation (MBTR), and a set of simple molecular descriptors (MD) were used to predict the total energies, highest occupied molecular orbital (HOMO) energies, and lowest unoccupied molecular orbital (LUMO) energies of a dataset containing ∼62k large organic molecules with complex aromatic rings and remarkably diverse functional groups. The results demonstrate that a combination of best performing molecular fingerprints can produce better results than the individual ones. The simple and flexible deep learning framework developed in this work can be easily adapted to incorporate other types of molecular fingerprints.

Sensitivity and dimensionality of atomic environment representations used for machine learning interatomic potentials.

Faithfully representing chemical environments is essential for describing materials and molecules with machine learning approaches. Here, we present a systematic classification of these representations and then investigate (i) the sensitivity to perturbations and (ii) the effective dimensionality of a variety of atomic environment representations and over a range of material datasets. Representations investigated include atom centered symmetry functions, Chebyshev Polynomial Symmetry Functions (CHSF), smooth overlap of atomic positions, many-body tensor representation, and atomic cluster expansion. In area (i), we show that none of the atomic environment representations are linearly stable under tangential perturbations and that for CHSF, there are instabilities for particular choices of perturbation, which we show can be removed with a slight redefinition of the representation. In area (ii), we find that most representations can be compressed significantly without loss of precision and, further, that selecting optimal subsets of a representation method improves the accuracy of regression models built for a given dataset.

Orbital graph convolutional neural network for material property prediction

Material representations that are compatible with machine learning models play a key role in developing models that exhibit high accuracy for property prediction. Atomic orbital interactions are one of the important factors that govern the properties of crystalline materials, from which the local chemical environments of atoms is inferred. Therefore, to develop robust machine learningmodels for material properties prediction, it is imperative to include features representing such chemical attributes. Here, we propose the Orbital Graph Convolutional Neural Network (OGCNN), a crystal graph convolutional neural network framework that includes atomic orbital interaction features that learns material properties in a robust way. In addition, we embedded an encoder-decoder network into the OGCNN enabling it to learn important features among basic atomic (elemental features), orbital-orbital interactions, and topological features. We examined the performance of this model on a broad range of crystalline material data to predict different properties. We benchmarked the performance of the OGCNN model with that of: 1) the crystal graph convolutional neural network (CGCNN), 2) other state-of-the-art descriptors for material representations including Many-body Tensor Representation (MBTR) and the Smooth Overlap of Atomic Positions (SOAP), and 3) other conventional regression machine learning algorithms where different crystal featurization methods have been used. We find that OGCNN significantly outperforms them. The OGCNN model with high predictive accuracy can be used to discover new materials among the immense phase and compound spaces of materials

DScribe: Library of descriptors for machine learning in materials science

DScribe is a software package for machine learning that provides popular feature transformations (“descriptors”) for atomistic materials simulations. DScribe accelerates the application of machine learning for atomistic property prediction by providing user-friendly, off-the-shelf descriptor implementations. The package currently contains implementations for Coulomb matrix, Ewald sum matrix, sine matrix, Many-body Tensor Representation (MBTR), Atom-centered Symmetry Function (ACSF) and Smooth Overlap of Atomic Positions (SOAP). Usage of the package is illustrated for two different applications: formation energy prediction for solids and ionic charge prediction for atoms in organic molecules. The package is freely available under the open-source Apache License 2.0. Program summaryProgram Title: DScribeProgram Files doi:http://dx.doi.org/10.17632/vzrs8n8pk6.1Licensing provisions: Apache-2.0Programming language: Python/C/C++Supplementary material: Supplementary Information as PDFNature of problem: The application of machine learning for materials science is hindered by the lack of consistent software implementations for feature transformations. These feature transformations, also called descriptors, are a key step in building machine learning models for property prediction in materials science.Solution method: We have developed a library for creating common descriptors used in machine learning applied to materials science. We provide an implementation the following descriptors: Coulomb matrix, Ewald sum matrix, sine matrix, Many-body Tensor Representation (MBTR), Atom-centered Symmetry Functions (ACSF) and Smooth Overlap of Atomic Positions (SOAP). The library has a python interface with computationally intensive routines written in C or C++. The source code, tutorials and documentation are provided online. A continuous integration mechanism is set up to automatically run a series of regression tests and check code coverage when the codebase is updated.

Chemical diversity in molecular orbital energy predictions with kernel ridge regression.

Instant machine learning predictions of molecular properties are desirable for materials design, but the predictive power of the methodology is mainly tested on well-known benchmark datasets. Here, we investigate the performance of machine learning with kernel ridge regression (KRR) for the prediction of molecular orbital energies on three large datasets: the standard QM9 small organic molecules set, amino acid and dipeptide conformers, and organic crystal-forming molecules extracted from the Cambridge Structural Database. We focus on the prediction of highest occupied molecular orbital (HOMO) energies, computed at the density-functional level of theory. Two different representations that encode the molecular structure are compared: the Coulomb matrix (CM) and the many-body tensor representation (MBTR). We find that KRR performance depends significantly on the chemistry of the underlying dataset and that the MBTR is superior to the CM, predicting HOMO energies with a mean absolute error as low as 0.09 eV. To demonstrate the power of our machine learning method, we apply our model to structures of 10k previously unseen molecules. We gain instant energy predictions that allow us to identify interesting molecules for future applications.

Machine learning hydrogen adsorption on nanoclusters through structural descriptors

Catalytic activity of the hydrogen evolution reaction on nanoclusters depends on diverse adsorption site structures. Machine learning reduces the cost for modelling those sites with the aid of descriptors. We analysed the performance of state-of-the-art structural descriptors Smooth Overlap of Atomic Positions, Many-Body Tensor Representation and Atom-Centered Symmetry Functions while predicting the hydrogen adsorption (free) energy on the surface of nanoclusters. The 2D-material molybdenum disulphide and the alloy copper–gold functioned as test systems. Potential energy scans of hydrogen on the cluster surfaces were conducted to compare the accuracy of the descriptors in kernel ridge regression. By having recourse to data sets of 91 molybdenum disulphide clusters and 24 copper–gold clusters, we found that the mean absolute error could be reduced by machine learning on different clusters simultaneously rather than separately. The adsorption energy was explained by the local descriptor Smooth Overlap of Atomic Positions, combining it with the global descriptor Many-Body Tensor Representation did not improve the overall accuracy. We concluded that fitting of potential energy surfaces could be reduced significantly by merging data from different nanoclusters.

An essay on mechanics and the progress of mechanical scienci—1824 to 1852

DScribe is a software package for machine learning that provides popular feature transformations (“descriptors”) for atomistic materials simulations. DScribe accelerates the application of machine learning for atomistic property prediction by providing user-friendly, off-the-shelf descriptor implementations. The package currently contains implementations for Coulomb matrix, Ewald sum matrix, sine matrix, Many-body Tensor Representation (MBTR), Atom-centered Symmetry Function (ACSF) and Smooth Overlap of Atomic Positions (SOAP). Usage of the package is illustrated for two different applications: formation energy prediction for solids and ionic charge prediction for atoms in organic molecules. The package is freely available under the open-source Apache License 2.0.Program Title: DScribeProgram Files doi: http://dx.doi.org/10.17632/vzrs8n8pk6.1Licensing provisions: Apache-2.0Programming language: Python/C/C++Supplementary material: Supplementary Information as PDFNature of problem: The application of machine learning for materials science is hindered by the lack of consistent software implementations for feature transformations. These feature transformations, also called descriptors, are a key step in building machine learning models for property prediction in materials science.Solution method: We have developed a library for creating common descriptors used in machine learning applied to materials science. We provide an implementation the following descriptors: Coulomb matrix, Ewald sum matrix, sine matrix, Many-body Tensor Representation (MBTR), Atom-centered Symmetry Functions (ACSF) and Smooth Overlap of Atomic Positions (SOAP). The library has a python interface with computationally intensive routines written in C or C++. The source code, tutorials and documentation are provided online. A continuous integration mechanism is set up to automatically run a series of regression tests and check code coverage when the codebase is updated.