String Representation Research Articles

AbstractTrace: The Use of Execution Traces to Cluster, Classify, Prioritize, and Optimize a Bloated Test Suite

Due to the incremental and iterative nature of the software testing process, a test suite may become bloated with redundant, overlapping, and similar test cases. This paper aims to optimize a bloated test suite by employing an execution trace that encodes runtime events into a sequence of characters forming a string. A dataset of strings, each of which represents the code coverage and execution behavior of a test case, is analyzed to identify similarities between test cases. This facilitates the de-bloating process by providing a formal mechanism to identify, remove, and reduce extra test cases without compromising software quality. This form of analysis allows for the clustering and classification of test cases based on their code coverage and similarity score. This paper explores three levels of execution traces and evaluates different techniques to measure their similarities. Test cases with the same code coverage should generate the exact string representation of runtime events. Various string similarity metrics are assessed to find the similarity score, which is used to classify, detect, and rank test cases accordingly. Additionally, this paper demonstrates the validity of the approach with two case studies. The first shows how to classify the execution behavior of various test cases, which can provide insight into each test case’s internal behavior. The second shows how to identify similar test cases based on their code coverage.

Generative Pretrained Transformer for Heterogeneous Catalysts.

Discovery of novel and promising materials is a critical challenge in the field of chemistry and material science, traditionally approached through methodologies ranging from trial-and-error to machine-learning-driven inverse design. Recent studies suggest that transformer-based language models can be utilized as material generative models to expand the chemical space and explore materials with desired properties. In this work, we introduce the catalyst generative pretrained transformer (CatGPT), trained to generate string representations of inorganic catalyst structures from a vast chemical space. CatGPT not only demonstrates high performance in generating valid and accurate catalyst structures but also serves as a foundation model for generating the desired types of catalysts by text-conditioning and fine-tuning. As an example, we fine-tuned the pretrained CatGPT using a binary alloy catalyst data set designed for screening two-electron oxygen reduction reaction (2e-ORR) catalyst and generated catalyst structures specialized for 2e-ORR. Our work demonstrates the potential of generative language models as generative tools for catalyst discovery.

Machine Learnable Language for the Chemical Space of Nanopores Enables Structure-Property Relationships in Nanoporous 2D Materials.

The synthesis of nanoporous two-dimensional (2D) materials has revolutionized fields such as membrane separations, DNA sequencing, and osmotic power harvesting. Nanopores in 2D materials significantly modulate their optoelectronic, magnetic, and barrier properties. However, the large number of possible nanopore isomers makes their study onerous, while the lack of machine-learnable representations stymies progress toward structure-property relationships. Here, we develop a language for nanopores in 2D materials, called STring Representation Of Nanopore Geometry (STRONG), that opens the field of 2D nanopore informatics. We show that STRONGs are naturally suited for machine learning via recurrent neural networks, predicting formation energies/times of arbitrary nanopores and transport barriers for CO2, N2, and O2 gas molecules, enabling structure-property relationships. The machine learning models enable the discovery of specific nanopore topologies to separate CO2/N2, O2/CO2, and O2/N2 gas mixtures with high selectivity ratios. We also enable the rapid enumeration of unique configurations of stable, functionalized nanopores in 2D materials via STRONGs, allowing systematic searching of the vast chemical space of nanopores. Using the STRONGs approach, we find that a mix of hydrogen and quinone functionalization results in the most stable functionalized nanopore configuration in graphene, a discovery made feasible by expedited chemical space exploration. Additionally, we also unravel the STRONGs approach as ∼1000 times faster than graph theory algorithms to distinguish nanopore shapes. These advances in the language-based representation of 2D nanopores will accelerate the tailored design of nanoporous materials.

What can attribution methods show us about chemical language models?

Language models trained on molecular string representations have shown strong performance in predictive and generative tasks. However, practical applications require not only making accurate predictions, but also explainability - the ability to explain the reasons and rationale behind the predictions. In this work, we explore explainability for a chemical language model by adapting a transformer-specific and a model-agnostic input attribution technique. We fine-tune a pretrained model to predict aqueous solubility, compare training and architecture variants, and evaluate visualizations of attributed relevance. The model-agnostic SHAP technique provides sensible attributions, highlighting the positive influence of individual electronegative atoms, but does not explain the model in terms of functional groups or explain how the model represents molecular strings internally to make predictions. In contrast, the adapted transformer-specific explainability technique produces sparse attributions, which cannot be directly attributed to functional groups relevant to solubility. Instead, the attributions are more characteristic of how the model maps molecular strings to its latent space, which seems to represent features relevant to molecular similarity rather than functional groups. These findings provide insight into the representations underpinning chemical language models, which we propose may be leveraged for the design of informative chemical spaces for training more accurate, advanced and explainable models.

Automated design of multi-target ligands by generative deep learning

Generative deep learning models enable data-driven de novo design of molecules with tailored features. Chemical language models (CLM) trained on string representations of molecules such as SMILES have been successfully employed to design new chemical entities with experimentally confirmed activity on intended targets. Here, we probe the application of CLM to generate multi-target ligands for designed polypharmacology. We capitalize on the ability of CLM to learn from small fine-tuning sets of molecules and successfully bias the model towards designing drug-like molecules with similarity to known ligands of target pairs of interest. Designs obtained from CLM after pooled fine-tuning are predicted active on both proteins of interest and comprise pharmacophore elements of ligands for both targets in one molecule. Synthesis and testing of twelve computationally favored CLM designs for six target pairs reveals modulation of at least one intended protein by all selected designs with up to double-digit nanomolar potency and confirms seven compounds as designed dual ligands. These results corroborate CLM for multi-target de novo design as source of innovation in drug discovery.

Evolutionary algorithms simulating molecular evolution: a new field proposal.

The genetic blueprint for the essential functions of life is encoded in DNA, which is translated into proteins-the engines driving most of our metabolic processes. Recent advancements in genome sequencing have unveiled a vast diversity of protein families, but compared with the massive search space of all possible amino acid sequences, the set of known functional families is minimal. One could say nature has a limited protein "vocabulary." A major question for computational biologists, therefore, is whether this vocabulary can be expanded to include useful proteins that went extinct long ago or have never evolved (yet). By merging evolutionary algorithms, machine learning, and bioinformatics, we can develop highly customized "designer proteins." We dub the new subfield of computational evolution, which employs evolutionary algorithms with DNA string representations, biologically accurate molecular evolution, and bioinformatics-informed fitness functions, Evolutionary Algorithms Simulating Molecular Evolution.

Molecular sharing and molecular-specific representations for multimodal molecular property prediction

Molecular property prediction plays a crucial role in drug discovery and development. However, traditional experimental measurements and Quantitative Structure-Activity Relationship (QSAR) models are often expensive, time-consuming, and data acquisition is challenging. To overcome these limitations and challenges, this study innovatively proposes a fusion molecular property prediction method called molecular property prediction model (MSSP) to address the non-uniqueness of Simplified Molecular Input Line Entry System (SMILES) string representation and the difficulty of capturing global information in molecular graphs. This method extracts multiple fingerprint features and utilizes graph neural network encoding to map different modalities of molecules into molecular sharing and molecular-specific representation spaces, achieving modal alignment and fusion of molecules by combining molecular invariance and representation specificity. To enhance the interpretability and visualization capabilities of the model, graph attention mechanisms are introduced, enabling the identification and inference of important chemical fragments within molecules. Experimental results on publicly available cell line phenotype and kinase activity datasets demonstrate that MSSP outperforms the current state-of-the-art methods in molecular property prediction. Additionally, MSSP exhibits strong competitiveness across nine benchmark molecular property prediction datasets. Furthermore, in the task of predicting SRC kinase data properties, this study successfully screens promising therapeutic compounds from compound libraries by validating the predictions of the MSSP model and combining them with traditional methods such as molecular docking and molecular dynamics simulations. Multiple potential Lyn inhibitors have been discovered through this approach. The application of MSSP model is helpful to discover new molecules with new drug properties or functions, accelerate the process of drug discovery, save time and resources, and provide important guidance for drug discovery.

Nonmesonic Quantum Many-Body Scars in a 1D Lattice Gauge Theory.

We investigate the meson excitations (particle-antiparticle bound states) in quantum many-body scars of a 1D Z_{2} lattice gauge theory coupled to a dynamical spin-1/2 chain as a matter field. By introducing a string representation of the physical Hilbert space, we express a scar state |Ψ_{n,l}⟩ as a superposition of all string bases with an identical string number n and a total length l. For the small-l scar state |Ψ_{n,l}⟩, the gauge-invariant spin exchange correlation function of the matter field hosts an exponential decay as the distance increases, indicating the existence of stable mesons. However, for large l, the correlation function exhibits a power-law decay, signaling the emergence of nonmesonic excitations. Furthermore, we show that this mesonic-nonmesonic crossover can be detected by the quench dynamics, starting from two low-entangled initial states, respectively, which are experimentally feasible in quantum simulators. Our results expand the physics of quantum many-body scars in lattice gauge theories and reveal that the nonmesonic state can also manifest ergodicity breaking.

Enhancing deep learning predictive models with HAPPY (Hierarchically Abstracted rePeat unit of PolYmers) representation

We introduce HAPPY (Hierarchically ed rePeat unit of PolYmers), a string representation for polymers, designed to efficiently encapsulate essential polymer structure features for property prediction. HAPPY assigns single constituent elements to groups of sub-structures and employs grammatically complete and independent connectors between chemical linkages. Using a limited number of datapoints, we trained neural networks utilizing both HAPPY and conventional SMILES encoding of repeated unit structures and compared their performance in predicting five polymer properties: dielectric constant, glass transition temperature, thermal conductivity, solubility, and density. The results showed that the HAPPY-based network could achieve higher prediction R-squared score and two-fold faster training times. We further tested the robustness and versatility of HAPPY-based network with an augmented training dataset. Additionally, we present topo-HAPPY (Topological HAPPY), an extension that incorporates topological details of the constituent connectivity, leading to improved solubility and glass transition temperature prediction R-squared score.

Efficient non-isomorphic graph enumeration algorithms for several intersection graph classes

Intersection graphs are well-studied in the area of graph algorithms. Some intersection graph classes are known to have algorithms enumerating all unlabeled graphs by reverse search. Since these algorithms output graphs one by one and the numbers of graphs in these classes are vast, they work only for a small number of vertices. Binary decision diagrams (BDDs) are compact data structures for various types of data and useful for solving optimization and enumeration problems. This study proposes enumeration algorithms for five intersection graph classes, which admit O(n)-bit string representations for their member graphs. Our algorithm for each class enumerates all unlabeled graphs with n vertices over BDDs representing the binary strings in time polynomial in n. Moreover, our algorithms are extended so that it enumerates those with constraints on the maximum (bi)clique size and/or the number of edges.

Difficulty in chirality recognition for Transformer architectures learning chemical structures from string representations

Recent years have seen rapid development of descriptor generation based on representation learning of extremely diverse molecules, especially those that apply natural language processing (NLP) models to SMILES, a literal representation of molecular structure. However, little research has been done on how these models understand chemical structure. To address this black box, we investigated the relationship between the learning progress of SMILES and chemical structure using a representative NLP model, the Transformer. We show that while the Transformer learns partial structures of molecules quickly, it requires extended training to understand overall structures. Consistently, the accuracy of molecular property predictions using descriptors generated from models at different learning steps was similar from the beginning to the end of training. Furthermore, we found that the Transformer requires particularly long training to learn chirality and sometimes stagnates with low performance due to misunderstanding of enantiomers. These findings are expected to deepen the understanding of NLP models in chemistry.

Making the InChI FAIR and sustainable while moving to inorganics.

The InChI (International Chemical Identifier) standard stands as a cornerstone in chemical informatics, facilitating the structure-based identification and exchange chemical information about compounds across various platforms and databases. The InChI as a unique canonical line notation has made chemical structures searchable on the internet at a broad scale. The largest repositories working with InChIs contain more than 1 billion structures. Central to the functionality of the InChI is its codebase, which orchestrates a series of intricate steps to generate unique identifiers for chemical compounds. Up to now, these steps have been sparsely documented and the InChI algorithm had to be seen as a black box. For the new v1.07 release, the code has been analyzed and the major steps documented, more than 3000 bugs and security issues, as well as nearly 60 Google OSS-Fuzz issues have been fixed. New test systems have been implemented that allow users to directly test the code developments. The move to GitHub has not only made the development more transparent but will also enable external contributors to join the further development of the InChI code. Motivation for this modernisation was the urgency to treat molecular inorganic compounds by the InChI in a meaningful way. Until now, no classic string representation fulfills this need of molecular inorganic chemistry. Currently bonds to metal centers are by definition disconnected which makes most inorganic InChIs meaningless at the moment. Herein, we propose new routines to remedy this problem in the representation of molecular inorganic compounds by the InChI.

DEEPSIDE A DEEP LEARNING FRAMEWORK FOR DRUG SIDE EFFECT PREDICTION

Drug failures brought on by unanticipated side effects during clinical trials put participants' health at risk and result in significant financial losses. Algorithms for predicting side effects may direct the development of new drugs. The LINCS L1000 dataset establishes a knowledge foundation for context-specific characteristics and offers a wealth of cell line gene expression data that has been altered by various medications. The state-of-the-art method, which seeks to use context-specific information, discards a significant number of the trials and only uses the high-quality experiments in LINCS L1000. Our aim in this research is to maximize the prediction performance by fully using this data. Five deep learning architectures are used in our experiments. We discover that when drug chemical structure (CS) and the whole collection of drug altered gene expression profiles (GEX) are employed as modalities, a multi-modal architecture yields the greatest prediction performance among multi-layer perceptron-based designs. We find that overall, the CS provides more information than the GEX. The best results are obtained by a convolutional neural network-based model that just employs the SMILES string representation of the medications; this model outperforms the state-of-the-art by 3:1% micro-AUC and 13:0% macro-AUC. Additionally, we demonstrate that the model can forecast drug-side effect couples that are absent from the ground truth side effect dataset but are described in the literature.

DEEP SIDE-A DEEP LEARNING FRAMEWORK FOR DRUG SIDE EFFECT PREDICTION

Drug failures due to unforeseen adverse effects at clinical trials pose health risks for the participants and lead to substantial financial losses. Side effect prediction algorithms have the potential to guide the drug design process. LINCS L1000 dataset provides a vast resource of cell line gene expression data perturbed by different drugs and creates a knowledge base for context specific features. The state-of-the-art approach that aims at using context specific information relies on only the high quality experiments in LINCS L1000 and discards a large portion of the experiments. In this study, our goal is to boost the prediction performance by utilizing this data to its full extent. We experiment with 5 deep learning architectures. We find that a multimodal architecture produces the best predictive performance among multi-layer perceptron-based architectures when drug chemical structure (CS), and the full set of drug perturbed gene expression profiles (GEX) are used as modalities. Overall, we observe that the CS is more informative than the GEX. A convolutional neural network-based model that uses only SMILES string representation of the drugs achieves the best results and provides 13:0% macro-AUC and 3:1% micro-AUC improvements over the state-of-the-art. We also show that the model is able to predict side effect-drug pairs that are reported in the literature but was missing in the ground truth side effect dataset.

Transforming data from the image to the text domain: benign versus malignant micro-calcification classification

In this paper we present a new approach for the classification of benign and malignant micro-calcification clusters by transforming data from the image to the text domain. A string representation is extracted from binary micro-calcification segmentation images. We extracted two different features from the strings and combined different machine learning techniques towards benign versus malignant classification. We evaluated our proposed method on the DDSM database and experimental results indicates a Classification Accuracy equal to 92%.

Improved ant colony algorithm for the mixed-model parallel two-sided assembly lines balancing problem

ABSTRACT In response to the increasing demand for individualized products in the ‘small-lot, multi-variety’ market, there is an urgent need to enhance the efficiency and intelligence level of assembly lines. To address this, a novel solution method for the mixed-model parallel two-sided assembly lines balancing problem (MMPTSALBP) is proposed. The first step is to define the type and layout of the parallel two-sided assembly lines. Next, a mathematical model is constructed with the objective of minimizing the number of workstations required for the MMPTSALBP, proposing an improved ant colony algorithm that utilizes double string representation for the initial solution encoding method and a double pheromone matrix update to solve the model. Its efficiency is tested on benchmark datasets according to some performance measures and classifications. A study validates its effectiveness on 25 classical arithmetic cases by comparison with the solutions of an heuristic algorithm and an artificial fish swarm algorithm that had been reported to perform well.

MOF-GRU: A MOFid-Aided Deep Learning Model for Predicting the Gas Separation Performance of Metal-Organic Frameworks.

The remarkable versatility of metal-organic frameworks (MOFs) stems from their rich chemical information, leading to numerous successful applications. However, identifying optimal MOFs for specific tasks necessitates a thorough assessment of their chemical attributes. Conventional machine learning approaches for MOF prediction have relied on intricate chemical and structural details, hampering rapid evaluations. Drawing inspiration from recent advancements exemplified by Snurr et al., wherein a text string was used to represent a MOF (MOFid), we introduce a MOFid-aided deep learning model, named the MOF-GRU model. This model, founded on natural language processing principles and utilizing the gated recurrent unit architecture, leverages the serialized text string representation of metal-organic frameworks (MOFs) to forecast gas separation performance. Through a focused study on CH4/N2 separation, we substantiate the efficacy of this approach. Comparative assessments against traditional machine learning techniques underscore our model's superior predictive accuracy and its capacity to handle extensive data sets adeptly. The MOF-GRU model remarkably uncovers latent structure-performance relationships with only MOF sequences, obviating the necessity for intricate three-dimensional (3D) structural information. Overall, this model's judicious design empowers efficient data utilization, thereby hastening the discovery of high-performance materials tailored for gas separation applications.

Modelling orthographic similarity effects in recognition memory reveals support for open bigram representations of letter coding

A variety of letter string representations has been proposed in the reading literature to account for empirically established orthographic similarity effects from masked priming studies. However, these similarity effects have not been explored in episodic memory paradigms and very few memory models have employed orthographic representation of words. In the current work, through two recognition memory experiments employing word and pseudoword stimuli respectively, we empirically established a set of key orthographic similarity effects for the first time in recognition memory – namely the substitution effect, transposition effect and reverse effect in recognition memory of words and pseudowords, and a start-letter importance in recognition memory of words. Subsequently, we compared orthographic representations from the reading literature including slot coding, closed-bigram, open-bigram and the overlap model. Each of these representations was situated in a global matching model and fitted to recognition performance via Luce’s choice rule in a hierarchical Bayesian framework. Model selection results showed support for the open-bigram representation in both experiments.

Study of Superconductivity in Restricted Quantum Chromo-Dynamics in non-Abelian Gauge Theory

An Abelian Higgs model with dual superconductivity and confinement has been constructed and its string representation is described in terms of the average of Wilson loops in this QCD Abelian projection. Investigations has been carried out on monopole condensation and chromo-magnetic superconductivity within the framework of restricted quantum chromodynamics in the SU(2) gauge theory.

Using alternative SMILES representations to identify novel functional analogues in chemical similarity vector searches

Chemical similarity searches are a widely used family of in silico methods for identifying pharmaceutical leads. These methods historically relied on structure-based comparisons to compute similarity. Here, we use a chemical language model to create a vector-based chemical search. We extend previous implementations by creating a prompt engineering strategy that utilizes two different chemical string representation algorithms: one for the query and the other for the database. We explore this method by reviewing search results from nine queries with diverse targets. We find that the method identifies molecules with similar patent-derived functionality to the query, as determined by our validated LLM-assisted patent summarization pipeline. Further, many of these functionally similar molecules have different structures and scaffolds from the query, making them unlikely to be found with traditional chemical similarity searches. This method may serve as a new tool for the discovery of novel molecular structural classes that achieve target functionality.