String Representation Research Articles

Application and Need-Based Architecture Design of Deep Neural Networks

This paper applies a hybrid evolutionary approach to a convolutional neural network (CNN) and determines the number of layers and filters based on the application and user need. It integrates compact genetic algorithm with stochastic gradient descent (SGD) for simultaneously evolving structure and parameters of the CNN. It defines an effectual string representation for combining structure and parameters of the CNN. The compact genetic algorithm helps in the evolution of network structure by optimizing the number of convolutional layers and number of filters in each convolutional layer. At the same time, an optimal set of weight parameters of the network is obtained using the SGD law. This approach amalgamates exploration in network space by compact genetic algorithm and exploitation in weight space with SGD in an effective manner. The proposed approach also incorporates user-defined parameters in the cost function in an elegant manner which controls the network structure and hence the performance of the network based on the users need. The effectiveness of the proposed approach has been demonstrated on four benchmark datasets, namely MNIST, COIL-100, CIFAR-10 and CIFAR-100. The obtained results clearly demonstrate the potential of the proposed approach by evolving architectures based on the nature of the application and the need of the user.

Fully Quantum String Representation of a Wilson Loop in the Finite-Temperature 3D Yang–Mills Theory

We demonstrate the emergence of the Polchinski–Strominger term in the string representation of a Wilson loop in the confinement phase of the finite-temperature 3D Yang–Mills theory. At a temperature which is roughly twice smaller than the deconfinement critical temperature, the value of the coupling of that term becomes such that the string conformal anomaly cancels out, thereby admitting a fully quantum description of the quark–antiquark string in 3D rather than 26D.

Fast Approximations of Activation Functions in Deep Neural Networks when using Posit Arithmetic.

With increasing real-time constraints being put on the use of Deep Neural Networks (DNNs) by real-time scenarios, there is the need to review information representation. A very challenging path is to employ an encoding that allows a fast processing and hardware-friendly representation of information. Among the proposed alternatives to the IEEE 754 standard regarding floating point representation of real numbers, the recently introduced Posit format has been theoretically proven to be really promising in satisfying the mentioned requirements. However, with the absence of proper hardware support for this novel type, this evaluation can be conducted only through a software emulation. While waiting for the widespread availability of the Posit Processing Units (the equivalent of the Floating Point Unit (FPU)), we can already exploit the Posit representation and the currently available Arithmetic-Logic Unit (ALU) to speed up DNNs by manipulating the low-level bit string representations of Posits. As a first step, in this paper, we present new arithmetic properties of the Posit number system with a focus on the configuration with 0 exponent bits. In particular, we propose a new class of Posit operators called L1 operators, which consists of fast and approximated versions of existing arithmetic operations or functions (e.g., hyperbolic tangent (TANH) and extended linear unit (ELU)) only using integer arithmetic. These operators introduce very interesting properties and results: (i) faster evaluation than the exact counterpart with a negligible accuracy degradation; (ii) an efficient ALU emulation of a number of Posits operations; and (iii) the possibility to vectorize operations in Posits, using existing ALU vectorized operations (such as the scalable vector extension of ARM CPUs or advanced vector extensions on Intel CPUs). As a second step, we test the proposed activation function on Posit-based DNNs, showing how 16-bit down to 10-bit Posits represent an exact replacement for 32-bit floats while 8-bit Posits could be an interesting alternative to 32-bit floats since their performances are a bit lower but their high speed and low storage properties are very appealing (leading to a lower bandwidth demand and more cache-friendly code). Finally, we point out how small Posits (i.e., up to 14 bits long) are very interesting while PPUs become widespread, since Posit operations can be tabulated in a very efficient way (see details in the text).

Malware Detection by Risky Zone “Signature” Extraction from API Calls String Transformation using (AOSSR) Technique

Low accuracy of malware detection exists in malware detectors that are based on various malware representation architectures due to several problems as in case of API call graph construction and matching algorithms, where a major issue of building a precise call graph from the information collected about malware samples, also graph matching algorithms are having NP-Complete Problems and slow because of their computational complexity [1], [2]. Moreover, increasing the malware detection accuracy based on API call graphs by enhancing the graph matching and construction algorithms, experiences more computational time taken for matching process and in construction process as there are many graphs created which makes it so difficult to fetch or identify the malware (Elhadi et al., 2013) [3]. It has been further argued by (Li et al., 2018) that in case of malevolent activities, it is comparatively intricate to find whether the software will fall under the spell of malicious occurrences or not, since the duty to determine whether the conduct of a program is malicious is quite complex [4]. This research proposes enhancement of API String-based representation technique through the implementation of a malware detection framework that adopts String-based representation of the malware signature which is a compact representation of the malware risky zone where only the set of API calls representing the actual malware behaviour is accounted in the String using Absolute Order Signature String Representation technique (AOSSR) to represent the malware Strings resulting in a better performance of malware detection accuracy. The Methodology this research work follows mainly composed of three phases. The first phase deals with the conversion of the known malware samples from text format to string format. The second phase addresses the extractaction of the risky zone of an input file which is the file that needs to be checked, by the help of the file signatures already been presented in the database. The third phase addresses how to match two strings efficiency. The Application of the research is very clear as last experiment conducted on 515 malware samples demonstrates that the proposed malware detection architecture has 98% accuracy and 0 false positive rates. Comparing three families, using Analysis of Variance (ANOVA) test it is proven there is no significant difference (p>0.05) in the detection rate of algorithm across the three families of malwares. The algorithms performance is consistent. The result also shows that the Receiver Operating Characteristics (ROC) curves display a better True Positives Rate (TPR) for the proposed architecture over the previous attempts, which reflect significant improvement of TPR.

Bidirectional Molecule Generation with Recurrent Neural Networks.

Recurrent neural networks (RNNs) are able to generate de novo molecular designs using simplified molecular input line entry systems (SMILES) string representations of the chemical structure. RNN-based structure generation is usually performed unidirectionally, by growing SMILES strings from left to right. However, there is no natural start or end of a small molecule, and SMILES strings are intrinsically nonunivocal representations of molecular graphs. These properties motivate bidirectional structure generation. Here, bidirectional generative RNNs for SMILES-based molecule design are introduced. To this end, two established bidirectional methods were implemented, and a new method for SMILES string generation and data augmentation is introduced-the bidirectional molecule design by alternate learning (BIMODAL). These three bidirectional strategies were compared to the unidirectional forward RNN approach for SMILES string generation, in terms of the (i) novelty, (ii) scaffold diversity, and (iii) chemical-biological relevance of the computer-generated molecules. The results positively advocate bidirectional strategies for SMILES-based molecular de novo design, with BIMODAL showing superior results to the unidirectional forward RNN for most of the criteria in the tested conditions. The code of the methods and the pretrained models can be found at URL https://github.com/ETHmodlab/BIMODAL.

Gas Chromatographic Retention Index Prediction Using Multimodal Machine Learning

Gas chromatography is a widely used method in analytical chemistry and metabolomics. Using gas chromatography, vaporizable compounds can be separated for their further identification. Retention indices are standardized values that depend only on a chemical structure of a compound and on a stationary phase and characterize the retention of a compound in a chromatographic system. Retention index prediction is an important task because databases contain experimental values for a small fraction of all possible molecules, while this information is usable for untargeted analysis. In this work, we consider four machine learning models for retention index prediction: 1D and 2D convolutional neural networks, deep residual multilayer perceptron, and gradient boosting. String representation of the molecule, 2D representation of the chemical structure, molecular descriptors and fingerprints, and molecular descriptors are used as inputs of these four models, respectively, along with information about the stationary phase. The first and third models show the best performance, while the other two perform slightly worse. The models predict retention index values for various standard and semi-standard non-polar stationary phases. Further improvement in performance was achieved using a linear model that uses the results of four previous models as inputs (model stacking). The models were tested using various diverse data sets: flavor compounds, essential oils, metabolomics-related compounds. Achieved accuracy: median absolute and percentage errors – 6–40 units and 0.8-2.2%. Accuracy depends on a test data set. The stacking model outperforms previously reported approaches for all test data sets. Parameters of a pre-trained model and some source code are provided.

Randomized SMILES strings improve the quality of molecular generative models

Recurrent Neural Networks (RNNs) trained with a set of molecules represented as unique (canonical) SMILES strings, have shown the capacity to create large chemical spaces of valid and meaningful structures. Herein we perform an extensive benchmark on models trained with subsets of GDB-13 of different sizes (1 million, 10,000 and 1000), with different SMILES variants (canonical, randomized and DeepSMILES), with two different recurrent cell types (LSTM and GRU) and with different hyperparameter combinations. To guide the benchmarks new metrics were developed that define how well a model has generalized the training set. The generated chemical space is evaluated with respect to its uniformity, closedness and completeness. Results show that models that use LSTM cells trained with 1 million randomized SMILES, a non-unique molecular string representation, are able to generalize to larger chemical spaces than the other approaches and they represent more accurately the target chemical space. Specifically, a model was trained with randomized SMILES that was able to generate almost all molecules from GDB-13 with a quasi-uniform probability. Models trained with smaller samples show an even bigger improvement when trained with randomized SMILES models. Additionally, models were trained on molecules obtained from ChEMBL and illustrate again that training with randomized SMILES lead to models having a better representation of the drug-like chemical space. Namely, the model trained with randomized SMILES was able to generate at least double the amount of unique molecules with the same distribution of properties comparing to one trained with canonical SMILES.

Convolutional Neural Networks for the Design and Analysis of Non-Fullerene Acceptors.

Convolutional neural network (CNN) is employed to construct generative and prediction models for the design and analysis of non-fullerene acceptors (NFAs) in organic solar cells. It is demonstrated that the dilated causal CNN can be trained as a good string-based molecular generation model, and the diversity of the generated NFAs is influenced by the depth of convolutional layers. In the property prediction model, the features of NFAs are extracted from the string representations by the dilated CNN. Specially, the attention mechanism is adopted to pool the extracted information, from which the contributions of fragments to molecular properties can be obtained by calculating the corresponding weighted sum. The promising NFAs among the predicted molecules are further verified by quantum chemistry calculations. The proposed generative, prediction models and the theoretical calculations perform as a complete cycle from molecular generation and property prediction to verification, which offer a strategy for the application of CNN in material discovery.

Exploring the GDB-13 chemical space using deep generative models

Recent applications of recurrent neural networks (RNN) enable training models that sample the chemical space. In this study we train RNN with molecular string representations (SMILES) with a subset of the enumerated database GDB-13 (975 million molecules). We show that a model trained with 1 million structures (0.1% of the database) reproduces 68.9% of the entire database after training, when sampling 2 billion molecules. We also developed a method to assess the quality of the training process using negative log-likelihood plots. Furthermore, we use a mathematical model based on the “coupon collector problem” that compares the trained model to an upper bound and thus we are able to quantify how much it has learned. We also suggest that this method can be used as a tool to benchmark the learning capabilities of any molecular generative model architecture. Additionally, an analysis of the generated chemical space was performed, which shows that, mostly due to the syntax of SMILES, complex molecules with many rings and heteroatoms are more difficult to sample.

On loop space-self avoiding string representations for Q.C.D(SU(∞))

We present several clarifying comments on the loop space-self avoid string representation for Q C D SU . . ( ( )) ∞ proposed by this author along last decades

Solving Differential Equation by Modified Genetic Algorithms

Differential equation is a mathematical equation which contains the derivatives of a variable, such as the equation which represent physical quantities, In this paper we introduced modified on the method which proposes a polynomial to solve the ordinary differential equation (ODEs) of second order and by using the evolutionary algorithm to find the coefficients of the propose a polynomial [1] . Our method propose a polynomial to solve the ordinary differential equations (ODEs) of nth order and partial differential equations(PDEs) of order two by using the Genetic algorithm to find the coefficients of the propose a polynomial ,since Evolution Strategies (ESs) use a string representation of the solution to some problem and attempt to evolve a good solution through a series of fitness –based evolutionary steps .unlike (GA) ,an ES will typically not use a population of solution but instead will make a sequence of mutations of an individual solution ,using fitness as a guide[2] . A numerical example with good result show the accuracy of our method compared with some existing methods .and the best error of method it’s not much larger than the error in best of the numerical method solutions.

UHPLC Analysis of Saffron (Crocus sativus L.): Optimization of Separation Using Chemometrics and Detection of Minor Crocetin Esters.

Ultra-high performance liquid chromatography (UHPLC) coupled with diode array detection (DAD) was applied to improve separation and detection of mono- and bis-glucosyl esters of crocetin (crocins), the main red-colored constituents of saffron (Crocus sativus L.), and other polar components. Response surface methodology (RSM) was used to optimise the chromatographic resolution on the Kinetex C18 (Phenomenex) column taking into account of the combined effect of the column temperature, the eluent flow rate and the slope of a linear eluent concentration gradient. A three-level full-factorial design of experiments was adopted to identify suitable combinations of the above factors. The influence of the separation conditions on the resolutions of 22 adjacent peaks was simultaneously modelled by a multi-layer artificial neural network (ANN) in which a bit string representation was used to identify the target analytes. The chromatogram collected under the optimal separation conditions revealed a higher number of crocetin esters than those already characterised by means of mass-spectrometry data and usually detected by HPLC. Ultra-high performance liquid chromatography analyses carried out on the novel Luna Omega Polar C18 (Phenomenex) column confirmed the large number of crocetin derivatives. Further work is in progress to acquire mass-spectrometry data and to clarify the chemical structure to the newly found saffron components.

A similarity study of I/O traces via string kernels

Understanding I/O for data-intense applications is the foundation for the optimization of these applications. The classification of the applications according to the expressed I/O access pattern eases the analysis. An access pattern can be seen as fingerprint of an application. In this paper, we address the classification of traces. Firstly, we convert them first into a weighted string representation. Due to the fact that string objects can be easily compared using kernel methods, we explore their use for fingerprinting I/O patterns. To improve accuracy, we propose a novel string kernel function called kast2 spectrum kernel. The similarity matrices, obtained after applying the mentioned kernel over a set of examples from a real application, were analyzed using kernel principal component analysis and hierarchical clustering. The evaluation showed that two out of four I/O access pattern groups were completely identified, while the other two groups conformed a single cluster due to the intrinsic similarity of their members. The proposed strategy can be promisingly applied to other similarity problems involving tree-like structured data.

Machine learning-based screening of complex molecules for polymer solar cells.

Polymer solar cells admit numerous potential advantages including low energy payback time and scalable high-speed manufacturing, but the power conversion efficiency is currently lower than for their inorganic counterparts. In a Phenyl-C_61-Butyric-Acid-Methyl-Ester (PCBM)-based blended polymer solar cell, the optical gap of the polymer and the energetic alignment of the lowest unoccupied molecular orbital (LUMO) of the polymer and the PCBM are crucial for the device efficiency. Searching for new and better materials for polymer solar cells is a computationally costly affair using density functional theory (DFT) calculations. In this work, we propose a screening procedure using a simple string representation for a promising class of donor-acceptor polymers in conjunction with a grammar variational autoencoder. The model is trained on a dataset of 3989 monomers obtained from DFT calculations and is able to predict LUMO and the lowest optical transition energy for unseen molecules with mean absolute errors of 43 and 74 meV, respectively, without knowledge of the atomic positions. We demonstrate the merit of the model for generating new molecules with the desired LUMO and optical gap energies which increases the chance of finding suitable polymers by more than a factor of five in comparison to the randomised search used in gathering the training set.

Reinforced Adversarial Neural Computer for de Novo Molecular Design.

In silico modeling is a crucial milestone in modern drug design and development. Although computer-aided approaches in this field are well-studied, the application of deep learning methods in this research area is at the beginning. In this work, we present an original deep neural network (DNN) architecture named RANC (Reinforced Adversarial Neural Computer) for the de novo design of novel small-molecule organic structures based on the generative adversarial network (GAN) paradigm and reinforcement learning (RL). As a generator RANC uses a differentiable neural computer (DNC), a category of neural networks, with increased generation capabilities due to the addition of an explicit memory bank, which can mitigate common problems found in adversarial settings. The comparative results have shown that RANC trained on the SMILES string representation of the molecules outperforms its first DNN-based counterpart ORGANIC by several metrics relevant to drug discovery: the number of unique structures, passing medicinal chemistry filters (MCFs), Muegge criteria, and high QED scores. RANC is able to generate structures that match the distributions of the key chemical features/descriptors (e.g., MW, logP, TPSA) and lengths of the SMILES strings in the training data set. Therefore, RANC can be reasonably regarded as a promising starting point to develop novel molecules with activity against different biological targets or pathways. In addition, this approach allows scientists to save time and covers a broad chemical space populated with novel and diverse compounds.

Novel strategy to implement active-space coupled-cluster methods.

A new approach is presented for the efficient implementation of coupled-cluster (CC) methods including higher excitations based on a molecular orbital space partitioned into active and inactive orbitals. In the new framework, the string representation of amplitudes and intermediates is used as long as it is beneficial, but the contractions are evaluated as matrix products. Using a new diagrammatic technique, the CC equations are represented in a compact form due to the string notations we introduced. As an application of these ideas, a new automated implementation of the single-reference-based multi-reference CC equations is presented for arbitrary excitation levels. The new program can be considered as an improvement over the previous implementations in many respects; e.g., diagram contributions are evaluated by efficient vectorized subroutines. Timings for test calculations for various complete active-space problems are presented. As an application of the new code, the weak interactions in the Be dimer were studied.

Adversarial Threshold Neural Computer for Molecular de Novo Design.

In this article, we propose the deep neural network Adversarial Threshold Neural Computer (ATNC). The ATNC model is intended for the de novo design of novel small-molecule organic structures. The model is based on generative adversarial network architecture and reinforcement learning. ATNC uses a Differentiable Neural Computer as a generator and has a new specific block, called adversarial threshold (AT). AT acts as a filter between the agent (generator) and the environment (discriminator + objective reward functions). Furthermore, to generate more diverse molecules we introduce a new objective reward function named Internal Diversity Clustering (IDC). In this work, ATNC is tested and compared with the ORGANIC model. Both models were trained on the SMILES string representation of the molecules, using four objective functions (internal similarity, Muegge druglikeness filter, presence or absence of sp3-rich fragments, and IDC). The SMILES representations of 15K druglike molecules from the ChemDiv collection were used as a training data set. For the different functions, ATNC outperforms ORGANIC. Combined with the IDC, ATNC generates 72% of valid and 77% of unique SMILES strings, while ORGANIC generates only 7% of valid and 86% of unique SMILES strings. For each set of molecules generated by ATNC and ORGANIC, we analyzed distributions of four molecular descriptors (number of atoms, molecular weight, logP, and tpsa) and calculated five chemical statistical features (internal diversity, number of unique heterocycles, number of clusters, number of singletons, and number of compounds that have not been passed through medicinal chemistry filters). Analysis of key molecular descriptors and chemical statistical features demonstrated that the molecules generated by ATNC elicited better druglikeness properties. We also performed in vitro validation of the molecules generated by ATNC; results indicated that ATNC is an effective method for producing hit compounds.

Utilizing new approaches to address the fuzzy fixed charge transportation problem

The Fuzzy Fixed Charge Transportation Problem in which both fixed and transportation cost are fuzzy numbers is considered in this paper. Due to NP-hardness of the problem, we utilize three types of Electromagnetism-like Algorithms (EM), Genetic Algorithm (GA), and Simulated Annealing (SA) which are firstly being proposed and comprised in this research area. Besides, our other novelty approach is the use of new encoding mechanism, namely string representation, for the first time which is employed for the problem and can be used in any extended transportation problems. Also, the last version of EM is being firstly developed and proposed in this paper. The employed operators and parameters are calibrated to ensure the best performance of the algorithms. Besides, different problem sizes are considered at random to study the impacts of the rise in the problem size on the performance of the algorithms.

The use of Classical Rolling Pendulum Bearings (CRPB) for vibration control of stay-cables

Cables are efficient structural elements that are used in cable-stayed bridges, suspension bridges and other cable structures. A significant problem which arose from the praxis is the cables’ rain-wind induced vibrations as these cables are subjected to environmental excitations. Rain-wind induced stay-cable vibrations may occur at different cable eigenfrequencies. Large amplitude Rain-Wind-Induced-Vibrations (RWIV) of stay cables are a challenging problem in the design of cable-stayed bridges. Several methods, including aerodynamic or structural means, have been investigated in order to control the vibrations of bridge’s stay-cables. The present research focuses on the effectiveness of a movable anchorage system with a Classical Rolling Pendulum Bearing (CRPB) device. An analytical model of cable-damper system is developed based on the taut string representation of the cable. The gathered integral-differential equations are solved through the use of the Lagrange transformation. . Finally, a case study with realistic geometrical parameters is also presented to establish the validity of the proposed system.

Source Code Documentation Generation Using Program Execution

Automated source code documentation approaches often describe methods in abstract terms, using the words contained in the static source code or code excerpts from repositories. In this paper, we describe DynamiDoc: a simple automated documentation generator based on dynamic analysis. Our representation-based approach traces the program being executed and records string representations of concrete argument values, a return value and a target object state before and after each method execution. Then, for each method, it generates documentation sentences with examples, such as “When called on [3, 1.2] with element = 3, the object changed to [1.2]”. Advantages and shortcomings of the approach are listed. We also found out that the generated sentences are substantially shorter than the methods they describe. According to our small-scale study, the majority of objects in the generated documentation have their string representations overridden, which further confirms the potential usefulness of our approach. Finally, we propose an alternative, variable-based approach that describes the values of individual member variables, rather than the state of an object as a whole.