Sequence-based Model Research Articles

AMPDeep: hemolytic activity prediction of antimicrobial peptides using transfer learning.

BackgroundDeep learning’s automatic feature extraction has proven to give superior performance in many sequence classification tasks. However, deep learning models generally require a massive amount of data to train, which in the case of Hemolytic Activity Prediction of Antimicrobial Peptides creates a challenge due to the small amount of available data.ResultsThree different datasets for hemolysis activity prediction of therapeutic and antimicrobial peptides are gathered and the AMPDeep pipeline is implemented for each. The result demonstrate that AMPDeep outperforms the previous works on all three datasets, including works that use physicochemical features to represent the peptides or those who solely rely on the sequence and use deep learning to learn representation for the peptides. Moreover, a combined dataset is introduced for hemolytic activity prediction to address the problem of sequence similarity in this domain. AMPDeep fine-tunes a large transformer based model on a small amount of peptides and successfully leverages the patterns learned from other protein and peptide databases to assist hemolysis activity prediction modeling.ConclusionsIn this work transfer learning is leveraged to overcome the challenge of small data and a deep learning based model is successfully adopted for hemolysis activity classification of antimicrobial peptides. This model is first initialized as a protein language model which is pre-trained on masked amino acid prediction on many unlabeled protein sequences in a self-supervised manner. Having done so, the model is fine-tuned on an aggregated dataset of labeled peptides in a supervised manner to predict secretion. Through transfer learning, hyper-parameter optimization and selective fine-tuning, AMPDeep is able to achieve state-of-the-art performance on three hemolysis datasets using only the sequence of the peptides. This work assists the adoption of large sequence-based models for peptide classification and modeling tasks in a practical manner.

Geographic encoding of transcripts enabled high-accuracy and isoform-aware deep learning of RNA methylation.

As the most pervasive epigenetic mark present on mRNA and lncRNA, N6-methyladenosine (m6A) RNA methylation regulates all stages of RNA life in various biological processes and disease mechanisms. Computational methods for deciphering RNA modification have achieved great success in recent years; nevertheless, their potential remains underexploited. One reason for this is that existing models usually consider only the sequence of transcripts, ignoring the various regions (or geography) of transcripts such as 3′UTR and intron, where the epigenetic mark forms and functions. Here, we developed three simple yet powerful encoding schemes for transcripts to capture the submolecular geographic information of RNA, which is largely independent from sequences. We show that m6A prediction models based on geographic information alone can achieve comparable performances to classic sequence-based methods. Importantly, geographic information substantially enhances the accuracy of sequence-based models, enables isoform- and tissue-specific prediction of m6A sites, and improves m6A signal detection from direct RNA sequencing data. The geographic encoding schemes we developed have exhibited strong interpretability, and are applicable to not only m6A but also N1-methyladenosine (m1A), and can serve as a general and effective complement to the widely used sequence encoding schemes in deep learning applications concerning RNA transcripts.

Current sequence-based models capture gene expression determinants in promoters but mostly ignore distal enhancers

Current sequence-based models capture gene expression determinants in promoters but mostly ignore distal enhancers

A Deep Sequence Learning Framework for Action Recognition in Small-Scale Depth Video Dataset.

Depth video sequence-based deep models for recognizing human actions are scarce compared to RGB and skeleton video sequences-based models. This scarcity limits the research advancements based on depth data, as training deep models with small-scale data is challenging. In this work, we propose a sequence classification deep model using depth video data for scenarios when the video data are limited. Unlike summarizing the frame contents of each frame into a single class, our method can directly classify a depth video, i.e., a sequence of depth frames. Firstly, the proposed system transforms an input depth video into three sequences of multi-view temporal motion frames. Together with the three temporal motion sequences, the input depth frame sequence offers a four-stream representation of the input depth action video. Next, the DenseNet121 architecture is employed along with ImageNet pre-trained weights to extract the discriminating frame-level action features of depth and temporal motion frames. The extracted four sets of feature vectors about frames of four streams are fed into four bi-directional (BLSTM) networks. The temporal features are further analyzed through multi-head self-attention (MHSA) to capture multi-view sequence correlations. Finally, the concatenated genre of their outputs is processed through dense layers to classify the input depth video. The experimental results on two small-scale benchmark depth datasets, MSRAction3D and DHA, demonstrate that the proposed framework is efficacious even for insufficient training samples and superior to the existing depth data-based action recognition methods.

Cost analysis of lurasidone for the treatment of schizophrenia in adolescents and adults within the United Kingdom

BackgroundSchizophrenia is a serious mental health condition characterised by distortions in thought processes, perception, mood, sense of self, and behaviour. Lurasidone, a second-generation atypical antipsychotic, represents an additional treatment option alongside existing antipsychotics for adolescents and adults with schizophrenia. An economic model was developed to evaluate the incremental costs of lurasidone as a first-line treatment option compared to existing antipsychotics.MethodsA Markov model was developed to estimate the cost impact of lurasidone as a first-line treatment option for both adolescents and adults. The sequence-based model incorporated the following health states: stable (no relapse or discontinuation), discontinuation (due to adverse events or other reasons), and relapse. Data used to determine the movement of patients between health states were obtained from network meta-analyses (NMAs). The time horizon ranged from three to five years (depending on the patient population) and a six-weekly cycle length was used. Unit costs and resource use were reflective of the UK NHS and Personal Social Services and consisted of the following categories: outpatient, adverse events, primary and residential care. Extensive deterministic sensitivity analysis was undertaken to assess the level of uncertainty associated with the base case results.ResultsLurasidone is demonstrated to be cost-saving as a first-line treatment within the adolescent and adult populations when compared to second-line and third-line respectively. Lurasidone is more expensive in terms of treatment costs, resource use (in the stable health state) and the treatment of adverse events. However, these costs are outweighed by the savings associated with the relapse health state. Lurasidone remains cost-saving when inputs are varied in sensitivity analysis and scenario analysis.ConclusionsLurasidone is a cost-saving first-line treatment for schizophrenia for both adolescents and adults.

Prediction of anti-inflammatory peptides by a sequence-based stacking ensemble model named AIPStack

SummaryAccurate and efficient identification of anti-inflammatory peptides (AIPs) is crucial for the treatment of inflammation. Here, we proposed a two-layer stacking ensemble model, AIPStack, to effectively predict AIPs. At first, we constructed a new dataset for model building and validation. Then, peptide sequences were represented by hybrid features, which were fused by two amino acid composition descriptors. Next, the stacking ensemble model was constructed by random forest and extremely randomized tree as the base-classifiers and logistic regression as the meta-classifier to receive the outputs from the base-classifiers. AIPStack achieved an AUC of 0.819, accuracy of 0.755, and MCC of 0.510 on the independent set 3, which were higher than other AIP predictors. Furthermore, the essential sequence features were highlighted by the Shapley Additive exPlanation (SHAP) method. It is anticipated that AIPStack could be used for AIP prediction in a high-throughput manner and facilitate the hypothesis-driven experimental design.

ScBasset: sequence-based modeling of single-cell ATAC-seq using convolutional neural networks.

Single-cell assay for transposase-accessible chromatin using sequencing (scATAC) shows great promise for studying cellular heterogeneity in epigenetic landscapes, but there remain important challenges in the analysis of scATAC data due to the inherent high dimensionality and sparsity. Here we introduce scBasset, a sequence-based convolutional neural network method to model scATAC data. We show that by leveraging the DNA sequence information underlying accessibility peaks and the expressiveness of a neural network model, scBasset achieves state-of-the-art performance across a variety of tasks on scATAC and single-cell multiome datasets, including cell clustering, scATAC profile denoising, data integration across assays and transcription factor activity inference.

AthEDL: Identifying Enhancers in Arabidopsis thaliana Using an Attention-based Deep Learning Method

Background: Enhancers are key cis-function elements of DNA structure that are crucial in gene regulation and the function of a promoter in eukaryotic cells. Availability of accurate identification of the enhancers would facilitate the understanding of DNA functions and their physiological roles. Previous studies have revealed the effectiveness of computational methods for identifying enhancers in other organisms. To date, a huge number of enhancers remain unknown, especially in the field of plant species. Objective: In this study, the aim is to build an efficient attention-based neural network model for the identification of Arabidopsis thaliana enhancers. Method: A sequence-based model using convolutional and recurrent neural networks was proposed for the identification of enhancers. The input DNA sequences are represented as feature vectors by 4-mer. A neural network model consists of CNN and Bi-RNN as sequence feature extractors, and the attention mechanism is suggested to improve the prediction performance. Results: We implemented an ablation study on validation set to select and evaluate the effectiveness of our proposed model. Moreover, our model showed remarkable performance on the test set achieving the Mcc of 0.955, the AUPRC of 0.638, and the AUROC of 0.837, which are significantly higher than state-of-the-art methods, respectively. Conclusion: The proposed computational framework aims at solving similar problems in non-coding genomic regions, thereby providing valuable insights into the prediction about the enhancers of plants.

SeBPPI: A Sequence-Based Protein–Protein Binding Predictor

Protein–protein interaction (PPI) plays an important role in biological processes such as signal transduction, immune response and membrane transport, etc. In this work, a protein sequence-based machine learning model, SeBPPI, to predict protein–protein binding was proposed. In this model, the descriptors were generated from three pre-trained models: Unirep, ESM and TAPE. The performance of SeBPPI with these pre-trained models was evaluated on several different test datasets. The accuracy of our binary prediction model shows improvement over the existing methods. We also compared the performance of two classification heads: The Recurrent convolution neural network (RCNN) and the fully connected neural network (FNN) and found that the use of RCNN is beneficial for the overall improvement in the accuracy of the model. This study helps to improve the accuracy in sequence-based protein–protein binding predictions. The model used in this work is integrated in the web server http://www.icdrug.com/ICDrug/SeBPPI .

Crowding-induced protein destabilization in the absence of soft attractions

It is generally assumed that volume exclusion by macromolecular crowders universally stabilizes the native states of proteins and destabilization suggests soft attractions between crowders and protein. Here we show that proteins can be destabilized even by crowders that are purely repulsive. With a coarse-grained sequence-based model, we study the folding thermodynamics of two sequences with different native folds, a helical hairpin and a β-barrel, in a range of crowder volume fractions, φc. We find that the native state, N, remains structurally unchanged under crowded conditions, while the size of the unfolded state, U, decreases monotonically with φc. Hence, for all φc>0, U is entropically disfavored relative to N. This entropy-centric view holds for the helical hairpin protein, which is stabilized under all crowded conditions as quantified by changes in either the folding midpoint temperature, Tm, or the free energy of folding. We find, however, that the β-barrel protein is destabilized under low-T, low-φc conditions. This destabilization can be understood from two characteristics of its folding: 1) a relatively compact U at T<Tm, such that U is only weakly disfavored entropically by the crowders; and 2) a transient, compact, and relatively low-energy nonnative state that has a maximum population of only a few percent at φc=0, but increasing monotonically with φc. Overall, protein destabilization driven by hard-core effects appears possible when a compaction of U leads to even a modest population of compact nonnative states that are energetically competitive with N.

Characterizing collaborative transcription regulation with a graph-based deep learning approach.

Human epigenome and transcription activities have been characterized by a number of sequence-based deep learning approaches which only utilize the DNA sequences. However, transcription factors interact with each other, and their collaborative regulatory activities go beyond the linear DNA sequence. Therefore leveraging the informative 3D chromatin organization to investigate the collaborations among transcription factors is critical. We developed ECHO, a graph-based neural network, to predict chromatin features and characterize the collaboration among them by incorporating 3D chromatin organization from 200-bp high-resolution Micro-C contact maps. ECHO predicted 2,583 chromatin features with significantly higher average AUROC and AUPR than the best sequence-based model. We observed that chromatin contacts of different distances affected different types of chromatin features' prediction in diverse ways, suggesting complex and divergent collaborative regulatory mechanisms. Moreover, ECHO was interpretable via gradient-based attribution methods. The attributions on chromatin contacts identify important contacts relevant to chromatin features. The attributions on DNA sequences identify TF binding motifs and TF collaborative binding. Furthermore, combining the attributions on contacts and sequences reveals important sequence patterns in the neighborhood which are relevant to a target sequence's chromatin feature prediction.

Novel MILP and CP models for distributed hybrid flowshop scheduling problem with sequence-dependent setup times

As regards distributed hybrid flow shop scheduling with sequence-dependent setup times (DHFSP-SDST), three novel mixed-integer linear programming (MILP) models and a constraint programming (CP) model are formulated for the same-factory and different-factory environments. The three novel MILP models are based on two different modeling ideas. The existing MILP model and the three proposed MILP models are compared in detail from several aspects, such as binary decision variables, continuous decision variables, constraints, solution performance and solution time. By solving the benchmarks in existing studies, the effectiveness and superiority of the proposed MILP and CP models are proved. Experimental results show that the MILP model of sequence-based modeling idea performs best, the MILP model of adjacent sequence-based modeling idea takes the second place and the existing MILP model of position-based modeling idea performs worst. The CP model is more efficient and effective than MILP models. In addition, compared with the existing meta-heuristic algorithms (e.g., DABC and IABC), the proposed MILP models prove the optimal solutions of 37 instances and improve 17 current best solutions. The CP model solves all the 45 instances to optimality and improves 19 current best solutions for benchmarks in the existing studies

A Novel Ensemble Learning-Based Computational Method to Predict Protein-Protein Interactions from Protein Primary Sequences.

Simple SummaryProtein–protein interactions (PPIs) play a central role in the evolution and progression of various biological processes. In this article, we constructed a novel ensemble-learning-based model to predict potential PPIs, which only utilized the protein sequence information. The presented method used Discrete Hilbert transform to extract amino acid sequence information from position-specific scoring matrices. Then these extracted features were fed into rotation forest for training and predicting. When applying our method to the three datasets (Yeast, Human, and Oryza sativa) for detecting PPIs, we obtained excellent prediction performance. Furthermore, the comparison results indicated that our computational model is effective and robust in predicting potential PPI pairs.Protein–protein interactions (PPIs) are crucial for understanding the cellular processes, including signal cascade, DNA transcription, metabolic cycles, and repair. In the past decade, a multitude of high-throughput methods have been introduced to detect PPIs. However, these techniques are time-consuming, laborious, and always suffer from high false negative rates. Therefore, there is a great need of new computational methods as a supplemental tool for PPIs prediction. In this article, we present a novel sequence-based model to predict PPIs that combines Discrete Hilbert transform (DHT) and Rotation Forest (RoF). This method contains three stages: firstly, the Position-Specific Scoring Matrices (PSSM) was adopted to transform the amino acid sequence into a PSSM matrix, which can contain rich information about protein evolution. Then, the 400-dimensional DHT descriptor was constructed for each protein pair. Finally, these feature descriptors were fed to the RoF classifier for identifying the potential PPI class. When exploring the proposed model on the Yeast, Human, and Oryza sativa PPIs datasets, it yielded excellent prediction accuracies of 91.93, 96.35, and 94.24%, respectively. In addition, we also conducted numerous experiments on cross-species PPIs datasets, and the predictive capacity of our method is also very excellent. To further access the prediction ability of the proposed approach, we present the comparison of RoF with four powerful classifiers, including Support Vector Machine (SVM), Random Forest (RF), K-nearest Neighbor (KNN), and AdaBoost. We also compared it with some existing superiority works. These comprehensive experimental results further confirm the excellent and feasibility of the proposed approach. In future work, we hope it can be a supplemental tool for the proteomics analysis.

Sequence-based modeling of three-dimensional genome architecture from kilobase to chromosome scale.

To learn how genomic sequence influences multiscale three-dimensional (3D) genome architecture, this manuscript presents a sequence-based deep-learning approach, Orca, that predicts directly from sequence the 3D genome architecture from kilobase to whole-chromosome scale. Orca captures the sequence dependencies of structures including chromatin compartments and topologically associating domains, as well as diverse types of interactions from CTCF-mediated to enhancer-promoter interactions and Polycomb-mediated interactions with cell-type specificity. Orca enables various applications including predicting structural variant effects on multiscale genome organization and it recapitulated effects of experimentally studied variants at varying sizes (300 bp to 90 Mb). Moreover, Orca enables in silico virtual screens to probe the sequence basis of 3D genome organization at different scales. At the submegabase scale, it predicted specific transcription factor motifs underlying cell-type-specific genome interactions. At the compartment scale, virtual screens of sequence activities suggest a model for the sequence basis of chromatin compartments with a prominent role of transcription start sites.

A hybrid differential evolution algorithm for flexible job shop scheduling with outsourcing operations and job priority constraints

Owing to the increasing complexity of products and the specialization of enterprises, production outsourcing has become a common practice in industrial manufacturing. Moreover, different jobs feature various priorities in actual production. The previous research that aims to minimize the makespan may not be applicable in real scenarios. Therefore, this study investigates a flexible job shop scheduling problem with outsourcing operations and job priority constraints. We propose a sequence-based mathematical model aiming at minimizing weighted overdue days, which considers the outsourcing constraints and different overdue weights of jobs with different priorities. An efficient hybrid self-adaptive differential evolution algorithm with heuristic strategies (HSDE) is proposed to address this problem. In HSDE, a well-designed chromosome encoding and decoding method is presented. To eliminate individuals that do not satisfy the outsourcing constraints, we add a penalty term to improve the objective function. By considering heuristic strategies for initial chromosome generation, the proposed approach is able to achieve a high-quality initial population. Crossover and mutation operators with self-adaptive control of the parameters are established to enlarge the search range and accelerate the convergence speed. Finally, several experiments are conducted to verify the effectiveness of the proposed model and algorithm. Experimental results confirm that the proposed algorithm outperforms other algorithms both in efficiency and accuracy.

Unified Deep Learning Model for Multitask Reaction Predictions with Explanation.

There is significant interest and importance to develop robust machine learning models to assist organic chemistry synthesis. Typically, task-specific machine learning models for distinct reaction prediction tasks have been developed. In this work, we develop a unified deep learning model, T5Chem, for a variety of chemical reaction predictions tasks by adapting the "Text-to-Text Transfer Transformer" (T5) framework in natural language processing (NLP). On the basis of self-supervised pretraining with PubChem molecules, the T5Chem model can achieve state-of-the-art performances for four distinct types of task-specific reaction prediction tasks using four different open-source data sets, including reaction type classification on USPTO_TPL, forward reaction prediction on USPTO_MIT, single-step retrosynthesis on USPTO_50k, and reaction yield prediction on high-throughput C-N coupling reactions. Meanwhile, we introduced a new unified multitask reaction prediction data set USPTO_500_MT, which can be used to train and test five different types of reaction tasks, including the above four as well as a new reagent suggestion task. Our results showed that models trained with multiple tasks are more robust and can benefit from mutual learning on related tasks. Furthermore, we demonstrated the use of SHAP (SHapley Additive exPlanations) to explain T5Chem predictions at the functional group level, which provides a way to demystify sequence-based deep learning models in chemistry. T5Chem is accessible through https://yzhang.hpc.nyu.edu/T5Chem.

LBCEPred: a machine learning model to predict linear B-cell epitopes.

B-cell epitopes have the capability to recognize and attach to the surface of antigen receptors to stimulate the immune system against pathogens. Identification of B-cell epitopes from antigens has a great significance in several biomedical and biotechnological applications, provides support in the development of therapeutics, design and development of an epitope-based vaccine and antibody production. However, the identification of epitopes with experimental mapping approaches is a challenging job and usually requires extensive laboratory efforts. However, considerable efforts have been placed for the identification of epitopes using computational methods in the recent past but deprived of considerable achievements. In this study, we present LBCEPred, a python-based web-tool (http://lbcepred.pythonanywhere.com/), build with random forest classifier and statistical moment-based descriptors to predict the B-cell epitopes from the protein sequences. LBECPred outperforms all sequence-based available models that are currently in use for the B-cell epitopes prediction, with 0.868 accuracy value and 0.934 area under the curve. Moreover, the prediction performance of proposed models compared to other state-of-the-art models is 56.3% higher on average for Mathews Correlation Coefficient. LBCEPred is easy to use tool even for novice users and has also shown the models stability and reliability, thus we believe in its significant contribution to the research community and the area of bioinformatics.

Orca: Sequence-based modeling of genome 3D architecture from kilobase to chromosome-scale (Part3)

Orca: Sequence-based modeling of genome 3D architecture from kilobase to chromosome-scale (Part3)

IThermo: A Sequence-Based Model for Identifying Thermophilic Proteins Using a Multi-Feature Fusion Strategy.

Thermophilic proteins have important application value in biotechnology and industrial processes. The correct identification of thermophilic proteins provides important information for the application of these proteins in engineering. The identification method of thermophilic proteins based on biochemistry is laborious, time-consuming, and high cost. Therefore, there is an urgent need for a fast and accurate method to identify thermophilic proteins. Considering this urgency, we constructed a reliable benchmark dataset containing 1,368 thermophilic and 1,443 non-thermophilic proteins. A multi-layer perceptron (MLP) model based on a multi-feature fusion strategy was proposed to discriminate thermophilic proteins from non-thermophilic proteins. On independent data set, the proposed model could achieve an accuracy of 96.26%, which demonstrates that the model has a good application prospect. In order to use the model conveniently, a user-friendly software package called iThermo was established and can be freely accessed at http://lin-group.cn/server/iThermo/index.html. The high accuracy of the model and the practicability of the developed software package indicate that this study can accelerate the discovery and engineering application of thermally stable proteins.

ASPIRER: a new computational approach for identifying non-classical secreted proteins based on deep learning.

Protein secretion has a pivotal role in many biological processes and is particularly important for intercellular communication, from the cytoplasm to the host or external environment. Gram-positive bacteria can secrete proteins through multiple secretion pathways. The non-classical secretion pathway has recently received increasing attention among these secretion pathways, but its exact mechanism remains unclear. Non-classical secreted proteins (NCSPs) are a class of secreted proteins lacking signal peptides and motifs. Several NCSP predictors have been proposed to identify NCSPs and most of them employed the whole amino acid sequence of NCSPs to construct the model. However, the sequence length of different proteins varies greatly. In addition, not all regions of the protein are equally important and some local regions are not relevant to the secretion. The functional regions of the protein, particularly in the N- and C-terminal regions, contain important determinants for secretion. In this study, we propose a new hybrid deep learning-based framework, referred to as ASPIRER, which improves the prediction of NCSPs from amino acid sequences. More specifically, it combines a whole sequence-based XGBoost model and an N-terminal sequence-based convolutional neural network model; 5-fold cross-validation and independent tests demonstrate that ASPIRER achieves superior performance than existing state-of-the-art approaches. The source code and curated datasets of ASPIRER are publicly available at https://github.com/yanwu20/ASPIRER/. ASPIRER is anticipated to be a useful tool for improved prediction of novel putative NCSPs from sequences information and prioritization of candidate proteins for follow-up experimental validation.