Representation Of Sequences Research Articles

Nested compressed co-representations of multiple sequential experiences during sleep.

Animals encounter and remember multiple experiences daily. During sleep, hippocampal neuronal ensembles replay past experiences and preplay future ones. Although most previous studies investigated p/replay of a single experience, it remains unclear how the hippocampus represents many experiences without major interference during sleep. By monitoring hippocampal neuronal ensembles as rats encountered 15 distinct linear track experiences, we uncovered principles for efficient multi-experience compressed p/replay representation. First, we found a serial position effect whereby the earliest and the most recent experiences had the strongest representations. Second, distinct experiences were co-represented in a multiplexed, flickering manner during nested p/replay events, which greatly enhanced the network's representational capacity. Third, spatially contiguous and disjunct track pairs were bound together into contiguous conjunctive representations during sleep. Finally, sequences spanning day-long multi-track experiences were p/replayed at hyper-compressed ratios during sleep. These coding schemes efficiently parallelize, bind and compress multiple sequential representations with reduced interference and enhanced capacity during sleep.

On leveraging self-supervised learning for accurate HCV genotyping

Hepatitis C virus (HCV) is a major global health concern, affecting millions of individuals worldwide. While existing literature predominantly focuses on disease classification using clinical data, there exists a critical research gap concerning HCV genotyping based on genomic sequences. Accurate HCV genotyping is essential for patient management and treatment decisions. While the neural models excel at capturing complex patterns, they still face challenges, such as data scarcity, that exist a lot in computational genomics. To overcome this challenges, this paper introduces an advanced deep learning approach for HCV genotyping based on the graphical representation of nucleotide sequences that outperforms classical approaches. Notably, it is effective for both partial and complete HCV genomes and addresses challenges associated with imbalanced datasets. In this work, ten HCV genotypes: 1a, 1b, 2a, 2b, 2c, 3a, 3b, 4, 5, and 6 were used in the analysis. This study utilizes Chaos Game Representation for 2D mapping of genomic sequences, employing self-supervised learning using convolutional autoencoder for deep feature extraction, resulting in an outstanding performance for HCV genotyping compared to various machine learning and deep learning models. This baseline provides a benchmark against which the performance of the proposed approach and other models can be evaluated. The experimental results showcase a remarkable classification accuracy of over 99%, outperforming traditional deep learning models. This performance demonstrates the capability of the proposed model to accurately identify HCV genotypes in both partial and complete sequences and in dealing with data scarcity for certain genotypes. The results of the proposed model are compared to NCBI genotyping tool.

VirRep: a hybrid language representation learning framework for identifying viruses from human gut metagenomes

Identifying viruses from metagenomes is a common step to explore the virus composition in the human gut. Here, we introduce VirRep, a hybrid language representation learning framework, for identifying viruses from human gut metagenomes. VirRep combines a context-aware encoder and an evolution-aware encoder to improve sequence representation by incorporating k-mer patterns and sequence homologies. Benchmarking on both simulated and real datasets with varying viral proportions demonstrates that VirRep outperforms state-of-the-art methods. When applied to fecal metagenomes from a colorectal cancer cohort, VirRep identifies 39 high-quality viral species associated with the disease, many of which cannot be detected by existing methods.

Duplications and Retrogenes Are Numerous and Widespread in Modern Canine Genomic Assemblies.

Recent years have seen a dramatic increase in the number of canine genome assemblies available. Duplications are an important source of evolutionary novelty and are also prone to misassembly. We explored the duplication content of nine canine genome assemblies using both genome self-alignment and read-depth approaches. We find that 8.58% of the genome is duplicated in the canFam4 assembly, derived from the German Shepherd Dog Mischka, including 90.15% of unplaced contigs. Highlighting the continued difficulty in properly assembling duplications, less than half of read-depth and assembly alignment duplications overlap, but the mCanLor1.2 Greenland wolf assembly shows greater concordance. Further study shows the presence of multiple segments that have alignments to four or more duplicate copies. These high-recurrence duplications correspond to gene retrocopies. We identified 3,892 candidate retrocopies from 1,316 parental genes in the canFam4 assembly and find that ∼8.82% of duplicated base pairs involve a retrocopy, confirming this mechanism as a major driver of gene duplication in canines. Similar patterns are found across eight other recent canine genome assemblies, with metrics supporting a greater quality of the PacBio HiFi mCanLor1.2 assembly. Comparison between the wolf and other canine assemblies found that 92% of retrocopy insertions are shared between assemblies. By calculating the number of generations since genome divergence, we estimate that new retrocopy insertions appear, on average, in 1 out of 3,514 births. Our analyses illustrate the impact of retrogene formation on canine genomes and highlight the variable representation of duplicated sequences among recently completed canine assemblies.

An initial model construction method constrained by stratigraphic sequence representation for pre‐stack seismic inversion

AbstractThe construction of an accurate and high‐resolution reservoir parameter model is crucial for reservoir characterization. However, due to the band‐limited characteristics of seismic data, the inversion results heavily rely on the accuracy of the initial model. Most existing techniques for constructing an initial model interpolate well logging data within the stratigraphic framework, neglecting the effect of the stratigraphic sequence, which compromises the reliability of the initial model. The stratigraphic sequence is essential for dividing stratigraphic evolution stages and defining a geological relationship between reservoirs within the stratigraphic framework. Therefore, an initial model construction method constrained by stratigraphic sequence representation is proposed for pre‐stack seismic inversion. The process begins with establishing the stratigraphic framework using horizon and fault data. Subsequently, the collaborative sparse representation algorithm is used to learn a joint dictionary that captures the relationship of structural features between seismic data and stratigraphic sequence from the well logging data. In the process of seismic data representation, the stratigraphic sequence is accurately represented in three‐dimensional space by sharing sparse coefficients in the joint dictionary. Finally, the elastic parameter model is constructed by integrating the stratigraphic framework, stratigraphic sequence and well logging data, providing a reliable initial model for pre‐stack seismic inversion. The main innovation of the proposed method is the three‐dimensional representation of the stratigraphic sequence. A synthetic example demonstrates that the proposed method produces a more accurate initial model than conventional interpolation methods. Additionally, when applied to field data, it yields satisfactory results even without complete S‐wave velocity well logging data.

Enhancing generalizability and performance in drug-target interaction identification by integrating pharmacophore and pre-trained models.

In drug discovery, it is crucial to assess the drug-target binding affinity (DTA). Although molecular docking is widely used, computational efficiency limits its application in large-scale virtual screening. Deep learning-based methods learn virtual scoring functions from labeled datasets and can quickly predict affinity. However, there are three limitations. First, existing methods only consider the atom-bond graph or one-dimensional sequence representations of compounds, ignoring the information about functional groups (pharmacophores) with specific biological activities. Second, relying on limited labeled datasets fails to learn comprehensive embedding representations of compounds and proteins, resulting in poor generalization performance in complex scenarios. Third, existing feature fusion methods cannot adequately capture contextual interaction information. Therefore, we propose a novel DTA prediction method named HeteroDTA. Specifically, a multi-view compound feature extraction module is constructed to model the atom-bond graph and pharmacophore graph. The residue concat graph and protein sequence are also utilized to model protein structure and function. Moreover, to enhance the generalization capability and reduce the dependence on task-specific labeled data, pre-trained models are utilized to initialize the atomic features of the compounds and the embedding representations of the protein sequence. A context-aware nonlinear feature fusion method is also proposed to learn interaction patterns between compounds and proteins. Experimental results on public benchmark datasets show that HeteroDTA significantly outperforms existing methods. In addition, HeteroDTA shows excellent generalization performance in cold-start experiments and superiority in the representation learning ability of drug-target pairs. Finally, the effectiveness of HeteroDTA is demonstrated in a real-world drug discovery study. The source code and data are available at https://github.com/daydayupzzl/HeteroDTA.

Molecular sequence classification using efficient kernel based embedding

The alarming spread of diseases across the globe has become a major concern for global healthcare agencies. The research community is actively involved in inventing better and more efficient ways of detecting and treating diseases to solve this global challenge. The abundance of molecular sequence data has eased the path for researchers to develop Machine Learning (ML) based solutions. The performance of the ML models used to classify molecular sequences depends heavily on the type of embedding used to obtain an appropriate numerical representation of the molecular sequences. In recent years, many embedding approaches have been introduced for molecular sequence analysis. However, there is still a need for improvement as far as the efficiency of the methods is concerned (i.e., the ability to capture pairwise relationships and patterns effectively, which could affect the classification performance). To provide a solution to this problem, we propose an efficient kernel-based technique for embedding generation from molecular sequences, which involves computing a kernel matrix using the Sinkhorn-Knopp algorithm and the normalized pairwise distances between k-mers in a manner that satisfies the constraints of a probability distribution. Further, kernel principal component analysis (PCA) is applied to get the top PCs, which are then used as the final embedding. As a result of the experiments, we obtained an ROC-AUC score of 0.657 for our method, which is higher than the scores obtained using baselines. This clearly shows that the low-dimensional embedding obtained through the proposed approach provides an efficient and effective solution for molecular sequence analysis.

The Heterogeneity of Form and Meaning in Chinese Political Discourse Chunk Constructions and its Interpretation in English Translation

The trend of "Translating China" emphasizes the importance of understanding and disseminating the source language. This paper discusses the representation of spatial chunk construction sequences, such as the "4±0" chunk pattern, in political discourse and their heterogeneity in form and meaning, as well as the cognitive interpretation and processing paradigms of translators under the theory of spatial-temporal thinking differences between English and Chinese. The study finds that: 1) The "4±0" chunk pattern and its chunk construction sequences both exhibit strong spatial characteristics in their chunk-like and discrete features; 2) The preference for semantic pairing within the "4±0" chunk pattern is driven by the spatial preferences in Chinese, leading to differences in semantic background; 3) The strong spatial thinking in Chinese influences the dynamic pairing of chunks in the "4±0" sequences, resulting in significant heterogeneity of form and meaning, which creates translation bottlenecks; 4) The spatial-temporal differences between English and Chinese drive the deep interpretation and reprocessing of micro-macro semantic logic within and between chunks in the source language, while constraining the representation of temporal characteristics in the target language translation.

Dynamic 3D model for decoding archaeological complexity of funerary contexts. Phenomena of collapse and fragmentation in an Iron Age burial of Padua (Italy)

Archaeological settings are intrinsically dynamic, undergoing transformations over time that significantly impact the archaeological record. These changes result from both deliberate human interventions and natural degradation processes. Post-depositional phenomena often distort our understanding of ancient contexts, complicating the identification of the original arrangement and hindering the interpretation of archaeological findings. This issue is particularly pronounced in funerary contexts, such as burials, which are susceptible to various natural and human-induced alterations. This study shows the potential of analysing funerary contexts within a processual framework and reconstructing them in a dynamic 3D environment. By employing metrically and morphologically accurate 3D reconstructions, it becomes possible to simulate, isolate, and analyze post-depositional phenomena. The precision of 3D simulation increases significantly when considering factors such as gravity. The goal of this study is to assess changes resulting from transformative phenomena, with a specific focus on creating a sequential representation that elucidates the burial's transformation processes, spanning from deposition to excavation phases.

ESMSec: Prediction of Secreted Proteins in Human Body Fluids Using Protein Language Models and Attention.

The secreted proteins of human body fluid have the potential to be used as biomarkers for diseases. These biomarkers can be used for early diagnosis and risk prediction of diseases, so the study of secreted proteins of human body fluid has great application value. In recent years, the deep-learning-based transformer language model has transferred from the field of natural language processing (NLP) to the field of proteomics, leading to the development of protein language models (PLMs) for protein sequence representation. Here, we propose a deep learning framework called ESM Predict Secreted Proteins (ESMSec) to predict three types of proteins secreted in human body fluid. The ESMSec is based on the ESM2 model and attention architecture. Specifically, the protein sequence data are firstly put into the ESM2 model to extract the feature information from the last hidden layer, and all the input proteins are encoded into a fixed 1000 × 480 matrix. Secondly, multi-head attention with a fully connected neural network is employed as the classifier to perform binary classification according to whether they are secreted into each body fluid. Our experiment utilized three human body fluids that are important and ubiquitous markers. Experimental results show that ESMSec achieved average accuracy of 0.8486, 0.8358, and 0.8325 on the testing datasets for plasma, cerebrospinal fluid (CSF), and seminal fluid, which on average outperform the state-of-the-art (SOTA) methods. The outstanding performance results of ESMSec demonstrate that the ESM can improve the prediction performance of the model and has great potential to screen the secretion information of human body fluid proteins.

Beyond Fidelity: Explaining Vulnerability Localization of Learning-Based Detectors

Vulnerability detectors based on deep learning (DL) models have proven their effectiveness in recent years. However, the shroud of opacity surrounding the decision-making process of these detectors makes it difficult for security analysts to comprehend. To address this, various explanation approaches have been proposed to explain the predictions by highlighting important features, which have been demonstrated effective in domains such as computer vision and natural language processing. Unfortunately, there is still a lack of in-depth evaluation of vulnerability-critical features, such as fine-grained vulnerability-related code lines, learned and understood by these explanation approaches. In this study, we first evaluate the performance of ten explanation approaches for vulnerability detectors based on graph and sequence representations, measured by two quantitative metrics including fidelity and vulnerability line coverage rate. Our results show that fidelity alone is insufficent for evaluating these approaches, as fidelity incurs significant fluctuations across different datasets and detectors. We subsequently check the precision of the vulnerability-related code lines reported by the explanation approaches, and find poor accuracy in this task among all of them. This can be attributed to the inefficiency of explainers in selecting important features and the presence of irrelevant artifacts learned by DL-based detectors.

Using a hybrid neural network architecture for DNA sequence representation: A study on [formula omitted]-methylcytosine sites

N4-methylcytosine (4mC) is a modified form of cytosine found in DNA, contributing to epigenetic regulation. It exists in various genomes, including the Rosaceae family encompassing significant fruit crops like apples, cherries, and roses. Previous investigations have examined the distribution and functional implications of 4mC sites within the Rosaceae genome, focusing on their potential roles in gene expression regulation, environmental adaptation, and evolution. This research aims to improve the accuracy of predicting 4mC sites within the genome of Fragaria vesca, a Rosaceae plant species. Building upon the original 4mc-w2vec method, which combines word embedding processing and a convolutional neural network (CNN), we have incorporated additional feature encoding techniques and leveraged pre-trained natural language processing (NLP) models with different deep learning architectures including different forms of CNN, recurrent neural networks (RNN) and long short-term memory (LSTM). Our assessments have shown that the best model is derived from a CNN model using fastText encoding. This model demonstrates enhanced performance, achieving a sensitivity of 0.909, specificity of 0.77, and accuracy of 0.879 on an independent dataset. Furthermore, our model surpasses previously published works on the same dataset, thus showcasing its superior predictive capabilities.

Sequence-based data-constrained deep learning framework to predict spider dragline mechanical properties

Spider dragline silk is known for its exceptional strength and toughness; hence understanding the link between its primary sequence and mechanics is crucial. Here, we establish a deep-learning framework to clarify this link in dragline silk. The method utilizes sequence and mechanical property data of dragline spider silk as well as enriching descriptors such as residue-level mobility (B-factor) predictions. Our sequence representation captures the relative position, repetitiveness, as well as descriptors of amino acids that serve to physically enrich the model. We obtain high Pearson correlation coefficients (0.76–0.88) for strength, toughness, and other properties, which show that our B-factor based representation outperforms pure sequence-based models or models that use other descriptors. We prove the utility of our framework by identifying influential motifs and demonstrating how the B-factor serves to pinpoint potential mutations that improve strength and toughness, thereby establishing a validated, predictive, and interpretable sequence model for designing tailored biomaterials.

Multifractal Properties of Human Chromosome Sequences

The intricacy and fractal properties of human DNA sequences are examined in this work. The core of this study is to discern whether complete DNA sequences present distinct complexity and fractal attributes compared with sequences containing exclusively exon regions. In this regard, the entire base pair sequences of DNA are extracted from the NCBI (National Center for Biotechnology Information) database. In order to create a time series representation for the base pair sequence {G,C,T,A}, we use the Chaos Game Representation (CGR) approach and a mapping rule f, which enables us to apply the metric known as the Complexity–Entropy Plane (CEP) and multifractal detrended fluctuation analysis (MF-DFA). To carry out our investigation, we divided human DNA into two groups: the first is composed of the 24 chromosomes, which comprises all the base pairs that form the DNA sequence, and another group that also includes the 24 chromosomes, but the DNA sequences rely only on the exons’ presence. The results show that both sets provide fractal patterns in their structure, as obtained by the CGR approach. Complete DNA sequences show a sharper visual fractal pattern than sequences composed only of exons. Moreover, the sequences occupy distinct areas of the complexity–entropy plane, and the complete DNA sequences lead to greater statistical complexity and lower entropy than the exon sequences. Also, we observed that different fractal parameters between chromosomes indicate diversity in genomic sequences. All these results occur in different scales for all chromosomes.

CELA-MFP: a contrast-enhanced and label-adaptive framework for multi-functional therapeutic peptides prediction.

Functional peptides play crucial roles in various biological processes and hold significant potential in many fields such as drug discovery and biotechnology. Accurately predicting the functions of peptides is essential for understanding their diverse effects and designing peptide-based therapeutics. Here, we propose CELA-MFP, a deep learning framework that incorporates feature Contrastive Enhancement and Label Adaptation for predicting Multi-Functional therapeutic Peptides. CELA-MFP utilizes a protein language model (pLM) to extract features from peptide sequences, which are then fed into a Transformer decoder for function prediction, effectively modeling correlations between different functions. To enhance the representation of each peptide sequence, contrastive learning is employed during training. Experimental results demonstrate that CELA-MFP outperforms state-of-the-art methods on most evaluation metrics for two widely used datasets, MFBP and MFTP. The interpretability of CELA-MFP is demonstrated by visualizing attention patterns in pLM and Transformer decoder. Finally, a user-friendly online server for predicting multi-functional peptides is established as the implementation of the proposed CELA-MFP and can be freely accessed at http://dreamai.cmii.online/CELA-MFP.

Self-Supervised pre-training model based on Multi-view for MOOC Recommendation

Recommendation strategies based on concepts of knowledge are gradually applied to personalized course recommendation to promote model learning from implicit feedback data. However, existing approaches typically overlook the prerequisite dependency between concepts, which is the significant basis for connecting courses, and they fail to effectively model the relationship between items and attributes of courses, leading to inadequate capturing of associations between data and ineffective integration of implicit semantics into sequence representations. In this paper, we propose Self-Supervised pre-training model based on Multi-view for MOOC Recommendation (SSM4MR) that exploits non-explicit but inherently correlated features to guide the representation learning of users’ course preferences. In particular, to keep the model from relying solely on course prediction loss and overmphasising on the final performance, we treat the concepts of knowledge, course items and learning paths as different views, then sufficiently model the intrinsic relevance among multi-view through formulating multiple specific self-supervised objectives. As such, our model enhances the sequence representation and ultimately achieves high-performance course recommendation. All the extensive experiments and analyses provide persuasive support for the superiority of the model design and the recommendation results.

Protein sequence analysis in the context of drug repurposing

MotivationDrug repurposing speeds up the development of new treatments, being less costly, risky, and time consuming than de novo drug discovery. There are numerous biological elements that contribute to the development of diseases and, as a result, to the repurposing of drugs.MethodsIn this article, we analysed the potential role of protein sequences in drug repurposing scenarios. For this purpose, we embedded the protein sequences by performing four state of the art methods and validated their capacity to encapsulate essential biological information through visualization. Then, we compared the differences in sequence distance between protein-drug target pairs of drug repurposing and non - drug repurposing data. Thus, we were able to uncover patterns that define protein sequences in repurposing cases.ResultsWe found statistically significant sequence distance differences between protein pairs in the repurposing data and the rest of protein pairs in non-repurposing data. In this manner, we verified the potential of using numerical representations of sequences to generate repurposing hypotheses in the future.

It’s not distance but similarity of distance: changing stimulus relations affect the control of action sequences

Interacting with our environment happens on different levels of complexity: While there are individual and simple actions like an isolated button press, most actions are more complex and involve sequences of simpler actions. The degree to which multiple simple actions are represented as one action sequence can be measured via so-called response-response binding effects. When two or more responses are executed consecutively, they are integrated into one representation so that repetition of one response can start retrieval of the other. Executing such an action sequence typically involves responding to multiple objects or stimuli. Here, we investigated whether the spatial relation of these stimuli affects action sequence execution. To that end, we varied the distance between stimuli in a response-response binding task. Stimulus distance might affect response-response binding effects in one of two ways: It might directly affect the representation of the response sequence, making integration and retrieval between responses more likely if the responses relate to close stimuli. Alternatively, the similarity of stimulus distribution during integration and retrieval might be decisive, leading to larger binding effects if stimulus distance is identical during integration and retrieval. We found stronger binding effects with constant than with changing stimulus distance, indicating that action integration and retrieval can easily affect performance also if responses refer to separated objects. However, this effect on performance is diminished by changing spatial distribution of stimuli at the times of integration and retrieval.

Comparing natural language processing representations of coded disease sequences for prediction in electronic health records.

Natural language processing (NLP) algorithms are increasingly being applied to obtain unsupervised representations of electronic health record (EHR) data, but their comparative performance at predicting clinical endpoints remains unclear. Our objective was to compare the performance of unsupervised representations of sequences of disease codes generated by bag-of-words versus sequence-based NLP algorithms at predicting clinically relevant outcomes. This cohort study used primary care EHRs from 6286233 people with Multiple Long-Term Conditions in England. For each patient, an unsupervised vector representation of their time-ordered sequences of diseases was generated using 2 input strategies (212 disease categories versus 9462 diagnostic codes) and different NLP algorithms (Latent Dirichlet Allocation, doc2vec, and 2 transformer models designed for EHRs). We also developed a transformer architecture, named EHR-BERT, incorporating sociodemographic information. We compared the performance of each of these representations (without fine-tuning) as inputs into a logistic classifier to predict 1-year mortality, healthcare use, and new disease diagnosis. Patient representations generated by sequence-based algorithms performed consistently better than bag-of-words methods in predicting clinical endpoints, with the highest performance for EHR-BERT across all tasks, although the absolute improvement was small. Representations generated using disease categories perform similarly to those using diagnostic codes as inputs, suggesting models can equally manage smaller or larger vocabularies for prediction of these outcomes. Patient representations produced by sequence-based NLP algorithms from sequences of disease codes demonstrate improved predictive content for patient outcomes compared with representations generated by co-occurrence-based algorithms. This suggests transformer models may be useful for generating multi-purpose representations, even without fine-tuning.

Biometric identity recognition based on contrastive positive-unlabeled learning

With the development of neural networks, identity recognition techniques utilizing human physiological signals have found applications in healthcare systems. However, many current physiological signal datasets have to spend a lot of resources in the pre-labeling phase, which leads to longer training times. To reduce the cost of labeling, we propose a self-supervised time-series representation learning framework based on contrastive positive-unlabeled learning (TS − CPUL), to learn the representations of temporal neighborhood sequences. To obtain the representations, the unit root test divides the unlabeled time series into neighborhood samples and unlabeled samples. Secondly, we use neighborhood sample representations and unlabeled sample representations to do self-supervised contrastive learning and positive-unlabeled learning with a negative sample selector. Both approaches can deepen the model’s understanding of the sample representation. Lastly, to enhance the model’s capability in discriminating between ambiguous sample pairs, we design the distance-polarized matrix. The proposed TS-CPUL is evaluated through experiments on three physiological signal datasets. The results demonstrate its effectiveness in understanding time-series representations and performing well in classification. For the identity recognition task, we achieve accuracy comparable to the supervised model by fine-tuning the model with only 10% of the labeled data.