Representation Of Sequences Research Articles

Pangenomic analyses of tuberculosis strains to identify resistomes using computational approaches.

To locate resistomes in tuberculosis strains, to determine the severity of drug resistance, and to infer its implications with respect to high tuberculosis prevalence in a Third World setting. The pangenomic study was conducted from October 2022 to January 2023 in Sir Syed University of Engineering and Technology, Karachi, and comprised 2012-22 data on multiple sequence alignment to assess the genetic evolution of tuberculosis strains. Antibiotic resistance drug classes were identified using the Canadian Antibiotic Resistance Database, which entailed multidrug-resistant and extremely drug-resistant strains. Also, GenBank was used for tuberculosis genome FASTA (fast-all; nucleotide and protein sequence representation) files, prediction of resistome sequences on the basis of Canadian Antibiotic Resistance Database, and multiple sequence alignment was done in Mauve. Evolutionarily, the 6 strains identified were structurally similar with polymorphisms in their core chromosomal regions. Their resistome genes showed perfect hits for isoniazid, rifamycin, cephalosporin, fluoroquinolone, aminoglycosides, penem, penam and cephamycin. Drugs discovered in antibiotic resistance genes are now less effective in treatment, and have the potential to develop into more dangerous bacteria, if not monitored. For treatment, staying long durations in hospitals for quality healthcare and supervision in third world countries is unaffordable.

Triple Sequence Learning for Cross-domain Recommendation

Cross-domain recommendation (CDR) aims at leveraging the correlation of users’ behaviors in both the source and target domains to improve the user preference modeling in the target domain. Conventional CDR methods typically explore the dual-relations between the source and target domains’ behaviors. However, this may ignore the informative mixed behaviors that naturally reflect the user’s global preference. To address this issue, we present a novel framework, termed triple sequence learning for cross-domain recommendation (Tri-CDR), which jointly models the source, target, and mixed behavior sequences to highlight the global and target preference and precisely model the triple correlation in CDR. Specifically, Tri-CDR independently models the hidden representations for the triple behavior sequences and proposes a triple cross-domain attention (TCA) method to emphasize the informative knowledge related to both user’s global and target-domain preference. To comprehensively explore the cross-domain correlations, we design a triple contrastive learning (TCL) strategy that simultaneously considers the coarse-grained similarities and fine-grained distinctions among the triple sequences, ensuring the alignment while preserving information diversity in multi-domain. We conduct extensive experiments and analyses on six cross-domain settings. The significant improvements of Tri-CDR with different sequential encoders verify its effectiveness and universality. The source code is available at https://github.com/hulkima/Tri-CDR .

Dynamic predictive coding: A model of hierarchical sequence learning and prediction in the neocortex.

We introduce dynamic predictive coding, a hierarchical model of spatiotemporal prediction and sequence learning in the neocortex. The model assumes that higher cortical levels modulate the temporal dynamics of lower levels, correcting their predictions of dynamics using prediction errors. As a result, lower levels form representations that encode sequences at shorter timescales (e.g., a single step) while higher levels form representations that encode sequences at longer timescales (e.g., an entire sequence). We tested this model using a two-level neural network, where the top-down modulation creates low-dimensional combinations of a set of learned temporal dynamics to explain input sequences. When trained on natural videos, the lower-level model neurons developed space-time receptive fields similar to those of simple cells in the primary visual cortex while the higher-level responses spanned longer timescales, mimicking temporal response hierarchies in the cortex. Additionally, the network's hierarchical sequence representation exhibited both predictive and postdictive effects resembling those observed in visual motion processing in humans (e.g., in the flash-lag illusion). When coupled with an associative memory emulating the role of the hippocampus, the model allowed episodic memories to be stored and retrieved, supporting cue-triggered recall of an input sequence similar to activity recall in the visual cortex. When extended to three hierarchical levels, the model learned progressively more abstract temporal representations along the hierarchy. Taken together, our results suggest that cortical processing and learning of sequences can be interpreted as dynamic predictive coding based on a hierarchical spatiotemporal generative model of the visual world.

Horadam Spinors

Spinors can be expressed as Lie algebra of infinitesimal rotations. Spinors are also defined as elements of a vector space which carries a linear representation of the Clifford algebra typically. The motivation for this study is to define a new and particular sequence. An essential feature of this sequence is that while a generalization is being made, spinors, which have a lot of use in physics, are used. This new sequence defined using spinor representations is called the Horadam spinor sequence; formulas such as the Binet formula, generating function formula, and Cassini formula are given. The Horadam spinors given in this study are a generalization of the spinor representations of Horadam quaternion sequences.

Emotional arousal lingers in time to bind discrete episodes in memory

ABSTRACT Temporal stability and change in neutral contexts can transform continuous experiences into distinct and memorable events. However, less is known about how shifting emotional states influence these memory processes, despite ample evidence that emotion impacts non-temporal aspects of memory. Here, we examined if emotional stimuli influence temporal memory for recent event sequences. Participants encoded lists of neutral images while listening to auditory tones. At regular intervals within each list, participants heard emotional positive, negative, or neutral sounds, which served as “emotional event boundaries” that divided each sequence into discrete events. Temporal order memory was tested for neutral item pairs that either spanned an emotional sound or were encountered within the same auditory event. Encountering a highly arousing event boundary led to faster response times for items encoded within the next event. Critically, we found that highly arousing sounds had different effects on binding ongoing versus ensuing sequential representations in memory. Specifically, highly arousing sounds were significantly more likely to enhance temporal order memory for ensuing information compared to information that spanned those boundaries, especially for boundaries with negative valence. These findings suggest that within aversive emotional contexts, fluctuations in arousal help shape the temporal organisation of events in memory.

Interpretable feature extraction and dimensionality reduction in ESM2 for protein localization prediction.

As the application of large language models (LLMs) has broadened into the realm of biological predictions, leveraging their capacity for self-supervised learning to create feature representations of amino acid sequences, these models have set a new benchmark in tackling downstream challenges, such as subcellular localization. However, previous studies have primarily focused on either the structural design of models or differing strategies for fine-tuning, largely overlooking investigations into the nature of the features derived from LLMs. In this research, we propose different ESM2 representation extraction strategies, considering both the character type and position within the ESM2 input sequence. Using model dimensionality reduction, predictive analysis and interpretability techniques, we have illuminated potential associations between diverse feature types and specific subcellular localizations. Particularly, the prediction of Mitochondrion and Golgi apparatus prefer segments feature closer to the N-terminal, and phosphorylation site-based features could mirror phosphorylation properties. We also evaluate the prediction performance and interpretability robustness of Random Forest and Deep Neural Networks with varied feature inputs. This work offers novel insights into maximizing LLMs' utility, understanding their mechanisms, and extracting biological domain knowledge. Furthermore, we have made the code, feature extraction API, and all relevant materials available at https://github.com/yujuan-zhang/feature-representation-for-LLMs.

PREDAC-CNN: predicting antigenic clusters of seasonal influenza A viruses with convolutional neural network.

Vaccination stands as the most effective and economical strategy for prevention and control of influenza. The primary target of neutralizing antibodies is the surface antigen hemagglutinin (HA). However, ongoing mutations in the HA sequence result in antigenic drift. The success of a vaccine is contingent on its antigenic congruence with circulating strains. Thus, predicting antigenic variants and deducing antigenic clusters of influenza viruses are pivotal for recommendation of vaccine strains. The antigenicity of influenza A viruses is determined by the interplay of amino acids in the HA1 sequence. In this study, we exploit the ability of convolutional neural networks (CNNs) to extract spatial feature representations in the convolutional layers, which can discern interactions between amino acid sites. We introduce PREDAC-CNN, a model designed to track antigenic evolution of seasonal influenza A viruses. Accessible at http://predac-cnn.cloudna.cn, PREDAC-CNN formulates a spatially oriented representation of the HA1 sequence, optimized for the convolutional framework. It effectively probes interactions among amino acid sites in the HA1 sequence. Also, PREDAC-CNN focuses exclusively on physicochemical attributes crucial for the antigenicity of influenza viruses, thereby eliminating unnecessary amino acid embeddings. Together, PREDAC-CNN is adept at capturing interactions of amino acid sites within the HA1 sequence and examining the collective impact of point mutations on antigenic variation. Through 5-fold cross-validation and retrospective testing, PREDAC-CNN has shown superior performance in predicting antigenic variants compared to its counterparts. Additionally, PREDAC-CNN has been instrumental in identifying predominant antigenic clusters for A/H3N2 (1968-2023) and A/H1N1 (1977-2023) viruses, significantly aiding in vaccine strain recommendation.

ProkBERT family: genomic language models for microbiome applications.

In the evolving landscape of microbiology and microbiome analysis, the integration of machine learning is crucial for understanding complex microbial interactions, and predicting and recognizing novel functionalities within extensive datasets. However, the effectiveness of these methods in microbiology faces challenges due to the complex and heterogeneous nature of microbial data, further complicated by low signal-to-noise ratios, context-dependency, and a significant shortage of appropriately labeled datasets. This study introduces the ProkBERT model family, a collection of large language models, designed for genomic tasks. It provides a generalizable sequence representation for nucleotide sequences, learned from unlabeled genome data. This approach helps overcome the above-mentioned limitations in the field, thereby improving our understanding of microbial ecosystems and their impact on health and disease. ProkBERT models are based on transfer learning and self-supervised methodologies, enabling them to use the abundant yet complex microbial data effectively. The introduction of the novel Local Context-Aware (LCA) tokenization technique marks a significant advancement, allowing ProkBERT to overcome the contextual limitations of traditional transformer models. This methodology not only retains rich local context but also demonstrates remarkable adaptability across various bioinformatics tasks. In practical applications such as promoter prediction and phage identification, the ProkBERT models show superior performance. For promoter prediction tasks, the top-performing model achieved a Matthews Correlation Coefficient (MCC) of 0.74 for E. coli and 0.62 in mixed-species contexts. In phage identification, ProkBERT models consistently outperformed established tools like VirSorter2 and DeepVirFinder, achieving an MCC of 0.85. These results underscore the models' exceptional accuracy and generalizability in both supervised and unsupervised tasks. The ProkBERT model family is a compact yet powerful tool in the field of microbiology and bioinformatics. Its capacity for rapid, accurate analyses and its adaptability across a spectrum of tasks marks a significant advancement in machine learning applications in microbiology. The models are available on GitHub (https://github.com/nbrg-ppcu/prokbert) and HuggingFace (https://huggingface.co/nerualbioinfo) providing an accessible tool for the community.

Discovering genomic islands in unannotated bacterial genomes using sequence embedding.

Genomic islands (GEIs) are clusters of genes in bacterial genomes that are typically acquired by horizontal gene transfer. GEIs play a crucial role in the evolution of bacteria by rapidly introducing genetic diversity and thus helping them adapt to changing environments. Specifically of interest to human health, many GEIs contain pathogenicity and antimicrobial resistance genes. Detecting GEIs is, therefore, an important problem in biomedical and environmental research. There have been many previous studies for computationally identifying GEIs. Still, most of these studies rely on detecting anomalies in the unannotated nucleotide sequences or on a fixed set of known features on annotated nucleotide sequences. Here, we present TreasureIsland, which uses a new unsupervised representation of DNA sequences to predict GEIs. We developed a high-precision boundary detection method featuring an incremental fine-tuning of GEI borders, and we evaluated the accuracy of this framework using a new comprehensive reference dataset, Benbow. We show that TreasureIsland's accuracy rivals other GEI predictors, enabling efficient and faster identification of GEIs in unannotated bacterial genomes. TreasureIsland is available under an MIT license at: https://github.com/FriedbergLab/GenomicIslandPrediction.

PseAAC2Vec protein encoding for TCR protein sequence classification

The classification and prediction of T-cell receptors (TCRs) protein sequences are of significant interest in understanding the immune system and developing personalized immunotherapies. In this study, we propose a novel approach using Pseudo Amino Acid Composition (PseAAC) protein encoding for accurate TCR protein sequence classification. The PseAAC2Vec encoding method captures the physicochemical properties of amino acids and their local sequence information, enabling the representation of protein sequences as fixed-length feature vectors. By incorporating physicochemical properties such as hydrophobicity, polarity, charge, molecular weight, and solvent accessibility, PseAAC2Vec provides a comprehensive and informative characterization of TCR protein sequences. To evaluate the effectiveness of the proposed PseAAC2Vec encoding approach, we assembled a large dataset of TCR protein sequences with annotated classes. We applied the PseAAC2Vec encoding scheme to each sequence and generated feature vectors based on a specified window size. Subsequently, we employed state-of-the-art machine learning algorithms, such as support vector machines (SVM) and random forests (RF), to classify the TCR protein sequences. Experimental results on the benchmark dataset demonstrated the superior performance of the PseAAC2Vec-based approach compared to existing methods. The PseAAC2Vec encoding effectively captures the discriminative patterns in TCR protein sequences, leading to improved classification accuracy and robustness. Furthermore, the encoding scheme showed promising results across different window sizes, indicating its adaptability to varying sequence contexts.

TFCSRec: Time–frequency consistency based contrastive learning for sequential recommendation

Sequential recommendation aims to predict future user interactions by analyzing dynamic patterns within their historical behavior sequences. Deep neural networks have recently become popular for learning representations of these sequences in the time domain. However, representing users’ intentions in the time domain faces challenges such as noise in interactions and sparsity of data. Contrastive learning and representation learning in the frequency domain can mitigate these issues from different perspectives. In this paper, to fully integrate time-domain sequence representations, frequency-domain sequence representations and contrastive learning based on them, we propose a model called Time–Frequency Consistency based contrastive learning for Sequential Recommendation (TFCSRec). TFCSRec utilizes a time-domain encoder with a fully connected network and a filter network to extract high-order features and catch pure sequential patterns. Then, a learnable frequency-domain encoder with a recurrent neural network is designed to capture sequential characteristics in the frequency-domain space. Finally, TFCSRec combines a recommendation task and two contrastive learning tasks to optimize the two user representation encoders. Its contrastive learning is designed to minimize a contrastive regularization loss and a time–frequency consistency loss, which for the first time is constructed directly on the time-domain sequence representation and the frequency-domain sequence representation. Experiments on five benchmark datasets show that the proposed TFCSRec model outperforms other sequential recommendation models based on deep neural networks.

A hybrid CNN–vision transformer structure for remote sensing scene classification

ABSTRACT Vision Transformers (ViTs) have become one of the main architectures in deep learning with the self-attention mechanism, and are becoming an alternative to Convolutional Neural Networks (CNNs) for remote sensing scene classification tasks. However, the earlier self-attention layer of ViTs focuses on local features rather than global features, and the deeper self-attention layer focuses on global features but ignores the different impact of different frequency information. This will greatly increase the training and computational cost due to the quadratic complexity of the self-attention mechanism on the long sequence representation. In this paper, we propose a hybrid CNN – vision transformer structure (HCVNet), which uses convolutional layers to replace the earlier self-attention layers, and a novel Frequency Multi-head Self Attention (F-MSA) mechanism to replace the deeper self-attention layers. Specifically, F-MSA is a dual-stream structure that reduces the computational cost and improves the classification performance by encoding the high/low frequency information separately. In addition, a Semantic-aware Localization (SaL) module is introduced, which can guide the selection of crop by learning prior knowledge, avoiding the issue of pure background sampling. Our method performed an accuracy value of 97.20 ± 0.02% on the Aerial Image Dataset and 93.89 ± 0.03% on the NWPU-RESISC45 Dataset, with low complexity costs.

On the representation of an exponential type sequence

On the representation of an exponential type sequence

Polarization- and CGR-based binary representations as identifiers of the nucleotide sequences in bioinformatics

Purpose of this work is the comparative analysis of two approaches to the synthesis of two-dimensional binary identifiers of nucleotide sequences obtained using DNA sequencing of biological objects. Methods. One of the approaches is based on modeling the polarization-dependent diffraction of a coherent readout beam on a two-dimensional phase-modulating structure (phase screen) associated with the symbolic sequence obtained as a result of DNA sequencing. Another approach uses a two-dimensional representation of the symbolic sequence using a chaos game representation (CGR). To obtain a finite-element CGR mapping, it is fragmented into a given number of cells, ensuring acceptable sensitivity of the synthesized binary identifier to structural changes in the displayed sequence. Results. The comparative analysis was carried out using fragments of symbol sequences corresponding to various strains (Wuhan, Delta, Omicron) of the SarSCoV2 virus. In the course of the analysis, the correlation coefficients between the binary identifiers corresponding to various strains were obtained and compared with each other. Conclusion. It has been established that binary identifiers synthesized using the polarization encoding technique are characterized by significantly higher sensitivity to structural changes in the analyzed sequences and smaller sizes compared to CGR binary identifiers.

Cortical ROI importance improves MI decoding from EEG using fused light neural network.

Decoding motor imagery (MI) using deep learning in cortical level has potential in brain computer interface based intelligent rehabilitation. However, a mass of dipoles is inconvenient to extract the personalized features and requires a more complex neural network. In consideration of the structural and functional similarity of the neurons in a neuroanatomical region, i.e., a region of interest (ROI), we propose that the comprehensive performance of each ROI may be reflected by a specific representative dipole (RD), and the time-frequency spectrums of all RDs are applied simultaneously to Random Forest algorithm to give a quantitative metric of each ROI importance (RI). Then, the more divided sub-band spectral powers are reinforced by RI, and they are interpolated to a 2-dimensional (2D) plane transformed from 3D space of all RDs, yielding an ensemble representation of RD feature image sequences (ERDFIS). Furthermore, a lightweight network, including 2D separable convolution and gated recurrent unit (2DSCG), is developed to extract and classify the frequency-spatial and temporal features from ERDFIS, forming a novel MI decoding method in cortical level (called ERDFIS-2DSCG). Based on two public datasets, the decoding accuracies of ten-fold cross-validation are 89.89% and 94.35%, respectively. The results suggest that RD can embody the overall property of ROI in time-frequency-space domains, and ROI importance is helpful to highlight the subject-based characteristics of MI-EEG. Meanwhile, 2DSCG is matched well with ERDFIS, jointly improving the decoding performance.

Improved 3-D Representation of GPR Pipelines B-Scan Sequences Using a Neural Network Framework

Improved 3-D Representation of GPR Pipelines B-Scan Sequences Using a Neural Network Framework

Neuro or Symbolic? Fine-tuned Transformer with Unsupervised LDA Topic Clustering for Text Sentiment Analysis

For text sentiment analysis, state-of-the-art neural language models have demonstrated promising performance. However, they lack interpretability, require vast volumes of annotated data, and are typically specialized for tasks. In this paper, we explore a connection between fine-tuned Transformer models and unsupervised LDA approach to cope with text sentiment analysis tasks, inspired by the concept of Neuro-symbolic AI. The Transformer and LDA models are combined as a feature extractor to extract the hidden representations of the input text sequences. Subsequently, we employ a feedforward network to forecast various sentiment analysis tasks, such as multi-label emotion prediction, dialogue quality prediction, and nugget detection. Our proposed method obtains the best results in the NTCIR-16 dialogue evaluation (DialEval-2) task, as well as cutting-edge results in emotional intensity prediction using the Ren_CECps corpus. Extensive experiments show that our proposed method is highly explainable, cost-effective in training, and superior in terms of accuracy and robustness.

Representations of Numerical Sequences in Mexican Secondary School Textbooks

Representations of Numerical Sequences in Mexican Secondary School Textbooks

\(q\)-Analogue of the Generalized Fibonacci and Lucas Polynomials

In this article, we define \(q\)-generalized Fibonacci polynomials and \(q\)-generalized Lucas polynomials using \(q\)-binomial coefficient and obtain their recursive properties. In addition, we introduce generalized \(q\)-Fibonacci matrix and generalized \(q\)-Lucas matrix, then we derive their basic identities. We define \((k,q,t)\)-symmetric generalized Fibonacci matrix and \((k,q,t)\)-symmetric generalized Lucas matrix, then we give the Cholesky factorization of these matrices. Finally, we give determinantal and permanental representations of these new polynomial sequences.

Dynamic Dense Graph Convolutional Network for Skeleton-Based Human Motion Prediction.

Graph Convolutional Networks (GCN) which typically follows a neural message passing framework to model dependencies among skeletal joints has achieved high success in skeleton-based human motion prediction task. Nevertheless, how to construct a graph from a skeleton sequence and how to perform message passing on the graph are still open problems, which severely affect the performance of GCN. To solve both problems, this paper presents a Dynamic Dense Graph Convolutional Network (DD-GCN), which constructs a dense graph and implements an integrated dynamic message passing. More specifically, we construct a dense graph with 4D adjacency modeling as a comprehensive representation of motion sequence at different levels of abstraction. Based on the dense graph, we propose a dynamic message passing framework that learns dynamically from data to generate distinctive messages reflecting sample-specific relevance among nodes in the graph. Extensive experiments on benchmark Human 3.6M and CMU Mocap datasets verify the effectiveness of our DD-GCN which obviously outperforms state-of-the-art GCN-based methods, especially when using long-term and our proposed extremely long-term protocol.