Long-range Associations Research Articles

Optimizing protein sequence classification: integrating deep learning models with Bayesian optimization for enhanced biological analysis

Efforts to enhance the accuracy of protein sequence classification are of utmost importance in driving forward biological analyses and facilitating significant medical advancements. This study presents a cutting-edge model called ProtICNN-BiLSTM, which combines attention-based Improved Convolutional Neural Networks (ICNN) and Bidirectional Long Short-Term Memory (BiLSTM) units seamlessly. Our main goal is to improve the accuracy of protein sequence classification by carefully optimizing performance through Bayesian Optimisation. ProtICNN-BiLSTM combines the power of CNN and BiLSTM architectures to effectively capture local and global protein sequence dependencies. In the proposed model, the ICNN component uses convolutional operations to identify local patterns. Captures long-range associations by analyzing sequence data forward and backwards. In advanced biological studies, Bayesian Optimisation optimizes model hyperparameters for efficiency and robustness. The model was extensively confirmed with PDB-14,189 and other protein data. We found that ProtICNN-BiLSTM outperforms traditional categorization models. Bayesian Optimization’s fine-tuning and seamless integration of local and global sequence information make it effective. The precision of ProtICNN-BiLSTM improves comparative protein sequence categorization. The study improves computational bioinformatics for complex biological analysis. Good results from the ProtICNN-BiLSTM model improve protein sequence categorization. This powerful tool could improve medical and biological research. The breakthrough protein sequence classification model is ProtICNN-BiLSTM. Bayesian optimization, ICNN, and BiLSTM analyze biological data accurately.

A Novel Model Combining Transformer and Bi-LSTM for News Categorization

News categorization (NC), the aim of which is to identify distinct categories of news through analyzing the contents, has acquired substantial progress since deep learning was introduced into the natural language processing (NLP) field. As a state-of-art model, transformer’s classification performance is not satisfied compared with recurrent neural network (RNN) and convolutional neural network (CNN) if it does not get pretrained. Based on the transformer model, this article proposes a novel framework that combines bidirectional long short-term memory (Bi-LSTM) network and transformer to solve this problem. In the suggested framework, the self-attention mechanism is substituted with Bi-LSTM to capture the semantic information from sentences. Meanwhile, an attention mechanism model is applied to focus on those important words and adjust their weights to solve the problem of long-distance information loss. With pooling network, the network complexity can be reduced and the main features can be highlighted by halving the dimension of the hidden state. Finally, after acquiring the hidden representation by the above structures, we utilize a contraction network to further capture the long-range associations from a text. Experiments on three large-scale corpora were performed to evaluate the suggested framework, and the results demonstrate that our model outperforms other models such as deep pyramid CNN (DPCNN), transformer.

Prior-guided adaptive generative adversarial network method for various types of borehole image inpainting

During geophysical exploration, inpainting defective logging images caused by mismatches between logging tools and borehole sizes can affect fracture and hole identification, petrographic analysis, and stratigraphic studies. However, existing methods do not describe sufficient stratigraphic continuity. In addition, they ignore the completeness of characterization in terms of fractures, gravel structures, and fine-grained textures in the logging images. To address these issues, we develop a deep-learning method for inpainting stratigraphic features. First, to enhance the continuity of image inpainting, we build a generative adversarial network and train it on numerous natural images to extract relevant features that guide the recovery of continuity characteristics. Second, to ensure that complete structural and textural features are found in geologic formations, we introduce a feature-extraction-fusion module with a cooccurrence mechanism consisting of channel attention (CA) and self-attention (SA). CA improves texture effects by adaptively adjusting control parameters based on highly correlated prior features from electrical logging images. SA captures long-range contextual associations across preinpainted gaps to improve completeness in fractures and gravel structure representation. Our method has been tested on various borehole images demonstrating its reliability and robustness.

Abstract 892: Pan-cancer tumor microenvironment profiling with multiplexed immunofluorescence microscopy and self-supervised learning

Abstract Multiplexed immunofluorescence (mIF) microscopy reveals the spatial architecture of cancer tissue and its microenvironment that is not being fully explored with existing analysis methods. Most analysis approaches for mIF microscopy data focus on single-cell or region classification and rely on supervised machine learning. In this work, we present a self-supervised spatial profiling method for mIF images that takes local and global associations into account, and apply this method to profile cancer-associated fibroblasts (CAFs) in a pan-cancer dataset. We studied a 2-stage self-supervised training scheme to learn the representations of mIF tissue microarray (TMA) images in local and global scales (cellular and long-range associations). During the 1st self-supervised training stage, a Vision Transformer learns the local-scale representations by using small patches from TMA images. Then, the local representations are used as input to the 2nd stage where a similar self-supervised learning strategy is used to learn global patterns in the TMA images. We applied the method to profile multiple cohorts from three different solid tumors: prostate, renal, and lung cancer. In total, these cohorts include more than 5,000 TMA cores from over 1,750 patients extracted from the tumor center, tumor edge, and adjacent benign areas. The samples were stained with a CAF panel including FAP, aSMA, PDGFRB, pSTAT3/PDGFRA, nuclear and epithelial markers, and imaged with cyclic mIF microscopy. Samples were studied at the patch-level and TMA core-level. Small patches enable further analysis of associations in the local environment, whereas the core-level enables associations with patient clinical information. Clustering of patch-level (1st stage) and core-level (2nd stage) representations showed independently that self-supervised learning is capable of learning the representations of the mIF images. We were able to identify regions and cases with high pTNM staging from the prostate cancer samples and similar histological subtypes from renal cancer samples. We further validated the clustering using k-NN classification that showed high classification accuracy in all cohorts. Moreover, we developed a stopping criteria for the final model selection that balances between the similarities of samples and patches inside of samples to prevent overfitting. Our study shows that self-supervised learning enables unbiased discoveries from large-scale mIF microscopy imaging datasets. The developed method uncovers associations between imaging data and clinical information and highlights directly the patterns that are most meaningful for these associations. Citation Format: Gantugs Atarsaikhan, Isabel Mogollon, Katja Välimäki, Tuomas Mirtti, Teijo Pellinen, Lassi Paavolainen. Pan-cancer tumor microenvironment profiling with multiplexed immunofluorescence microscopy and self-supervised learning [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 892.

Subdomain-Alignment Data Augmentation for Pipeline Fault Diagnosis: An Adversarial Self-Attention Network

Data augmentation (DA) has the potential to address the issue of imbalanced and insufficient datasets (I&ID) in pipeline fault diagnosis. However, the majority of existing DA methods for time series are inspired by computer vision techniques, ignoring the temporal dynamic properties and fine-grained fault features, which leads to limited performance of the augmentation. To tackle this problem, we introduce a novel DA approach called the subdomain-alignment adversarial self-attention network (SA-ASN), which takes into account both temporal association and semantic correlation. Our approach features a novel temporal association learning (TAL) mechanism, which transfers temporal information from the discriminator to the generator via a customized knowledge-sharing structure, improving the reliability of synthetic long-range associations. Additionally, we introduce a prototype-assisted subdomain alignment (PASA) strategy that forms a hierarchical structure in the synthetic dataset by incorporating local semantic correlation into the model training. With the support of TAL and PASA, our SA-ASN algorithm enhances the authenticity of temporal structure at the instance level and improves the discriminability of fault features at the category level. Our experimental results show that the SA-ASN algorithm provides a more diverse and accurate augmentation of pipeline data. The effectiveness of our SA-ASN algorithm encourages the use of data-driven diagnostic models in complex real-world oilfield pipeline networks.

Prot2GO: Predicting GO Annotations From Protein Sequences and Interactions.

Protein is the main material basis of living organisms and plays crucial role in life activities. Understanding the function of protein is of great significance for new drug discovery, disease treatment and vaccine development. In recent years, with the widespread application of deep learning in bioinformatics, researchers have proposed many deep learning models to predict protein functions. However, the existing deep learning methods usually only consider protein sequences, and thus cannot effectively integrate multi-source data to annotate protein functions. In this article, we propose the Prot2GO model, which can integrate protein sequence and PPI network data to predict protein functions. We utilize an improved biased random walk algorithm to extract the features of PPI network. For sequence data, we use a convolutional neural network to obtain the local features of the sequence and a recurrent neural network to capture the long-range associations between amino acid residues in protein sequence. Moreover, Prot2GO adopts the attention mechanism to identify protein motifs and structural domains. Experiments show that Prot2GO model achieves the state-of-the-art performance on multiple metrics.

Chromosome-level organization of the regulatory genome in the Drosophila nervous system

Previous studies have identified topologically associating domains (TADs) as basic units of genome organization. We present evidence of a previously unreported level of genome folding, where distant TAD pairs, megabases apart, interact to form meta-domains. Within meta-domains, gene promoters and structural intergenic elements present in distant TADs are specifically paired. The associated genes encode neuronal determinants, including those engaged in axonal guidance and adhesion. These long-range associations occur in a large fraction of neurons but support transcription in only a subset of neurons. Meta-domains are formed by diverse transcription factors that are able to pair over long and flexible distances. We present evidence that two such factors, GAF and CTCF, play direct roles in this process. The relative simplicity of higher-order meta-domain interactions in Drosophila, compared with those previously described in mammals, allowed the demonstration that genomes can fold into highly specialized cell-type-specific scaffolds that enable megabase-scale regulatory associations.

GAGA-associated factor fosters loop formation in the Drosophila genome

The impact of genome organization on the control of gene expression persists as a major challenge in regulatory biology. Most efforts have focused on the role of CTCF-enriched boundary elements and TADs, which enable long-range DNA-DNA associations via loop extrusion processes. However, there is increasing evidence for long-range chromatin loops between promoters and distal enhancers formed through specific DNA sequences, including tethering elements, which bind the GAGA-associated factor (GAF). Previous studies showed that GAF possesses amyloid properties invitro, bridging separate DNA molecules. In this study, we investigated whether GAF functions as a looping factor in Drosophila development. We employed Micro-C assays to examine the impact of defined GAF mutants on genome topology. These studies suggest that the N-terminal POZ/BTB oligomerization domain is important for long-range associations of distant GAGA-rich tethering elements, particularly those responsible for promoter-promoter interactions that coordinate the activities of distant paralogous genes.

Deep learning prediction of amplitude death

Affected by parameter drift and coupling organization, nonlinear dynamical systems exhibit suppressed oscillations. This phenomenon is called amplitude death. In various complex systems, amplitude death is a typical critical phenomenon, which may lead to the functional collapse of the system. Therefore, an important issue is how to effectively predict critical phenomena based on the data in the system oscillation state. This paper proposes an enhanced Informer model to predict amplitude death. The model employs an attention mechanism to capture the long-range associations of the system time series and tracks the effect of parameter drift on the system dynamics through an accompanying parameter input channel. The experimental results based on the coupled Rössler and Lorentz systems show that the enhanced informer has higher prediction accuracy and longer effective prediction distance than the original algorithm and can predict the amplitude death of a system.

Insights into Allosteric Mechanisms of the Lung-Enriched p53 Mutants V157F and R158L.

Lung cancer is a leading fatal malignancy in humans. p53 mutants exhibit not only loss of tumor suppressor capability but also oncogenic gain-of-function, contributing to lung cancer initiation, progression and therapeutic resistance. Research shows that p53 mutants V157F and R158L occur with high frequency in lung squamous cell carcinomas. Revealing their conformational dynamics is critical for developing novel lung therapies. Here, we used all-atom molecular dynamics (MD) simulations to investigate the effect of V157F and R158L substitutions on the structural properties of the p53 core domain (p53C). Compared to wild-type (WT) p53C, both V157F and R158L mutants display slightly lesser β-sheet structure, larger radius of gyration, larger volume and larger exposed surface area, showing aggregation-prone structural characteristics. The aggregation-prone fragments (residues 249–267 and 268–282) of two mutants are more exposed to water solution than that of WT p53C. V157F and R158L mutation sites can affect the conformation switch of loop 1 through long-range associations. Simulations also reveal that the local structure and conformation around the V157F and R158L mutation sites are in a dynamic equilibrium between the misfolded and properly folded conformations. These results provide molecular mechanistic insights into allosteric mechanisms of the lung-enriched p53 mutants.

DPTCN: A novel deep CNN model for short text classification

As an important branch of Nature Language Processing (NLP), how to extract useful text information and effective long-range associations has always been a bottleneck for text classification. With the great effort of deep learning researchers, deep Convolutional Neural Networks (CNNs) have made remarkable achievements in Computer Vision but still controversial in NLP tasks. In this paper, we propose a novel deep CNN named Deep Pyramid Temporal Convolutional Network (DPTCN) for short text classification, which is mainly consisting of concatenated embedding layer, causal convolution, 1/2 max pooling down-sampling and residual blocks. It is worth mentioning that our work was highly inspired by two well-designed models: one is temporal convolutional network for sequential modeling; another is deep pyramid CNN for text categorization; as their applicability and pertinence remind us how to build a model in a special domain. In the experiments, we evaluate the proposed model on 7 datasets with 6 models and analyze the impact of three different embedding methods. The results prove that our work is a good attempt to apply word-level deep convolutional network in short text classification.

Genomic and phenotypic insights from an atlas of genetic effects on DNA methylation.

Characterizing genetic influences on DNA methylation (DNAm) provides an opportunity to understand mechanisms underpinning gene regulation and disease. Here we describe results of DNA methylation-quantitative trait locus (mQTL) analyses on 32,851 participants, identifying genetic variants associated with DNAm at 420,509 DNAm sites in blood. We present a database of >270,000 independent mQTLs of which 8.5% comprise long-range (trans) associations. Identified mQTL associations explain 15-17% of the additive genetic variance of DNAm. We show that the genetic architecture of DNAm levels is highly polygenic. Using shared genetic control between distal DNAm sites we construct networks, identifying 405 discrete genomic communities enriched for genomic annotations and complex traits. Shared genetic variants are associated with both DNAm levels and complex diseases but only in a minority of cases do these associations reflect causal relationships from DNAm to trait or vice versa, indicating a more complex genotype-phenotype map than previously anticipated.

Potential roles of condensin in genome organization and beyond in fission yeast.

The genome is highly organized hierarchically by the function of structural maintenance of chromosomes (SMC) complex proteins such as condensin and cohesin from bacteria to humans. Although the roles of SMC complex proteins have been well characterized, their specialized roles in nuclear processes remain unclear. Condensin and cohesin have distinct binding sites and mediate long-range and short-range genomic associations, respectively, to form cell cycle-specific genome organization. Condensin can be recruited to highly expressed genes as well as dispersed repeat genetic elements, such as Pol III-transcribed genes, LTR retrotransposon, and rDNA repeat. In particular, mitotic transcription factors Ace2 and Ams2 recruit condensin to their target genes, forming centromeric clustering during mitosis. Condensin is potentially involved in various chromosomal processes such as the mobility of chromosomes, chromosome territories, DNA reannealing, and transcription factories. The current knowledge of condensin in fission yeast summarized in this review can help us understand how condensin mediates genome organization and participates in chromosomal processes in other organisms.

Appraising trends and forecasting of hydroclimatic variables in the north and northeast regions of Bangladesh

Appraising trend and forecasting of seasonal and annual rainfall, temperature, river water discharge, and groundwater level are crucial for local water resource practices and designing agricultural crop production. However, the spatiotemporal trends of these hydroclimatic variables remain uncertain under the changing climate at the regional scale. Therefore, we analyzed hydroclimatic variable trends in the north and northeast regions of Bangladesh for 38 years (1980–2017) using a set of parametric and non-parametric techniques and forecasted by the Autoregressive Integrated Moving Average (ARIMA) model. Based on detrended fluctuation analysis, long-range associations of the hydroclimatic variables suggested that these variables will retain their current trend line in the forthcoming period. Annual temperature and river water discharge exhibited an increasing linear trend of 0.014 °C and 0.61 m3 s−1 per year, while annual rainfall demonstrated a declining linear trend of − 0.87 mm and resultant in an annual water level decrease of − 0.03 m per year. Seasonal maximum trends were observed for rainfall in the winter (− 2.25 mm), the temperature in monsoon (0.03 °C) and water discharge in post-monsoon (10.7 m3 s−1), and water level in post-monsoon (− 0.54 m) per year. The ARIMA model forecasted that the temperature would be increased by 0.14 °C/decade and water discharge will be 8.75 m3 s−1/decade, while rainfall and water level will be decreased by 90 mm/decade and 1.24 m/decade respectively for the period 2018–2027. The results found from this research can be a guideline for regional water practices and planning cropping systems in the region.

Finding Friends in the Crowd: Three-Dimensional Cliques of Topological Genomic Domains.

The mammalian genome is intricately folded in a three-dimensional topology believed to be important for the orchestration of gene expression regulating development, differentiation and tissue homeostasis. Important features of spatial genome conformation in the nucleus are promoter-enhancer contacts regulating gene expression within topologically-associated domains (TADs), short- and long-range interactions between TADs and associations of chromatin with nucleoli and nuclear speckles. In addition, anchoring of chromosomes to the nuclear lamina via lamina-associated domains (LADs) at the nuclear periphery is a key regulator of the radial distribution of chromatin. To what extent TADs and LADs act in concert as genomic organizers to shape the three-dimensional topology of chromatin has long remained unknown. A new study addressing this key question provides evidence of (i) preferred long-range associations between TADs forming TAD “cliques” which organize large heterochromatin domains, and (ii) stabilization of TAD cliques by LADs at the nuclear periphery after induction of terminal differentiation. Here, we review these findings, address the issue of whether TAD cliques exist in single cells and discuss the extent of cell-to-cell heterogeneity in higher-order chromatin conformation. The recent observations provide a first appreciation of changes in 4-dimensional higher-order genome topologies during differentiation.

Are associations formed across pairs? A test of learning by temporal contiguity in associative recognition.

In models such as the search of associative memory (SAM: Gillund & Shiffrin, Psychological Review, 91(1), 1-67 1984) model, associations in paired-associate tasks are only formed between the pair of to-be-remembered items. The temporal context model (TCM: Howard & Kahana, Journal of Mathematical Psychology, 46, 268-299 2002) deviates from SAM by positing that long-range associations are formed between the current item and all previously presented items, even in paired-associate tasks, where cross-pair associations are formed in addition to within-pair associations (Davis, Geller, Rizzuto, & Kahana, Psychonomic Bulletin & Review, 15, 64-69 2008). We tested this proposal in an associative recognition task by constructing rearranged pairs where the distance in within-list serial position between the two pair members was manipulated between one and five pairs. Models such as TCM would predict that FAR should be highest for rearranged pairs that are constructed from pair members that were adjacent to each other on the study list, whereas models such as SAM predict that FAR should be equal for rearranged pairs regardless of whether they are constructed from adjacent or remote pairs. Results from our experiment and from three archival datasets found that FAR for rearranged pairs did not depend on whether the constituent items came from nearby or remote pairs, suggesting that participants were not forming associations across pairs of items in the task.

Transvection-like interchromosomal interaction is not observed at the transcriptional level when tested in the Rosa26 locus in mouse.

Long-range associations between enhancers and their target gene promoters have been shown to play critical roles in executing genome function. Recent variations of chromosome capture technology have revealed a comprehensive view of intra- and interchromosomal contacts between specific genomic sites. The locus control region of the β-globin genes (β-LCR) is a super-enhancer that is capable of activating all of the β-like globin genes within the locus in cis through physical interaction by forming DNA loops. CTCF helps to mediate loop formation between LCR-HS5 and 3’HS1 in the human β-globin locus, in this way thought to contribute to the formation of a “chromatin hub”. The β-globin locus is also in close physical proximity to other erythrocyte-specific genes located long distances away on the same chromosome. In this case, erythrocyte-specific genes gather together at a shared “transcription factory” for co-transcription. Theoretically, enhancers could also activate target gene promoters at the identical loci, yet on different chromosomes in trans, a phenomenon originally described as transvection in Drosophilla. Although close physical proximity has been reported for the β-LCR and the β-like globin genes when integrated at the mouse homologous loci in trans, their structural and functional interactions were found to be rare, possibly because of a lack of suitable regulatory elements that might facilitate such trans interactions. Therefore, we re-evaluated presumptive transvection-like enhancer-promoter communication by introducing CTCF binding sites and erythrocyte-specific transcription units into both LCR-enhancer and β-promoter alleles, each inserted into the mouse ROSA26 locus on separate chromosomes. Following cross-mating of mice to place the two mutant loci at the identical chromosomal position and into active chromation in trans, their transcriptional output was evaluated. The results demonstrate that there was no significant functional association between the LCR and the β-globin gene in trans even in this idealized experimental context.

The Role of RNA Polymerase II Contiguity and Long-Range Interactions in the Regulation of Gene Expression in Human Pluripotent Stem Cells.

The eukaryotic nucleus is a highly complex structure that carries out multiple functions primarily needed for gene expression, and among them, transcription seems to be the most fundamental. Diverse approaches have demonstrated that transcription takes place at discrete sites known as transcription factories, wherein RNA polymerase II (RNAP II) is attached to the factory and immobilized while transcribing DNA. It has been proposed that transcription factories promote chromatin loop formation, creating long-range interactions in which relatively distant genes can be transcribed simultaneously. In this study, we examined long-range interactions between the POU5F1 gene and genes previously identified as being POU5F1 enhancer-interacting, namely, CDYL, TLE2, RARG, and MSX1 (all involved in transcriptional regulation), in human pluripotent stem cells (hPSCs) and their early differentiated counterparts. As a control gene, RUNX1 was used, which is expressed during hematopoietic differentiation and not associated with pluripotency. To reveal how these long-range interactions between POU5F1 and the selected genes change with the onset of differentiation and upon RNAP II inhibition, we performed three-dimensional fluorescence in situ hybridization (3D-FISH) followed by computational simulation analysis. Our analysis showed that the numbers of long-range interactions between specific genes decrease during differentiation, suggesting that the transcription of monitored genes is associated with pluripotency. In addition, we showed that upon inhibition of RNAP II, long-range associations do not disintegrate and remain constant. We also analyzed the distance distributions of these genes in the context of their positions in the nucleus and revealed that they tend to have similar patterns resembling normal distribution. Furthermore, we compared data created in vitro and in silico to assess the biological relevance of our results.

Invariant TAD Boundaries Constrain Cell-Type-Specific Looping Interactions between Promoters and Distal Elements around the CFTR Locus

Three-dimensional genome structure plays an important role in gene regulation. Globally, chromosomes are organized into active and inactive compartments while, at the gene level, looping interactions connect promoters to regulatory elements. Topologically associating domains (TADs), typically several hundred kilobases in size, form an intermediate level of organization. Major questions include how TADs are formed and how they are related to looping interactions between genes and regulatory elements. Here we performed a focused 5C analysis of a 2.8 Mb chromosome 7 region surrounding CFTR in a panel of cell types. We find that the same TAD boundaries are present in all cell types, indicating that TADs represent a universal chromosome architecture. Furthermore, we find that these TAD boundaries are present irrespective of the expression and looping of genes located between them. In contrast, looping interactions between promoters and regulatory elements are cell-type specific and occur mostly within TADs. This is exemplified by the CFTR promoter that in different cell types interacts with distinct sets of distal cell-type-specific regulatory elements that are all located within the same TAD. Finally, we find that long-range associations between loci located in different TADs are also detected, but these display much lower interaction frequencies than looping interactions within TADs. Interestingly, interactions between TADs are also highly cell-type-specific and often involve loci clustered around TAD boundaries. These data point to key roles of invariant TAD boundaries in constraining as well as mediating cell-type-specific long-range interactions and gene regulation.

Three-dimensional aspects of matrix assembly by cells in the developing cornea

Cell-directed deposition of aligned collagen fibrils during corneal embryogenesis is poorly understood, despite the fact that it is the basis for the formation of a corneal stroma that must be transparent to visible light and biomechanically stable. Previous studies of the structural development of the specialized matrix in the cornea have been restricted to examinations of tissue sections by conventional light or electron microscopy. Here, we use volume scanning electron microscopy, with sequential removal of ultrathin surface tissue sections achieved either by ablation with a focused ion beam or by serial block face diamond knife microtomy, to examine the microanatomy of the cornea in three dimensions and in large tissue volumes. The results show that corneal keratocytes occupy a significantly greater tissue volume than was previously thought, and there is a clear orthogonality in cell and matrix organization, quantifiable by Fourier analysis. Three-dimensional reconstructions reveal actin-associated tubular cell protrusions, reminiscent of filopodia, but extending more than 30 µm into the extracellular space. The highly extended network of these membrane-bound structures mirrors the alignment of collagen bundles and emergent lamellae and, we propose, plays a fundamental role in dictating the orientation of collagen in the developing cornea.