Pairwise Distance Matrix Research Articles

Scaling DEPP phylogenetic placement to ultra-large reference trees: a tree-aware ensemble approach.

Phylogenetic placement of a query sequence on a backbone tree is increasingly used across biomedical sciences to identify the content of a sample from its DNA content. The accuracy of such analyses depends on the density of the backbone tree, making it crucial that placement methods scale to very large trees. Moreover, a new paradigm has been recently proposed to place sequences on the species tree using single-gene data. The goal is to better characterize the samples and to enable combined analyses of marker-gene (e.g., 16S rRNA gene amplicon) and genome-wide data. The recent method DEPP enables performing such analyses using metric learning. However, metric learning is hampered by a need to compute and save a quadratically growing matrix of pairwise distances during training. Thus, the training phase of DEPP does not scale to more than roughly 10 000 backbone species, a problem that we faced when trying to use our recently released Greengenes2 (GG2) reference tree containing 331 270 species. This paper explores divide-and-conquer for training ensembles of DEPP models, culminating in a method called C-DEPP. While divide-and-conquer has been extensively used in phylogenetics, applying divide-and-conquer to data-hungry machine-learning methods needs nuance. C-DEPP uses carefully crafted techniques to enable quasi-linear scaling while maintaining accuracy. C-DEPP enables placing 20 million 16S fragments on the GG2 reference tree in 41 h of computation. The dataset and C-DEPP software are freely available at https://github.com/yueyujiang/dataset_cdepp/.

Generalized black hole clustering algorithm

The Black Hole Clustering (BHC) algorithm is a density-based partitional clustering method inspired by the Density-based Spatial Clustering of Applications with Noise (DBSCAN). It does not require the number of clusters nor the computation of the pair-wise distance matrix between the data points, making it faster than DBSCAN. Also, it only needs one parameter that is intuitively easier to set than the epsilon parameter of DBSCAN. However, BHC needs the allocation of the so-called black holes that have to be linearly independent, making the algorithm in its current version suitable only for two or three-dimensional data sets. In this paper, we propose a generalized version of the black hole clustering algorithm (GBHC) by introducing a novel black hole allocation procedure for higher-dimensional data spaces. Furthermore, the proposed method is data-independent, so we have to run it once to obtain the black hole positions for all finite-dimensional metric spaces. We performed extensive computational experiments to compare GBHC with DBSCAN. The results show that both algorithms obtain comparable clustering solutions. GBHC, however, outperforms DBSCAN in computational complexity and explainability.

ML-aVAT: A Novel 2-Stage Machine-Learning Approach for Automatic Clustering Tendency Assessment

Clustering tendency assessment, which aims to deduce if a dataset contains any cluster structure, and, if it does, how many clusters it has, is a critical problem in exploratory data analysis. The VAT family of algorithms provides a “visual” means to assess the clustering tendency for various datasets. The VAT algorithm operates by reordering the pairwise distance matrix of the input data. When viewed as a monochrome image, this reordered dissimilarity matrix is called a reordered dissimilarity image (RDI), showing possible data clusters by dark blocks along the diagonal. This process, however, requires human intervention to interpret an RDI. Moreover, for datasets having complex cluster structure or noise, dark blocks along the diagonal of the RDI are not easily distinguishable, making it difficult to count them accurately, and different individuals can report different numbers of dark blocks. Only a handful of approaches have been proposed in the literature to automatically (algorithmically) infer the cluster structure from a VAT-type RDI without requiring human input. However, these approaches do not perform well for several data types and have impractically high run-time. This paper proposes and develops ML-aVAT: a novel two-stage machine-learning-based approach for automatic clustering tendency assessment from VAT-type RDI. Besides estimating the number of clusters, ML-aVAT can also infer the clustering hierarchy, i.e., sub-clusters within each group, something none of the previously proposed algorithms could do. Numerical experiments performed on various synthetic and real-life labeled and unlabeled datasets prove the effectiveness of ML-aVAT in estimating clustering tendency and cluster hierarchy.

Molecular analysis of NS1 gene of Indian protoparvoviruses

Porcine parvovirus (PPV) is a predominant infectious agent responsible for infertility in pigs. Globally, it leads to colossal economic loss to the pig rearing communities, especially in the developing countries. The nonstructural gene 1 (NS1) of PPV is responsible for the virus replication, transcription regulation and cytotoxicity. Variations in the NS1 gene could cause increased virulence of the virus and the enhanced virulence raises concern about the effectiveness of the PPV vaccines against newly emerging strains. So, in the present study, we collected 84 samples from different regions of Punjab and Guwahati (Assam) during 2019-2022. Out of 84 samples, 12samples were positive for PPV-1. The NS1 gene was cloned and sequenced followed by analysis of Indian PPV-1 isolates to understand its evolutionary background, level of divergence and nucleotide/amino acid substitutions. The findings revealed that Indian PPV-1 isolates exhibit nucleotide substitutions with high percent sequence identity. Pairwise distance matrix values of the NS1 gene revealed that local PPV-1 isolates showed maximum divergence from sequences of Brazil, China, Korea and Germany. Furthermore, selection pressure analysis revealed that all the isolates were under positive selection. The findings of the current study warrant whole genome analysis of circulating PPVs in India to identify a putative vaccine strain for combating emerging PPVs.

Molecular analysis of NS1 gene of Indian protoparvoviruses

Porcine parvovirus (PPV) is a predominant infectious agent responsible for infertility in pigs. Globally, it leads to colossal economic loss to the pig rearing communities, especially in the developing countries. The nonstructural gene 1 (NS1) of PPV is responsible for the virus replication, transcription regulation and cytotoxicity. Variations in the NS1 gene could cause increased virulence of the virus and the enhanced virulence raises concern about the effectiveness of the PPV vaccines against newly emerging strains. So, in the present study, we collected 84 samples from different regions of Punjab and Guwahati (Assam) during 2019-2022. Out of 84 samples, 12samples were positive for PPV-1. The NS1 gene was cloned and sequenced followed by analysis of Indian PPV-1 isolates to understand its evolutionary background, level of divergence and nucleotide/amino acid substitutions. The findings revealed that Indian PPV-1 isolates exhibit nucleotide substitutions with high percent sequence identity. Pairwise distance matrix values of the NS1 gene revealed that local PPV-1 isolates showed maximum divergence from sequences of Brazil, China, Korea and Germany. Furthermore, selection pressure analysis revealed that all the isolates were under positive selection. The findings of the current study warrant whole genome analysis of circulating PPVs in India to identify a putative vaccine strain for combating emerging PPVs.

Uncovering the Morphological and Genetical Heterogeneity of Pyricularia oryzae (cooke) sacc. in Southwestern Region of Bangladesh

Significant yield losses are reported due to blast disease caused by Pyricularia oryzae across all rice growing areas of the world. Though, management strategies reduce disease significantly but blast epidemics are still common. This research was aimed to determine the variability among the 19 Pyricularia oryzae isolates of South-Western regions of Bangladesh. Morphological variability was determined based on nine morphological components like front mycelial color, reverse mycelial color, growth behavior, conidial length (μm), conidial width (μm), conidial color, radial growth at 12 days (mm), 14 days (mm) and 16 days (mm) after incubation. Cluster analysis considering nine morphological components showed five distinct groups. Front and reverse mycelial color, growth behavior and conidial length played more than 80% role in generating variation among the nine tested components. Higher genetic variation was also detected among the 19 isolates of P. oryzae. Here, larger numbers of amplified DNA fragments was formed by Primers UCB-155 which showed distinct polymorphism. Between two clusters, Cluster II consists of 18 isolates that were originated from different origin but having maximum 94% genetical similarity. Cluster I contain only one isolates PO-16. From pair wise distance matrix, maximum distance (80%) was found between PO-16 and PO-03 isolates and minimum distance (6%) was obtained between the isolates PO-11 and PO-12. Results of this experiment revealed that, higher genetic diversity among isolates of P. oryzae existed in the south western region of Bangladesh but there has no direct relation with genetic diversity and geographical origin.

Ksak: A high-throughput tool for alignment-free phylogenetics.

Phylogenetic tools are fundamental to the studies of evolutionary relationships. In this paper, we present Ksak, a novel high-throughput tool for alignment-free phylogenetic analysis. Ksak computes the pairwise distance matrix between molecular sequences, using seven widely accepted k-mer based distance measures. Based on the distance matrix, Ksak constructs the phylogenetic tree with standard algorithms. When benchmarked with a golden standard 16S rRNA dataset, Ksak was found to be the most accurate tool among all five tools compared and was 19% more accurate than ClustalW2, a high-accuracy multiple sequence aligner. Above all, Ksak was tens to hundreds of times faster than ClustalW2, which helps eliminate the computation limit currently encountered in large-scale multiple sequence alignment. Ksak is freely available at https://github.com/labxscut/ksak.

Integrative Approaches Establish Colour Polymorphism in the Bamboo-Feeding Leafhopper Mukaria splendida Distant (Hemiptera: Cicadellidae) from India

The leafhopper species, Mukaria splendida Distant, is economically important due to itsstatus as a pest on bamboo and was recently reported to bea vector for phytoplasmal disease. Morphological identification is often difficult and requires a high level of taxonomic expertise, with misidentifications causing problematic false-positive/negative results. In this paper, colour polymorphism was recorded in the bamboo-feedingleafhopper M. splendida (Distant, 1908), which is a major insect pest in the bamboo ecosystem based on explorations conducted in different locations of India. Ten morphs were identified for each sex of M. splendida Distant based on the colour pattern on the pronotum and forewings. However, in view of the economic importance of the species, the morphological studies were integrated with the molecular data for the accurate identification of the species. The morphological characteristics and sequence results of the amplified product of the mitochondrial cytochrome oxidase subunit I (mtCOI) gene confirmed that all the morphs were M. splendida and the pairwise distance matrix showed a negligible genetic distance in the COI mtDNA gene. Simulated future predictions, along with detailed notes on the colour polymorphic forms with illustrations, and additional distribution records as well as thebiology of M. splendida were discussed in light of the available literature, all of which will aid the authentic identification of this species.

Insights into the genetic landscape and presence of Cochliomyia hominivorax in the Caribbean.

The New World screwworm, Cochliomyia hominivorax, is a major parasite that causes myiasis in livestock, humans, and other warm-blooded animals in the western hemisphere. There is a permanent biological border that is maintained between Panama and Colombia, as it has been eradicated from North and Central America. However, it still exists in much of the Caribbean and South America causing an estimated annual loss of $3.6 billion dollars in South America alone. Less information is available for C. hominivorax in the Caribbean. Thus, here we examined its presence and genetic landscape in order to gain insights into this fly's distribution in this region. First, through sampling efforts, novel GPS (Global Positioning System) coordinates were collected. Second, the environmental correlates of those presence points were examined. Next, samples were sequenced in order to obtain a pairwise ΦIT genetic distance matrix. And lastly, this matrix was used to create a genetic landscape of divergence. The results of the genetic landscape show flies as more diverse in Trinidad and Tobago and less diverse in the Dominican Republic. This is perhaps due to the proximity of Trinidad to Venezuela and gene flow may be occurring between these two areas. This information will aid in screwworm surveillance and control programs by providing environmental correlates and a view into the distribution of these flies.

A deep learning approach to real-time HIV outbreak detection using genetic data.

Pathogen genomic sequence data are increasingly made available for epidemiological monitoring. A main interest is to identify and assess the potential of infectious disease outbreaks. While popular methods to analyze sequence data often involve phylogenetic tree inference, they are vulnerable to errors from recombination and impose a high computational cost, making it difficult to obtain real-time results when the number of sequences is in or above the thousands.Here, we propose an alternative strategy to outbreak detection using genomic data based on deep learning methods developed for image classification. The key idea is to use a pairwise genetic distance matrix calculated from viral sequences as an image, and develop convolutional neutral network (CNN) models to classify areas of the images that show signatures of active outbreak, leading to identification of subsets of sequences taken from an active outbreak. We showed that our method is efficient in finding HIV-1 outbreaks with R0 ≥ 2.5, and overall a specificity exceeding 98% and sensitivity better than 92%. We validated our approach using data from HIV-1 CRF01 in Europe, containing both endemic sequences and a well-known dual outbreak in intravenous drug users. Our model accurately identified known outbreak sequences in the background of slower spreading HIV. Importantly, we detected both outbreaks early on, before they were over, implying that had this method been applied in real-time as data became available, one would have been able to intervene and possibly prevent the extent of these outbreaks. This approach is scalable to processing hundreds of thousands of sequences, making it useful for current and future real-time epidemiological investigations, including public health monitoring using large databases and especially for rapid outbreak identification.

MicrobiomeGWAS: A Tool for Identifying Host Genetic Variants Associated with Microbiome Composition

The microbiome is the collection of all microbial genes and can be investigated by sequencing highly variable regions of 16S ribosomal RNA (rRNA) genes. Evidence suggests that environmental factors and host genetics may interact to impact human microbiome composition. Identifying host genetic variants associated with human microbiome composition not only provides clues for characterizing microbiome variation but also helps to elucidate biological mechanisms of genetic associations, prioritize genetic variants, and improve genetic risk prediction. Since a microbiota functions as a community, it is best characterized by β diversity; that is, a pairwise distance matrix. We develop a statistical framework and a computationally efficient software package, microbiomeGWAS, for identifying host genetic variants associated with microbiome β diversity with or without interacting with an environmental factor. We show that the score statistics have positive skewness and kurtosis due to the dependent nature of the pairwise data, which makes p-value approximations based on asymptotic distributions unacceptably liberal. By correcting for skewness and kurtosis, we develop accurate p-value approximations, whose accuracy was verified by extensive simulations. We exemplify our methods by analyzing a set of 147 genotyped subjects with 16S rRNA microbiome profiles from non-malignant lung tissues. Correcting for skewness and kurtosis eliminated the dramatic deviation in the quantile–quantile plots. We provided preliminary evidence that six established lung cancer risk SNPs were collectively associated with microbiome composition for both unweighted (p = 0.0032) and weighted (p = 0.011) UniFrac distance matrices. In summary, our methods will facilitate analyzing large-scale genome-wide association studies of the human microbiome.

Algorithm of ant colony optimization (ACO) for 3D variation traveling salesman problem

The article discusses the solution of the spatial traveling salesman problem (TSP 3D variation) using Ant Colony Optimization (ACO). The traveling salesman problem considers n bridges and a matrix of pairwise distances between them. It is necessary to find such an order of visiting cities so that the total distance traveled was minimal, each city was visited exactly once and the salesman returned to the city from which he began his route. In the TSP 3D variation problem, each “city” has 3 coordinates x, y, z. The analysis of the main methods of solving, in particular, the metaheuristic algorithms to which ACO belongs, is performed. At each iteration of these methods, a new solution of the problem is built, which is based not on one, but several solutions of the population. The ACO uses an idea that is based on collecting statistical information about the best solutions. The program code is implemented in MATLAB. During computational experiments, various network topologies were randomly generated, and the number of iterations at which the optimal cycle was achieved was recorded. The execution time of the code for the TSP 3D task is almost the same as the execution time of TSP 2D. The results can be used for spatial tasks of the salesman (TSP 3D-variation), which arise in the process of 3D printing, planning UAV trajectories (UAV) in mountain conditions or multi-story urban development, road planning in multi-story buildings.

PhyloM: A Computer Program for Phylogenetic Inference from Measurement or Binary Data, with Bootstrapping.

Quantitative and binary results are ubiquitous in biology. Inasmuch as an underlying genetic basis for the observed variation in these observations can be assumed, it is pertinent to infer the evolutionary relationships among the entities being measured. I present a computer program, PhyloM, that takes measurement data or binary data as input, using which, it directly generates a pairwise distance matrix that can then be subjected to the popular neighbor-joining (NJ) algorithm to produce a phylogenetic tree. PhyloM also has the option of nonparametric bootstrapping for testing the level of support for the inferred phylogeny. Finally, PhyloM also allows the user to root the tree on any desired branch. PhyloM was tested on Biolog Gen III growth data from isolates within the genus Chromobacterium and the closely related Aquitalea sp. This allowed a comparison with the genotypic tree inferred from whole-genome sequences for the same set of isolates. From this comparison, it was possible to infer parallel evolution. PhyloM is a stand-alone and easy-to-use computer program with a user-friendly graphical user interface that computes pairwise distances from measurement or binary data, which can then be used to infer phylogeny using NJ using a utility in the same program. Alternatively, the distance matrix can be downloaded for use in another program for phylogenetic inference or other purposes. It does not require any software to be installed or computer code written and is open source. The executable and computer code are available on GitHub.

Quantifying the Alignment of Graph and Features in Deep Learning.

We show that the classification performance of graph convolutional networks (GCNs) is related to the alignment between features, graph, and ground truth, which we quantify using a subspace alignment measure (SAM) corresponding to the Frobenius norm of the matrix of pairwise chordal distances between three subspaces associated with features, graph, and ground truth. The proposed measure is based on the principal angles between subspaces and has both spectral and geometrical interpretations. We showcase the relationship between the SAM and the classification performance through the study of limiting cases of GCNs and systematic randomizations of both features and graph structure applied to a constructive example and several examples of citation networks of different origins. The analysis also reveals the relative importance of the graph and features for classification purposes.

Understanding population structure in an evolutionary context: population-specific FST and pairwise FST.

Populations are shaped by their history. It is crucial to interpret population structure in an evolutionary context. Pairwise FST measures population structure, whereas population-specific FST measures deviation from the ancestral population. To understand the current population structure and a population’s history of range expansion, we propose a representation method that overlays population-specific FST estimates on a sampling location map, and on an unrooted neighbor-joining tree and a multi-dimensional scaling plot inferred from a pairwise FST distance matrix. We examined the usefulness of our procedure using simulations that mimicked population colonization from an ancestral population and by analyzing published human, Atlantic cod, and wild poplar data. Our results demonstrated that population-specific FST values identify the source population and trace the evolutionary history of its derived populations. Conversely, pairwise FST values represent the current population structure. By integrating the results of both estimators, we obtained a new picture of the population structure that incorporates evolutionary history. The generalized least squares estimate of genome-wide population-specific FST indicated that the wild poplar population expanded its distribution to the north, where daylight hours are long in summer, to coastal areas with abundant rainfall, and to the south where summers are dry. Genomic data highlight the power of the bias-corrected moment estimators of FST, whether global, pairwise, or population-specific, that provide unbiased estimates of FST. All FST moment estimators described in this paper have reasonable processing times and are useful in population genomics studies.

Five degree-of-freedom property interpolation of arbitrary grain boundaries via Voronoi fundamental zone framework

We introduce the Voronoi fundamental zone (VFZ) framework which is useful for grain boundary (GB) structure–property models and gaining insights about the nature of a five degree-of-freedom (5DOF) fundamental zone (FZ) for both cubic and non-cubic symmetries and potentially alloys. We cover the methods associated with the VFZ framework. The VFZ framework offers an advantage over other 5DOF based property interpolation methods because directly computed Euclidean distances approximate the original grain boundary octonion (GBO) distance with significantly reduced runtime (~7 CPU min vs. 153 CPU days for a 50,000×50,000 pairwise-distance matrix). We perform boundary energy (GBE) interpolation for a non-smooth validation function on sets of up to 50,000 GBs using four interpolation methods: barycentric interpolation, Gaussian process regression (GPR) or Kriging, inverse-distance weighting (IDW), and nearest neighbor (NN) interpolation. The best performance was achieved with GPR which results in a reduction of the root mean square error (RMSE) by 83% relative to RMSE of a constant, average model. This error is comparable to the minimum expected uncertainty associated with reconstruction of noise-free experimental polycrystalline data. We then use GPR to interpolate simulated bi-crystal datasets for Fe and Ni and demonstrate better than (34.4% vs. 21.2% improvement) and similar (57.6% vs. 56.4% improvement) performance to prior work, respectively. The noise and non-uniform sampling in the Fe dataset make it difficult to resolve low GBE (i.e. cusps) and to validate the model. We resolve the Ni dataset cusps with high accuracy, but uncertainty may be high in regions far from the input data. The trade-offs between noise, dataset size, sampling scheme, and repeat measurements must be carefully managed. We provide a vectorized, parallelized, MATLAB interpolation function (interp5DOF.m) and related routines ( github.com/sgbaird-5dof/interp) which can be applied to future datasets for a variety of GB properties and to better understand 5DOF FZs. For example, we estimate the maximum dimension of an Oh VFZ to be ~66.5° in the GBO sense.

Large-scale tandem mass spectrum clustering using fast nearest neighbor searching.

Advanced algorithmic solutions are necessary to process the ever-increasing amounts of mass spectrometry data that are being generated. In this study, we describe the falcon spectrum clustering tool for efficient clustering of millions of MS/MS spectra. falcon succeeds in efficiently clustering large amounts of mass spectral data using advanced techniques for fast spectrum similarity searching. First, high-resolution spectra are binned and converted to low-dimensional vectors using feature hashing. Next, the spectrum vectors are used to construct nearest neighbor indexes for fast similarity searching. The nearest neighbor indexes are used to efficiently compute a sparse pairwise distance matrix without having to exhaustively perform all pairwise spectrum comparisons within the relevant precursor mass tolerance. Finally, density-based clustering is performed to group similar spectra into clusters. Several state-of-the-art spectrum clustering tools were evaluated using a large draft human proteome data set consisting of 25 million spectra, indicating that alternative tools produce clustering results with different characteristics. Notably, falcon generates larger highly pure clusters than alternative tools, leading to a larger reduction in data volume without the loss of relevant information for more efficient downstream processing. falcon is a highly efficient spectrum clustering tool, which is publicly available as an open source under the permissive BSD license at https://github.com/bittremieux/falcon.

Chatbot to improve learning punctuation in Spanish and to enhance open and flexible learning environments

The objective of this article is to analyze the didactic functionality of a chatbot to improve the results of the students of the National University of Distance Education (UNED / Spain) in accessing the university in the subject of Spanish Language. For this, a quasi-experimental experiment was designed, and a quantitative methodology was used through pretest and posttest in a control and experimental group in which the effectiveness of two teaching models was compared, one more traditional based on exercises written on paper and another based on interaction with a chatbot. Subsequently, the perception of the experimental group in an academic forum about the educational use of the chatbot was analyzed through text mining with tests of Latent Dirichlet Allocation (LDA), pairwise distance matrix and bigrams. The quantitative results showed that the students in the experimental group substantially improved the results compared to the students with a more traditional methodology (experimental group / mean: 32.1346 / control group / mean: 28.4706). Punctuation correctness has been improved mainly in the usage of comma, colon and periods in different syntactic patterns. Furthermore, the perception of the students in the experimental group showed that they positively value chatbots in their teaching–learning process in three dimensions: greater “support” and companionship in the learning process, as they perceive greater interactivity due to their conversational nature; greater “feedback” and interaction compared to the more traditional methodology and, lastly, they especially value the ease of use and the possibility of interacting and learning anywhere and anytime.

Hierarchical Meta-Storms enables comprehensive and rapid comparison of microbiome functional profiles on a large scale using hierarchical dissimilarity metrics and parallel computing.

It is coded in C/C++ with parallel computing and released in two alternative forms: a standalone software (https://github.com/qdu-bioinfo/hierarchical-meta-storms) and an equivalent R package (https://github.com/qdu-bioinfo/hrms). Supplementary data are available at Bioinformatics Advances online.

A hybrid quantum regression model for the prediction of molecular atomization energies

Quantum machine learning is a relatively new research field that aims to combine the dramatic performance advantage offered by quantum computing and the ability of machine learning algorithms to learn complex distributions of high-dimensional data. The primary focus of this domain is the implementation of classical machine learning algorithms in the quantum mechanical domain and study of the speedup due to quantum parallelism, which could enable the development of novel techniques for solving problems such as quantum phase recognition and quantum error correction optimization. In this paper, we propose a hybrid quantum machine learning pipeline for predicting the atomization energies of various molecules using the nuclear charges and atomic positions of the constituent atoms. Firstly, we will be using a deep convolutional auto-encoder model for the feature extraction of data constructed from the eigenvalues and eigenvector centralities of the pairwise distance matrix calculated from atomic positions and the unrolled upper triangle of each Coulomb matrix calculated from nuclear charges, and we will then be using a quantum regression algorithm such as quantum linear regression, quantum radial basis function neural network and, a quantum neural network for estimating the atomization energy. The hybrid quantum neural network models do not seem to provide any speedup over their classical counterparts. Before implementing a quantum algorithm, we will also be using state-of-the-art classical machine learning and deep learning models such as XGBoost, multilayer perceptron, deep convolutional neural network, and a long short-term memory network to study the correlation between the extracted features and corresponding atomization energies of molecules.