Pairwise Distance Matrix Research Articles

Exploring Evolution of Public Opinions on Tianya Club Using Dynamic Topic Models

AbstractOnline media have brought tremendous changes to civic life, public opinions, and government administration. Compared with traditional media, online media not only allow individuals to browse news and express their views more freely, but also accelerate the transmission of opinions and expand influence. As public opinions may arouse societal unrest, it is worth detecting the primary topics and uncovering the evolution trends of public opinions for societal administration. Various algorithms are developed to deal with the huge volume of unstructured online media data. In this study, dynamic topic model is employed to explore topic content evolution and prevalence evolution using the original posts published from 2013 to 2017 on the Tianya Zatan Board of Tianya Club, which is one of the most popular BBS in China. Based on semantic similarities, topics are grouped into three themes: Family life, societal affairs, and government administration. The evolution of topic prevalence and content are affected by emergent incidents. Topics on family life become popular, while themes “societal affairs” and “government administration” with bigger standard deviations are more likely to be influenced by emergent hot events. Content evolution represented by monthly pairwise distance matrix is very easy to find change points of topic content.

Unblended disjoint tree merging using GTM improves species tree estimation

BackgroundPhylogeny estimation is an important part of much biological research, but large-scale tree estimation is infeasible using standard methods due to computational issues. Recently, an approach to large-scale phylogeny has been proposed that divides a set of species into disjoint subsets, computes trees on the subsets, and then merges the trees together using a computed matrix of pairwise distances between the species. The novel component of these approaches is the last step: Disjoint Tree Merger (DTM) methods.ResultsWe present GTM (Guide Tree Merger), a polynomial time DTM method that adds edges to connect the subset trees, so as to provably minimize the topological distance to a computed guide tree. Thus, GTM performs unblended mergers, unlike the previous DTM methods. Yet, despite the potential limitation, our study shows that GTM has excellent accuracy, generally matching or improving on two previous DTMs, and is much faster than both.ConclusionsThe proposed GTM approach to the DTM problem is a useful new tool for large-scale phylogenomic analysis, and shows the surprising potential for unblended DTM methods.

Unique k-mers as Strain-Specific Barcodes for Phylogenetic Analysis and Natural Microbiome Profiling.

The need for a comparative analysis of natural metagenomes stimulated the development of new methods for their taxonomic profiling. Alignment-free approaches based on the search for marker k-mers turned out to be capable of identifying not only species, but also strains of microorganisms with known genomes. Here, we evaluated the ability of genus-specific k-mers to distinguish eight phylogroups of Escherichia coli (A, B1, C, E, D, F, G, B2) and assessed the presence of their unique 22-mers in clinical samples from microbiomes of four healthy people and four patients with Crohn’s disease. We found that a phylogenetic tree inferred from the pairwise distance matrix for unique 18-mers and 22-mers of 124 genomes was fully consistent with the topology of the tree, obtained with concatenated aligned sequences of orthologous genes. Therefore, we propose strain-specific “barcodes” for rapid phylotyping. Using unique 22-mers for taxonomic analysis, we detected microbes of all groups in human microbiomes; however, their presence in the five samples was significantly different. Pointing to the intraspecies heterogeneity of E. coli in the natural microflora, this also indicates the feasibility of further studies of the role of this heterogeneity in maintaining population homeostasis.

Evaluating and presenting uncertainty in model-based unconstrained ordination.

Variability in ecological community composition is often analyzed by recording the presence or abundance of taxa in sample units, calculating a symmetric matrix of pairwise distances or dissimilarities among sample units and then mapping the resulting matrix to a low‐dimensional representation through methods collectively called ordination. Unconstrained ordination only uses taxon composition data, without any environmental or experimental covariates, to infer latent compositional gradients associated with the sampling units. Commonly, such distance‐based methods have been used for ordination, but recently there has been a shift toward model‐based approaches. Model‐based unconstrained ordinations are commonly formulated using a Bayesian latent factor model that permits uncertainty assessment for parameters, including the latent factors that correspond to gradients in community composition. While model‐based methods have the additional benefit of addressing uncertainty in the estimated gradients, typically the current practice is to report point estimates without summarizing uncertainty. To demonstrate the uncertainty present in model‐based unconstrained ordination, the well‐known spider and dune data sets were analyzed and shown to have large uncertainty in the ordination projections. Hence to understand the factors that contribute to the uncertainty, simulation studies were conducted to assess the impact of additional sampling units or species to help inform future ordination studies that seek to minimize variability in the latent factors. Accurate reporting of uncertainty is an important part of transparency in the scientific process; thus, a model‐based approach that accounts for uncertainty is valuable. An R package, UncertainOrd, contains visualization tools that accurately represent estimates of the gradients in community composition in the presence of uncertainty.

Model-free two-sample test for network-valued data

In the framework of Object Oriented Data Analysis, a permutation approach to the two-sample testing problem for network-valued data is proposed. In detail, the present framework proceeds in four steps: (i) matrix representation of the networks, (ii) computation of the matrix of pairwise (inter-point) distances, (iii) computation of test statistics based on inter-point distances and (iv) embedding of the test statistics within a permutation test. The proposed testing procedures are proven to be exact for every finite sample size and consistent. Two new test statistics based on inter-point distances (i.e., IP-Student and IP-Fisher) are defined and a method to combine them to get a further inferential tool (i.e., IP-StudentFisher) is introduced. Simulated data shows that tests with our statistic exhibit a statistical power that is either the best or second-best but very close to the best on a variety of possible alternatives hypotheses and other statistics. A second simulation study that aims at better understanding which features are captured by specific combinations of matrix representations and distances is presented. Finally, a case study on mobility networks in the city of Milan is carried out. The proposed framework is fully implemented in the R package nevada (NEtwork-VAlued Data Analysis).

Dynamic Meta-Storms enables comprehensive taxonomic and phylogenetic comparison of shotgun metagenomes at the species level.

An accurate and reliable distance (or dissimilarity) among shotgun metagenomes is fundamental to deducing the beta-diversity of microbiomes. To compute the distance at the species level, current methods either ignore the evolutionary relationship among species or fail to account for unclassified organisms that cannot be mapped to definite tip nodes in the phylogenic tree, thus can produce erroneous beta-diversity pattern. To solve these problems, we propose the Dynamic Meta-Storms (DMS) algorithm to enable the comprehensive comparison of metagenomes on the species level with both taxonomy and phylogeny profiles. It compares the identified species of metagenomes with phylogeny, and then dynamically places the unclassified species to the virtual nodes of the phylogeny tree via their higher-level taxonomy information. Its high speed and low memory consumption enable pairwise comparison of 100 000 metagenomes (synthesized from 3688 bacteria) within 6.4 h on a single computing node. An optimized implementation of DMS is available on GitHub (https://github.com/qibebt-bioinfo/dynamic-meta-storms) under a GNU GPL license. It takes the species-level profiles of metagenomes as input, and generates their pairwise distance matrix. The bacterial species-level phylogeny tree and taxonomy information of MetaPhlAn2 have been integrated into this implementation, while customized tree and taxonomy are also supported. Supplementary data are available at Bioinformatics online.

Comparison of distance-based and model-based ordinations.

Distance-based ordinations have played a critical role in community ecology for more than half a century, but are still under active development. These methods employ a matrix of pairwise distances or dissimilarities between sample units, and map sample units from the high-dimensional distance or dissimilarity space to a low-dimensional representation for analysis. Distance- or dissimilarity-based methods employ continuum or gradient ecological theory and a variety of statistical models to achieve the mapping. Recently, ecologists have developed model-based ordinations based on latent vectors and individual species response models. These methods employ the individualistic perspective of Gleason as the ecological model, and Bayesian or maximum-likelihood methods to estimate the parameters for the low dimensional space represented by the latent vectors. In this research I compared two distance-based methods (NMDS and t-SNE) with two model-based methods (BORAL and REO) on five data sets to determine which methods are superior for (1) extracting meaningful ecological drivers of variability in community composition, and (2) estimating sample unit locations in ordination space that maximize the goodness-of-fit of individual species response models to the estimated sample unit locations. Environmental variables and species were fitted to the ordinations by generalized additive models (GAMs) with Gaussian, negative binomial, or Poisson distribution models as appropriate. Across the five data sets, 22 models of environmental variability and 449 models of species distributions were calculated for each of the ordination methods. To minimize the effects of stochasticity the entire analysis was replicated three times and results averaged across the replicates. Results were evaluated by deviance explained and AIC for environmental variables and species distributions, averaged by ordination method for each data set, and ranked from best to worst. For the four assessments distance-based methods ranked 1 and 2 in three cases, and 1 and 3 in one case, significantly out performing the model-based methods. t-SNE was the top-performing method, out performing NMDS especially on the more complex data sets. In general the gradient-based theoretical basis and data sufficiency of distance-based methods allowed distance-based methods to outperform model-based methods in every assessment.

MiSTree: a Python package for constructing and analysing Minimum Spanning Trees

The minimum spanning tree (MST), a graph constructed from a distribution of points, draws lines between pairs of points so that all points are linked in a single skeletal structure that contains no loops and has minimal total edge length. The MST has been used in a broad range of scientific fields such as particle physics (to distinguish classes of events in collider collisions), in astronomy (to detect mass segregation in star clusters) and cosmology (to search for filaments in the cosmic web). Its success in these fields has been driven by its sensitivity to the spatial distribution of points and the patterns within. MiSTree, a public Python package, allows a user to construct the MST in a variety of coordinates systems, including Celestial coordinates used in astronomy. The package enables the MST to be constructed quickly by initially using a k-nearest neighbour graph (kNN, rather than a matrix of pairwise distances) which is then fed to Kruskal's algorithm to construct the MST. MiSTree enables a user to measure the statistics of the MST and provides classes for binning the MST statistics (into histograms) and plotting the distributions. Applying the MST will enable the inclusion of high-order statistics information from the cosmic web which can provide additional information to improve cosmological parameter constraints. This information has not been fully exploited due to the computational cost of calculating N-point statistics. MiSTree was designed to be used in cosmology but could be used in any field which requires extracting non-Gaussian information from point distributions. The source code for MiSTree is available on GitHub at https://github.com/knaidoo29/mistree

Estimating the phylogeny of geoemydid turtles (Cryptodira) from landmark data: an assessment of different methods.

BackgroundIn the last 20 years, a general picture of the evolutionary relationships between geoemydid turtles (ca. 70 species distributed over the Northern hemisphere) has emerged from the analysis of molecular data. However, there is a paucity of good traditional morphological characters that correlate with the phylogeny, which are essential for the robust integration of fossil and molecular data. Part of this problem might be due to intrinsic limitations of traditional discrete characters. Here, we explore the use of continuous data in the form of 3D coordinates of homologous landmarks on the turtle shell for phylogenetic inference and the phylogenetic placement of single species on a scaffold molecular tree. We focus on the performance yielded by sampling the carapace and/or plastral lobes and using various phylogenetic methods.MethodsWe digitised the landmark coordinates of the carapace and plastron of 42 and 46 extant geoemydid species, respectively. The configurations were superimposed and we estimated the phylogenetic tree of geoemydids with landmark analysis under parsimony, traditional Farris parsimony, unweighted squared-change parsimony, maximum likelihood with a Brownian motion model, and neighbour-joining on a matrix of pairwise Procrustes distances. We assessed the performance of those analyses by comparing the trees against a reference phylogeny obtained from seven molecular markers. For comparisons between trees we used difference measures based on quartets and splits. We used the same reference tree to evaluate phylogenetic placement performance by a leave-one-out validation procedure.ResultsWhatever method we used, similarity to the reference phylogeny was low. The carapace alone gave slightly better results than the plastron or the complete shell. Assessment of the potential for placement of single species on the reference tree with landmark data gave much better results, with similar accuracy and higher precision compared to the performance of discrete characters with parsimony.

Persistent homology of complex networks for dynamic state detection.

In this paper we develop an alternative topological data analysis (TDA) approach for studying graph representations of time series of dynamical systems. Specifically, we show how persistent homology, a tool from TDA, can be used to yield a compressed, multi-scale representation of the graph that can distinguish between dynamic states such as periodic and chaotic behavior. We show the approach for two graph constructions obtained from the time series. In the first approach the time series is embedded into a point cloud which is then used to construct an undirected k-nearest-neighbor graph. The second construct relies on the recently developed ordinal partition framework. In either case, a pairwise distance matrix is then calculated using the shortest path between the graph's nodes, and this matrix is utilized to define a filtration of a simplicial complex that enables tracking the changes in homology classes over the course of the filtration. These changes are summarized in a persistence diagram's two-dimensional summary of changes in the topological features. We then extract existing as well as new geometric and entropy point summaries from the persistence diagram and compare to other commonly used network characteristics. Our results show that persistence-based point summaries yield a clearer distinction of the dynamic behavior and are more robust to noise than existing graph-based scores, especially when combined with ordinal graphs.

Form and relationship of the social networks of the New Testament

The four canonical Gospels represent parallel narratives from multiple sources, and their relationship to each other has been studied using a variety of textual approaches. Here, social network analysis techniques are applied to social networks derived from the New Testament narratives contained in the Gospels and Acts, in order to explore their structure and relationships. Using maximum likelihood estimation, the degree distributions of the social networks of a Combined Gospel comprised of the four individual Gospels, and Acts, both approximate to a power law with exponential cutoff. The heavy-tailed forms of the degree distributions are emergent from the process of composition, and amalgamation of sources, and represent a complex process of self-organization. Nodes (characters) that deviate from the best fit line are shown to identify unusual or key features in a narrative. The use of network alignments, utilizing edge identity, is introduced in order to measure similarity between social networks. This approach allows the construction of a pairwise distance matrix for the four individual Gospels social networks, from which a ‘phylogenetic tree’ of social network relatedness can be generated. Using minimum evolution, least squares and neighbor-joining tree construction approaches, a Gospel ‘phylogenetic tree’ was calculated, utilizing nonparametric bootstrapping as a test for tree stability. The analysis recovered the close relationship between the three synoptic Gospels, with Mark most closely related to Matthew, consistent with the proposed heavy borrowing from Mark during the composition of Matthew. The Gospel of John is recovered as an outgroup, reflecting its divergent nature. Consequently, the network tree methodology appears a promising technique for delineating similarity between empirical networks in general, and a range of potential applications are described.

An alignment-free measure based on physicochemical properties of amino acids for protein sequence comparison

Sequence comparison is an important topic in bioinformatics. With the exponential increase of biological sequences, the traditional protein sequence comparison methods — the alignment methods become limited, so the alignment-free methods are widely proposed in the past two decades. In this paper, we considered not only the six typical physicochemical properties of amino acids, but also their frequency and positional distribution. A 51-dimensional vector was obtained to describe the protein sequence. We got a pairwise distance matrix by computing the standardized Euclidean distance, and discriminant analysis and phylogenetic analysis can be made. The results on the Influenza A virus and ND5 datasets indicate that our method is accurate and efficient for classifying proteins and inferring the phylogeny of species.

Space–Time Sequential Similarity for Identifying Factors of Activity‐Travel Pattern Segmentation: A Measure of Sequence Alignment and Path Similarity

The article develops a new method that compares activity‐travel patterns in both terms of the sequential order of activities and the shape of activity‐travel trajectory in time and space. The similarity of the list of activities and their order between activity‐travel patterns are computed by a sequence alignment method. The shape of activity‐travel trajectory is compared between the patterns using a path similarity technique that captures the direction and speed of a movement from the current location and the duration of staying at each location. The comparison results, therefore capture how people move around in three‐dimensional space–time choreography that indicates how people conduct which activities in what order. A total of 1,000 individuals are sampled from the data of 2016 Household Travel Survey, South Korea. The data provide the information of individual activity‐travel behavior and personal characteristics. The suggested method computes the pairwise distance matrix, and Ward clustering algorithm segments the pattern groups of similar activity sequences and space–time trajectories. A CHAID analysis then associates personal and household characteristics with the pattern groups to identify important factors for the segmentation. The analysis provides a significant implication in both terms of research and practice in transportation.

Clustering time series based on dependence structure

The clustering of time series has attracted growing research interest in recent years. The most popular clustering methods assume that the time series are only linearly dependent but this assumption usually fails in practice. To overcome this limitation, in this paper, we study clustering methods applicable to time series with a general and dependent structure. We propose a copula-based distance to measure dissimilarity among time series and consider an estimator for it, where the strong consistency of the estimator is guaranteed. Once the pairwise distance matrix for time series has been obtained, we apply a hierarchical clustering algorithm to cluster the time series and ensure its consistency. Numerical studies, including a large number of simulations and analysis of practical data, show that our method performs well.

Molecular characterization of Theileria spp. in livestock and the first report on the occurrence of Theileria sp. OT3 in Iran.

Theileria lestoquardi, T. ovis, and T. annulata are recognized as major causative agents for ovine and cattle theileriosis in Iran, respectively. Recently, there have been reports on the presence of Theileria spp. (Theileria sp. OT1, Theileria sp. OT3, and Theileria sp.). In this study, 37 blood samples were collected from sheep and cattle with clinically suspected Theileria infection in the Northwest of Iran. The samples were analyzed using a light microscope. DNA samples were amplified via nested-polymerase chain reaction (PCR) of 18S rRNA gene. The amplicons were digested with HpaII, following restriction fragment length polymorphism (RFLP) and sequenced to reconfirm Theileria species. The microscopic examination indicated that 4 out of 37 (10.8%) blood samples were infected with Theileria spp. Based on the nested PCR-RFLP and sequencing data, 5.4%, 13.5%, and 27% of blood samples were infected with Theileria sp. OT3, T. ovis, and T. annulata, respectively. The pairwise distance matrix of Theileria sp. OT3 showed 99.8-100% identity and 0-0.2% divergence in comparison with the registered sequences. The present study is the first report of Theileria sp. OT3 in Iran. To the evaluate evolution of Theileria spp. and providing resultant genetic data, further research with a larger sample is necessary in the region.

Genetic Diversity and Population Structure of the USDA Sweetpotato (Ipomoea batatas) Germplasm Collections Using GBSpoly.

Sweetpotato (Ipomoea batatas) plays a critical role in food security and is the most important root crop worldwide following potatoes and cassava. In the United States (US), it is valued at over $700 million USD. There are two sweetpotato germplasm collections (Plant Genetic Resources Conservation Unit and US Vegetable Laboratory) maintained by the USDA, ARS for sweetpotato crop improvement. To date, no genome-wide assessment of genetic diversity within these collections has been reported in the published literature. In our study, population structure and genetic diversity of 417 USDA sweetpotato accessions originating from 8 broad geographical regions (Africa, Australia, Caribbean, Central America, Far East, North America, Pacific Islands, and South America) were determined using single nucleotide polymorphisms (SNPs) identified with a genotyping-by-sequencing (GBS) protocol, GBSpoly, optimized for highly heterozygous and polyploid species. Population structure using Bayesian clustering analyses (STRUCTURE) with 32,784 segregating SNPs grouped the accessions into four genetic groups and indicated a high degree of mixed ancestry. A neighbor-joining cladogram and principal components analysis based on a pairwise genetic distance matrix of the accessions supported the population structure analysis. Pairwise FST values between broad geographical regions based on the origin of accessions ranged from 0.017 (Far East – Pacific Islands) to 0.110 (Australia – South America) and supported the clustering of accessions based on genetic distance. The markers developed for use with this collection of accessions provide an important genomic resource for the sweetpotato community, and contribute to our understanding of the genetic diversity present within the US sweetpotato collection and the species.

Molecular characterization of diversity in Annona squamosa L.: A high value tropical medicinal plant

Introduction: Annona squamosa L. commonly known as custard apple, is an important under-utilized fruit species from tropical and subtropical parts of India with high potential as commercial horticultural crop. The aim of this study was to determine the genetic diversity among populations spread across districts of Udaipur, Rajsamand and Chittaurgarh falling within Aravallis in Rajasthan. In all, 21 populations were identified and subjected to molecular evaluation using random amplified polymorphic DNA (RAPD) and inter simple sequence repeat (ISSR) markers. The cumulative analysis carried out on the basis of RAPD and ISSR data sets revealed 73.91% polymorphism. Methods: A total of 251 amplicons were produced using 19 RAPD and 15 ISSR primers. The pair-wise distance matrix calculated by Dice’s co-efficient showed a distance range of 0.67 to 0.95 when computed using cumulative data set. Results: Characterization of genetic diversity within populations based on combined data set showed that percentage of polymorphism bands (PPB) ranged from 29.19 to 50.93. Nei’s gene diversity (HE) values varied from 0.1002 to 0.1964 within a total genetic diversity (HT) of 0.2509. The average observed number of alleles per locus (Na) was 1.3602 and Ne was 1.2400. The Shanon’s index was 0.2002 (Hpop) and 0.3800 (Hsp) at population and species level, respectively. Conclusion: The moderate gene flow value (Nm=0.59) and significant coefficient of genetic differentiation (GST=0.458) revealed low level of genetic diversity within population and highly significant genetic differentiation among populations of A. squamosa. The UPGMA dendrogram resolved all the 21 populations representing 3 districts into various clusters according to their genetic distances. The principal component analysis (PCA) supported the UPGMA results and justified the relationship among populations.

A graph-based approach for detecting common actions in motion capture data and videos

We present a novel solution to the problem of detecting common actions in time series of motion capture data and videos. Given two action sequences, our method discovers all pairs of common subsequences, i.e. subsequences that represent the same or similar action. This is achieved in a completely unsupervised manner, i.e., without any prior knowledge of the type of actions, their number and their duration. These common subsequences (commonalities) may be located anywhere in the original sequences, may differ in duration and may be performed under different conditions e.g., by a different actor. The proposed method performs a very efficient graph-based search on the matrix of pairwise distances of frames of the two sequences. This search is supported by an objective function that captures the trade off between the similarity of the common subsequences and their lengths. The proposed method has been evaluated quantitatively on challenging datasets and in comparison to state of the art approaches. The obtained results demonstrate that the proposed method outperforms the state of the art methods both in the quality of the obtained solutions and in computational performance.

An iterated local search algorithm for the minimum differential dispersion problem

Given a set of n elements separated by a pairwise distance matrix, the minimum differential dispersion problem (Min-Diff DP) aims to identify a subset of m elements (m < n) such that the difference between the maximum sum and the minimum sum of the inter-element distances between any two chosen elements is minimized. We propose an effective iterated local search (denoted by ILS_MinDiff) for Min-Diff DP. To ensure an effective exploration and exploitation of the search space, ILS_MinDiff iterates through three sequential search phases: a fast descent-based neighborhood search phase to find a local optimum from a given starting solution, a local optima exploring phase to visit nearby high-quality solutions around a given local optimum, and a local optima escaping phase to move away from the current search region. Experimental results on six data sets of 190 benchmark instances demonstrate that ILS_MinDiff competes favorably with the state-of-the-art algorithms by finding 131 improved best results (new upper bounds).

A distance-based spectral clustering approach with applications to network community detection

Abstract Spectral clustering is an important unsupervised learning approach to many object partitioning and pattern analysis problems. In this paper, we present our work on a novel spectral clustering algorithm that groups a collection of objects using the spectrum of the pairwise distance matrix. If the points in a metric space can be associated with a well-defined distance, it is proven that the pairwise distance matrix is almost negative definite, and we show that the eigenvectors for its most significant negative eigenvalue can be used to approximate the solution to a quadratic binary partition problem. We define the quality measures for the one dimensional partitioning of the eigenvector entries, which are further applied to evaluate the partitioning results for the data points projected into the space spanned by the selected eigenvectors. Since the Lanczos iterative algorithm may be revised to find the eigenvalues efficiently in a distributed way, we adapt this algorithm to the network community detection problem using a decentralized multi-agent framework. The performance of the proposed approach is tested with different datasets, and the empirical experiments show that this approach is able to enhance the effectiveness of clustering.