Sequence Entropy Research Articles

EEG Microstate Sequences From Different Clustering Algorithms Are Information-Theoretically Invariant.

We analyse statistical and information-theoretical properties of EEG microstate sequences, as seen through the lens of five different clustering algorithms. Microstate sequences are computed for n = 20 resting state EEG recordings during wakeful rest. The input for all clustering algorithms is the set of EEG topographic maps obtained at local maxima of the spatial variance. This data set is processed by two classical microstate clustering algorithms (1) atomize and agglomerate hierarchical clustering (AAHC) and (2) a modified K-means algorithm, as well as by (3) K-medoids, (4) principal component analysis (PCA) and (5) fast independent component analysis (Fast-ICA). Using this technique, EEG topographies can be substituted with microstate labels by competitive fitting based on spatial correlation, resulting in a symbolic, non-metric time series, the microstate sequence. Microstate topographies and symbolic time series are further analyzed statistically, including static and dynamic properties. Static properties, which do not contain information about temporal dependencies of the microstate sequence include the maximum similarity of microstate maps within and between the tested clustering algorithms, the global explained variance and the Shannon entropy of the microstate sequences. Dynamic properties are sensitive to temporal correlations between the symbols and include the mixing time of the microstate transition matrix, the entropy rate of the microstate sequences and the location of the first local maximum of the autoinformation function. We also test the Markov property of microstate sequences, the time stationarity of the transition matrix and detect periodicities by means of time-lagged mutual information. Finally, possible long-range correlations of microstate sequences are assessed via Hurst exponent estimation. We find that while static properties partially reflect properties of the clustering algorithms, information-theoretical quantities are largely invariant with respect to the clustering method used. As each clustering algorithm has its own profile of computational speed, ease of implementation, determinism vs. stochasticity and theoretical underpinnings, our results convey a positive message concerning the free choice of method and the comparability of results obtained from different algorithms. The invariance of these quantities implies that the tested properties are algorithm-independent, inherent features of resting state EEG derived microstate sequences.

Complex network comparison based on communicability sequence entropy.

Identification and quantification of the structural dissimilarities between complex networks is a very important and challenging problem in network science. A simple and efficient way to quantify network differences remains unexplored. Although the methods for network comparison based on the probability distribution of a network descriptor have been shown to be effective for specific purposes, the information they provide is often limited or incomplete. In order to overcome the defect of the methods, here we use the communicability between two nodes to define the communicability sequence entropy of networks, and on the basis of this entropy measure, we propose a Jensen-Shannon divergence of two networks which can be used as natural distance measure between complex networks. By the extensive experiments, we find that the measure can accurately quantify the structural dissimilarities between synthetic networks. Most importantly, the measure can be able to identify the critical percolation probability of the random network in the evolution process, and it can also effectively guide us to choose a more suitable model to simulate real systems.

Quasi-uniform convergence in dynamical systems generated by an amenable group action

We study properties of the Weyl pseudometric associated with an action of a countable amenable group on a compact metric space. We prove that the topological entropy and the number of minimal subsets of the closure of an orbit are both lower semicontinuous with respect to the Weyl pseudometric. Furthermore, the simplex of invariant measures supported on the orbit closure varies continuously. We apply the Weyl pseudometric to Toeplitz sequences for amenable residually finite groups. We introduce the notion of a regular Toeplitz sequence for an arbitrary amenable residually finite group and demonstrate that all regular Toeplitz sequences define minimal and uniquely ergodic systems. We prove path-connectivity of this family with respect to the Weyl pseudometric. This leads to a new proof of a theorem of Fabrice Krieger, which says that the values of entropy of Toeplitz sequences can have arbitrary finite entropy.

Identification of Pulmonary Hypertension Using Entropy Measure Analysis of Heart Sound Signal.

This study introduced entropy measures to analyze the heart sound signals of people with and without pulmonary hypertension (PH). The lead II Electrocardiography (ECG) signal and heart sound signal were simultaneously collected from 104 subjects aged between 22 and 89. Fifty of them were PH patients and 54 were healthy. Eleven heart sound features were extracted and three entropy measures, namely sample entropy (SampEn), fuzzy entropy (FuzzyEn) and fuzzy measure entropy (FuzzyMEn) of the feature sequences were calculated. The Mann–Whitney U test was used to study the feature significance between the patient and health group. To reduce the age confounding factor, nine entropy measures were selected based on correlation analysis. Further, the probability density function (pdf) of a single selected entropy measure of both groups was constructed by kernel density estimation, as well as the joint pdf of any two and multiple selected entropy measures. Therefore, a patient or a healthy subject can be classified using his/her entropy measure probability based on Bayes’ decision rule. The results showed that the best identification performance by a single selected measure had sensitivity of 0.720 and specificity of 0.648. The identification performance was improved to 0.680, 0.796 by the joint pdf of two measures and 0.740, 0.870 by the joint pdf of multiple measures. This study showed that entropy measures could be a powerful tool for early screening of PH patients.

Translation: The Universal Structural Core of Life.

The Universal Gene Set of Life (UGSL) is common to genomes of all extant organisms. The UGSL is small, consisting of <100 genes, and is dominated by genes encoding the translation system. Here we extend the search for biological universality to three dimensions. We characterize and quantitate the universality of structure of macromolecules that are common to all of life. We determine that around 90% of prokaryotic ribosomal RNA (rRNA) forms a common core, which is the structural and functional foundation of rRNAs of all cytoplasmic ribosomes. We have established a database, which we call the Sparse and Efficient Representation of the Extant Biology (the SEREB database). This database contains complete and cross-validated rRNA sequences of species chosen, as far as possible, to sparsely and efficiently sample all known phyla. Atomic-resolution structures of ribosomes provide data for structural comparison and validation of sequence-based models. We developed a similarity statistic called pairing adjusted sequence entropy, which characterizes paired nucleotides by their adherence to covariation and unpaired nucleotides by conventional conservation of identity. For canonically paired nucleotides the unit of structure is the nucleotide pair. For unpaired nucleotides, the unit of structure is the nucleotide. By quantitatively defining the common core of rRNA, we systematize the conservation and divergence of the translational system across the tree of life, and can begin to understand the unique evolutionary pressures that cause its universality. We explore the relationship between ribosomal size and diversity, geological time, and organismal complexity.

Entropy of high-order Markov chains beyond the pair correlations

The main goal of this paper is to develop an estimate for the entropy of random stationary ergodic symbolic sequences with elements belonging to a finite alphabet. We present here the detailed analytical study of the entropy for the high-order Markov chain in the bilinear approximation. In the limit of weak correlations, we present analytically the entropy of sequence by means of correlation functions of the second and third orders. The appendix contains a short comprehensive introduction into the subject of study.

H1N1 seasonal influenza virus evolutionary rate changed over time

It was previously shown that the seasonal H1N1 influenza virus antigenic drift occurred at a slower rate than the seasonal H3N2 virus during the first decade of the 21th century. It was hypothesized that the slower antigenic evolution led to a decrease in average ages of infection, which in turn resulted in lower level of global viral circulation. It is unclear what caused the difference between the two viruses, but a plausible explanation may be related to the fact that the H1N1 virus had been in human population for much longer than the H3N2 virus. This would suggest that H1N1 antigenic drift in an earlier period may have been different from a more recent period. To test this hypothesis, we analyzed seasonal H1N1 influenza sequences during various time periods. In comparison to more recent H1N1 virus, the older H1N1 virus during the first half of the 20th century showed evidences of higher nonsynnonymous/synonymous ration (dN/dS) in its hemagglutinin (HA) gene. We compared amino acid sequence changes in the HA epitopes for each outbreak season and found that there were less changes in later years. Amino acid sequence diversity in the epitopes as measured by sequence entropy became smaller for each passing decade. These suggest that there might be some limit to the antigenic drift. The longer an influenza virus has drifted in human population, the less flexibility it may become. With less flexibility to adapt and escape the host immunity, the virus may have to rely more on younger naïve population.

Effect of Dry Needling Stimulation of Myofascial Trigger Point on Sample Entropy of Electromyography of Gastrocnemius Injured Site in Rats

To attempt to establish an objective quantitative indicator to characterize the trigger point activity, so as to evaluate the effect of dry needling on myofascial trigger point activity. Twenty-four male Sprague-Dawley rats were randomly divided into blank control group, dry needling (needling) group, stretching exercise (stretching) group and needling plus stretching group (n＝6 per group). The chronic myofascial pain (trigger point) model was established by freedom vertical fall of a wooden striking device onto the mid-point of gastrocnemius belly of the left hind-limb to induce contusion, followed by forcing the rat to make a continuous downgrade running exercise at a speed of 16 m/min for 90 min on the next day which was conducted once a week for 8 weeks. Electromyography (EMG) of the regional myofascial injured point was monitored and recorded using an EMG recorder via electrodes. It was considered success of the model if spontaneous electrical activities appeared in the injured site. After a 4 weeks' recovery, rats of the needling group were treated by filiform needle stimulation (lifting-thrusting-rotating) of the central part of the injured gastrocnemius belly (about 10 mm deep) for 6 min, and those of the stretching group treated by holding the rat's limb to make the hip and knee joints to an angle of about 180°, and the ankle-joint about 90° for 1 min every time, 3 times altogether (with an interval of 1 min between every 2 times). The activity of the trigger point was estimated by the sample entropy of the EMG signal sequence in reference to Richman's and Moorman's methods to estimate the curative effect of both needling and exercise. After the modeling cycle, the mean sample entropies of EMG signals was significantly decreased in the model groups (needling group [0.034±0.010], stretching group [0.045±0.023], needling plus stretching group [0.047±0.034]) relevant to the blank control group (0.985±0.196, P<0.01). After the treatment, the mean sample entropy of EMG signals was evidently increased in both needling (0.819±0.088), stretching (0.532±0.25) and needling plus stretching (0.810±0.117) groups (P<0.01). The mean sample entropy of the needling and needling plus stretching groups were significantly higher than that of the stretching group (P<0.01), without remarkable difference between the two needling groups in the mean sample entropy (P>0.05), suggesting a better efficacy of dry needling in easing trigger point activity. Dry needling is able to relieve myofascial trigger point activity in rats, which is better than that of simple passive stretching therapy.

A Novel Entropy-Based Decoding Algorithm for a Generalized High-Order Discrete Hidden Markov Model

The optimal state sequence of a generalized High-Order Hidden Markov Model (HHMM) is tracked from a given observational sequence using the classical Viterbi algorithm. This classical algorithm is based on maximum likelihood criterion. We introduce an entropy-based Viterbi algorithm for tracking the optimal state sequence of a HHMM. The entropy of a state sequence is a useful quantity, providing a measure of the uncertainty of a HHMM. There will be no uncertainty if there is only one possible optimal state sequence for HHMM. This entropy-based decoding algorithm can be formulated in an extended or a reduction approach. We extend the entropy-based algorithm for computing the optimal state sequence that was developed from a first-order to a generalized HHMM with a single observational sequence. This extended algorithm performs the computation exponentially with respect to the order of HMM. The computational complexity of this extended algorithm is due to the growth of the model parameters. We introduce an efficient entropy-based decoding algorithm that used reduction approach, namely, entropy-based order-transformation forward algorithm (EOTFA) to compute the optimal state sequence of any generalized HHMM. This EOTFA algorithm involves a transformation of a generalized high-order HMM into an equivalent first-order HMM and an entropy-based decoding algorithm is developed based on the equivalent first-order HMM. This algorithm performs the computation based on the observational sequence and it requires OTN~2 calculations, where N~ is the number of states in an equivalent first-order model and T is the length of observational sequence.

Analysis of Conserved, Computationally Predicted Epitope Regions for VP5 and VP7 Across three Orbiviruses.

Orbiviruses are double-stranded RNA viruses that have profound economic and veterinary significance, 3 of the most important being African horse sickness virus (AHSV), bluetongue virus (BTV), and epizootic hemorrhagic disease virus (EHDV). Currently, vaccination and vector control are used as preventative measures; however, there are several problems with the current vaccines. Comparing viral amino acid sequences, we obtained an AHSV-BTV-EHDV consensus sequence for VP5 (viral protein 5) and for VP7 (viral protein 7) and generated homology models for these proteins. The structures and sequences were analyzed for amino acid sequence conservation, entropy, surface accessibility, and epitope propensity, to computationally determine whether consensus sequences still possess potential epitope regions. In total, 5 potential linear epitope regions on VP5 and 11 on VP7, as well as potential discontinuous B-cell epitopes, were identified and mapped onto the homology models created. Regions identified for VP5 and VP7 could be important in vaccine design against orbiviruses.

Relationship between Li-Yorke chaos and positive topological sequence entropy in nonautonomous dynamical systems

We study chaotic properties of uniformly convergent nonautonomous dynamical systems. We show that, contrary to the autonomous systems on the compact interval, positivity of topological sequence entropy and occurrence of Li-Yorke chaos are not equivalent, more precisely, neither of the two possible implications is true.

Substitution Rates Predicted by Stability-Constrained Models of Protein Evolution Are Not Consistent with Empirical Data.

Protein structures strongly influence molecular evolution. In particular, the evolutionary rate of a protein site depends on the number of its native contacts. Stability-constrained models of protein evolution consider this influence of protein structure on evolution by predicting the effect of mutations on the stability of the native state, but they currently neglect how mutations affect the protein structure. These models predict that buried protein sites with more native contacts are more constrained by natural selection and less variable, as observed. Nevertheless, previous work did not consider the stability against compact misfolded conformations, although it is known that the negative design that destabilizes these misfolded conformations influences protein evolution significantly. Here, we show that stability-constrained models that consider misfolding predict that site-specific sequence entropy and substitution rate peak at amphiphilic sites with an intermediate number of contacts, as these sites are less constrained than exposed sites with few contacts whose hydrophobicity must be limited. This result holds both for a mean-field model with independent sites and for a pairwise model that takes as a reference the wild-type sequence, but it contrasts with the observations that indicate that the entropy and the substitution rate decrease monotonically with the number of contacts. Our work suggests that stability-constrained models overestimate the tolerance of amphiphilic sites against mutations, either because of the limits of the free energy function or, more importantly in our opinion, because they do not consider how mutations perturb the native protein structure.

Sequence entropy of folding and the absolute rate of amino acid substitutions.

SummaryAdequate representations of protein evolution should consider how the acceptance of mutations depends on the sequence context in which they arise. However, epistatic interactions among sites in a protein result in time and spatial substitution rate heterogeneity beyond the capabilities of current models. Here, we exploit parallels between amino acid substitutions and chemical reaction kinetics to develop an improved theory of protein evolution. We constructed a mechanistic framework for modelling amino acid substitution rates that employs the formalisms of statistical mechanics, with population genetics principles underlying the analysis. Theoretical analyses and computer simulations of proteins under purifying selection for thermodynamic stability show that substitution rates and the stabilisation of resident amino acids (the ‘evolutionary Stokes shift’) can be predicted from biophysics and the effect of sequence entropy alone. Furthermore, we demonstrate that substitutions predominantly occur when epistatic interactions result in near neutrality; substitution rates are determined by how often epistasis results in such nearly neutral conditions. This theory provides a general framework for modelling protein sequence change under purifying selection, potentially explains patterns of convergence and mutation rates in real proteins that are incompatible with previous models, and provides a better null model for the detection of adaptive changes.

Abstract 1653: Modeling tumor immuno-dynamics to predict patient survival and immunotherapy efficacy

Abstract State of the art models of cancer survival are increasingly utilizing molecular data that feature the number of non-synonymous mutations, or a network based stratification of mutations, as significant predictors. It is widely hypothesized that this association between mutations and survival is a result of neo-epitopes that induce a robust anti-tumor immune response. Naively this implies that patient neo-epitope burden should better predict survival than that of mutations alone, however this has generally been difficult to prove. Indeed, mutations and predicted neo-epitopes explain roughly the same survival variance, likely due to the intertwined difficulties of neo-epitope prioritization and the often immuno-suppressive tumor microenvironment. Reasoning that integrating leading order immune response with novel neo-epitope prioritization will lead to superior performance, we built a statistical model to quantify the influence of tumor immune-dynamics on patient survival. Neoepitopes (MHC-I/II) were predicted from mutations and filtered on a self-ligandome. The clonal structure of these mutations and the distribution of resulting epitopes were assessed as predictive features. Additionally, the existing response to epitopes was assayed by incorporating TCR and BCR sequence counts and entropy as a proxy for clonal tumor infiltrating lymphocyte expansion. Leveraging the cancer genome atlas (TCGA) for 9 major cancer types including bladder (BLCA), breast (BRCA), colorectal (COAD/READ), glioblastoma (GBM), liver (LIHC), lung (NSCLC), melanoma (SKCM), pancreatic (PAAD), and uterine (UCEC) cancer, we obtained omics data for a total of 2,866 patients. Random forest based recursive feature elimination was used to determine which features were most likely to be predictive while overfitting was controlled with k-fold cross validation. A cox proportional hazard model of survival as a function of mutation burden and functions of neoepitope and immune response predictors was constructed using the selected features. Our results confirm that neoepitope and immune response based predictors for survival often significantly outperform mutation burden alone and simultaneously suggest a quantitative classification of immunotherapy efficacy across cancer subtypes. Citation Format: Nicholas K. Akers, Eric E. Schadt, Bojan Losic. Modeling tumor immuno-dynamics to predict patient survival and immunotherapy efficacy [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2017; 2017 Apr 1-5; Washington, DC. Philadelphia (PA): AACR; Cancer Res 2017;77(13 Suppl):Abstract nr 1653. doi:10.1158/1538-7445.AM2017-1653

A Remark on Topological Sequence Entropy

Let [Formula: see text] be the supremum of all topological sequence entropies of a dynamical system [Formula: see text]. This paper obtains the iteration invariance and commutativity of [Formula: see text] and proves that if [Formula: see text] is a multisensitive transformation defined on a locally connected space, then [Formula: see text]. As an application, it is shown that a Cournot map is Li–Yorke chaotic if and only if its topological sequence entropy relative to a suitable sequence is positive.

Leaf Recognition Based on DPCNN and BOW

Leaf classification is an interesting and important research. Current work focuses mainly on feature extraction, especially on textural feature extraction. In this case, we propose a new method of leaf recognition based on bag of words (BOW) and entropy sequence (EnS). In our method, EnS is firstly obtained by dual-output pulse-coupled neural network and then it is improved by BOW. Locality-constrained linear coding method is used for sparse coding. Then, the classification system is built where the linear support vector machine is taken as classifier. Some representative datasets and existing methods are employed to evaluate the effect of the proposed method. Finally, experimental results show that the accuracy of our proposed method is better than existing methods.

XSuLT: a web server for structural annotation and representation of sequence-structure alignments.

The web server XSuLT, an enhanced version of the protein alignment annotation program JoY, formats a submitted multiple-sequence alignment using three-dimensional (3D) structural information in order to assist in the comparative analysis of protein evolution and in the optimization of alignments for comparative modelling and construct design. In addition to the features analysed by JoY, which include secondary structure, solvent accessibility and sidechain hydrogen bonds, XSuLT annotates each amino acid residue with residue depth, chain and ligand interactions, inter-residue contacts, sequence entropy, root mean square deviation and secondary structure and disorder prediction. It is also now integrated with built-in 3D visualization which interacts with the formatted alignment to facilitate inspection and understanding. Results can be downloaded as stand-alone HTML for the formatted alignment and as XML with the underlying annotation data. XSuLT is freely available at http://structure.bioc.cam.ac.uk/xsult/.

Functional neuroanatomy of speech signal decoding in primary progressive aphasias

The pathophysiology of primary progressive aphasias remains poorly understood. Here, we addressed this issue using activation fMRI in a cohort of 27 patients with primary progressive aphasia (nonfluent, semantic, and logopenic variants) versus 15 healthy controls. Participants listened passively to sequences of spoken syllables in which we manipulated 3-key auditory speech signal characteristics: temporal regularity, phonemic spectral structure, and pitch sequence entropy. Relative to healthy controls, nonfluent variant patients showed reduced activation of medial Heschl's gyrus in response to any auditory stimulation and reduced activation of anterior cingulate to temporal irregularity. Semantic variant patients had relatively reduced activation of caudate and anterior cingulate in response to increased entropy. Logopenic variant patients showed reduced activation of posterior superior temporal cortex to phonemic spectral structure. Taken together, our findings suggest that impaired processing of core speech signal attributes may drive particular progressive aphasia syndromes and could index a generic physiological mechanism of reduced computational efficiency relevant to all these syndromes, with implications for development of new biomarkers and therapeutic interventions.

An effective recommendation model using user access sequence and context entropy

An effective recommendation model using user access sequence and context entropy

Predictability Analysis of Spectrum State Evolution: Performance Bounds and Real-World Data Analytics

Predictability in spectrum prediction refers to the degree to which a correct prediction of the radio spectrum state (RSS) can be made quantitatively. It is obvious that the possibility that the future RSS is accurately predicted will be different when using different spectrum prediction algorithms. However, the fundamental limits on the accuracy of various spectrum prediction algorithms should exist and be worthwhile to be paid attention to. In this paper, we define these fundamental limits as the performance bounds of predictability, which can be the important indexes when evaluating the performance of different spectrum prediction algorithms. Real-world spectrum data is involved to present comprehensive and profound analysis of the predictability. We first transform large amount of spectrum data into symbol sequences by sampling and quantization, to calculate the entropy of the symbol sequence, which represents the randomness of the RSS evolution. Then, we derive the upper bound and the lower bound of the predictability mainly from entropies of the symbol sequences. Further, we conduct the detailed analysis on the performance bounds of the predictability of the RSS. Based on real-world data analytics, the key insights among others include: 1) entropies almost have no relationship with selection of sampling intervals in the data preprocessing; 2) the upper and the lower bounds of the predictability will both decrease as the quantization level rises and tend to be stable around a value at last; and 3) two kinds of lower bounds of the predictability are proposed, and one of the lower bounds, the regularity $R$ , can reveal the tidal effect of the evolution of the RSS.