Cumulative Entropy Research Articles

A parametric approach to acoustic entropy estimation for assessment of fatigue damage

A parametric approach to estimating the acoustic entropy detected over the course of fatigue damage is presented. Information entropy and relative entropy is estimated through a parametric approach where trial probability density functions (PDFs) are fitted to each individual received acoustic signal as the material degrades over the cycles of loading. The PDF that produces the maximum cumulative entropy is selected to model the signals. This selection criterion is due to the fact that the PDF with higher cumulative entropy results in less bias during the selection process. The evolution trends of both information entropy and relative entropy show the stages of fatigue damage observed in the fatigue indicators such as change in hardness. The acoustic entropy has an advantage over the conventional indices of damage as it can be employed directly in the online sensor based structural health monitoring schemes as a diagnosis feature.

Generalized cumulative entropy based on [formula omitted]th lower record values

In this paper, we propose a new extension of cumulative entropy based on kth lower record values. We also consider a dynamic version of this new cumulative entropy using the past lifetime. For these concepts, we study some properties and characterization results under some assumptions. Finally, we provide some examples of application of the results.

An Extension of Weighted Cumulative Inaccuracy Measure Based on Doubly Truncation

In the present communication, based on two extensions of Shannon entropy cumulative entropy by Rao, et al, 2004 and Kerridge’s inaccuarcy by Kerridge, (1961), we study and obtain some properties of weighted cumulative residual (past) inaccuracy for truncated random variables.‎ Several properties, including monotonicity, bounds and their connections to proportional (reversed) hazard model are obtained for left, right and doubly truncated random variables.‎ ‎‎ ‎

Unbiased Multivariate Correlation Analysis

Correlation measures are a key element of statistics and machine learning, and essential for a wide range of data analysis tasks. Most existing correlation measures are for pairwise relationships, but real-world data can also exhibit complex multivariate correlations, involving three or more variables. We argue that multivariate correlation measures should be comparable, interpretable, scalable and unbiased. However, no existing measures satisfy all these requirements. In this paper, we propose an unbiased multivariate correlation measure, called UMC, which satisfies all the above criteria. UMC is a cumulative entropy based non-parametric multivariate correlation measure, which can capture both linear and non-linear correlations for groups of three or more variables. It employs a correction for chance using a statistical model of independence to address the issue of bias. UMC has high interpretability and we empirically show it outperforms state-of-the-art multivariate correlation measures in terms of statistical power, as well as for use in both subspace clustering and outlier detection tasks.

BIVARIATE DYNAMIC CUMULATIVE RESIDUAL TSALLIS ENTROPY

Recently, Sati and Gupta (2015) proposed two measures of uncertainty based on non-extensive entropy, called the dynamic cumulative residual Tsallis entropy (DCRTE) and the empirical cumulative Tsallis entropy. In the present paper, we extend the definition of DCRTE into the bivariate setup and study its properties in the context of reliability theory. We also define a new class of life distributions based on bivariate DCRTE.

Using spectral and cumulative spectral entropy to classify anomalous diffusion in Sephadex™ gels

Sephadex™ gel beads are commonly used to separate mixtures of similar molecules based on trapping and size exclusion from internal submicron diameter cavities. Water, as it freely moves through the porous gel and enclosed chambers of Sephadex™ beads, exhibits both normal (Gaussian) and anomalous (non-Gaussian) water diffusion. The apparent diffusion coefficient (ADC) of water in Sephadex™ gels can be measured using magnetic resonance imaging (MRI) by applying diffusion-weighted pulse sequences. This study investigates the relationship between the ADC of water and the complexity (i.e., size and number of cavities) of a series of Sephadex™ beads. We first classified the stochastic movement of water by using the solution to the space and time fractional diffusion equation to extract the ADC and the fractional time and space parameters (α, β), which are essentially the order of the respective fractional derivatives in Fick’s second law. From the perspective of the continuous time random walk (CTRW) model of anomalous diffusion, these parameters reflect waiting times (trapping) and jump increments (nano-flow) of the water in the gels. The observed MRI diffusion signal decay represents the Fourier transform of the diffusion propagator (i.e., the characteristic function of the stochastic process). In two series of Sephadex™ gel beads, we observed a strong inverse correlation between bead porosity (which is also responsible for molecular size exclusion) and the fractional order parameters; as the gels become more heterogeneous, the ADC decreases, both α and β are reduced and the diffusion exhibits anomalous (sub-diffusion) behavior. In addition, as a new measure for the structural complexity in Sephadex™ gel beads, we propose using the spectral and the cumulative spectral entropy that are derived from the observed characteristic function. We find that both measures of entropy increase with the porosity and tortuosity of the gel in a manner consistent with fractional order diffusional dynamics.

Viral Phylogenomics Using an Alignment-Free Method: A Three-Step Approach to Determine Optimal Length of k-mer

The development of rapid, economical genome sequencing has shed new light on the classification of viruses. As of October 2016, the National Center for Biotechnology Information (NCBI) database contained >2 million viral genome sequences and a reference set of ~4000 viral genome sequences that cover a wide range of known viral families. Whole-genome sequences can be used to improve viral classification and provide insight into the viral “tree of life”. However, due to the lack of evolutionary conservation amongst diverse viruses, it is not feasible to build a viral tree of life using traditional phylogenetic methods based on conserved proteins. In this study, we used an alignment-free method that uses k-mers as genomic features for a large-scale comparison of complete viral genomes available in RefSeq. To determine the optimal feature length, k (an essential step in constructing a meaningful dendrogram), we designed a comprehensive strategy that combines three approaches: (1) cumulative relative entropy, (2) average number of common features among genomes, and (3) the Shannon diversity index. This strategy was used to determine k for all 3,905 complete viral genomes in RefSeq. The resulting dendrogram shows consistency with the viral taxonomy of the ICTV and the Baltimore classification of viruses.

Audio texturedness indicator based on a direct and reverse short listening time analysis

In this paper, we present an objective evaluation of audio texturedness level motivated by a subjective study which emphasizes the relevance of a direct and reverse short listening time analysis. This study was based on 77 undergraduate engineering students, where the concept of audio texturedness relied on audio and image analogies. Audio texturedness evaluation of a large audio database using a discrete [1 - 5] texturedness scale was performed. As a result, an objective audio texturedness indicator is proposed based on the cumulative average signal informational content change in the direct and reverse directions. This bidirectional cumulative entropy tracking was done in analogy with a classical multidirectional homogeneity method for image texture discrimination. As expected, the proposed indicator has ranked the class of noise on the high end of the [1 - 5] scale, whereas highly motivational speech signals ranked low on the proposed scale due to the large variations in their average informational content in the direct and reverse directions. This audio texturedness indicator offers a continuum of audio classes, in contrast with the classical noise, speech, and music sound categorization. The relevance of the proposed objective indicator auditory parameters was explored for a maximum objective-subjective cross-correlation and illustrated in a preliminary audio stream segmentation application.

On Cumulative Residual Renyi Entropy of Consecutive Order Statistics

Rao et al. (2004, IEEE Transactions on Information Theory, 50, 1220–1228). proposed cumulative residual entropy (CRE): a new and robust measure of information. Recently Sunoj and Linu (2012, Statistics, 46:1, 41–56,) discussed the cumulative residual Renyi’s entropy CRREα as a generalized of CRE. In this paper, we first provide a computational method that represents the joint CRREα of the first i order statistics as a single integral and it further, simplifying to simple expression as sum of the beta distributions. We then derive some recurrence relations for the CRREα in a consecutive order statistics to facilitate the computation. We also develop a simple formula for the sample estimate of the CRREα of a consecutive order statistics.

Some properties of cumulative Tsallis entropy of order $$\alpha $$ α

Tsallis entropy of order \(\alpha \) (see Tsallis in J Stat Phys 52(1–2):479–487, 1988) plays an important role in the measurement uncertainty of random variables. Recently, Sati and Gupta (J Probab Stat, doi:10.1155/2015/694203, 2015) introduced a cumulative Tsallis entropy of order \(\alpha \) and studied its various properties in the context of reliability modeling. In this paper, we introduce an alternate measure of cumulative Tsallis entropy of order \(\alpha \) and study its properties. Unlike the cumulative Tsallis entropy due to Sati and Gupta (J Probab Stat, doi:10.1155/2015/694203, 2015), the proposed measure has some additional features and has simple relationships with other important information and reliability measures.

Seismic systems, entropy production law, and ensembles of strong earthquakes

The regularities are revealed which, based on seismic statistics, govern the evolution of the dynamics in the preparation of strong earthquakes within discrete closed volumes of the lithosphere (seismic systems, SSs). This concept was previously formulated as a phenomenological approach based on seismic entropy for real-time monitoring the strong earthquakes. The seismic parameters of cumulative energy and entropy contain the energy–time memory about the previous earthquakes within the considered volume, which allows the real-time description of the nonequilibrium spatiotemporal dynamics of the active volume of the Earth’s lithosphere. The power law relating the information entropy and cumulative energy is theoretically derived, which supports the previously established empirical relationships. In the SSs, the strong earthquakes pertaining to a certain magnitude interval are united into the ensembles forming the seismic cycles which periodically restore the equilibrium state of the system. With the lapse of time, the strong earthquakes of the ensemble tend to fill up the SS. The configuration, the size of the SS, and the minimal parameter of action lead to the selectivity and discreteness of the notion of magnitude of an earthquake, depending on the scale of the system. Each SS has a certain minimal threshold magnitude above which the Gutenberg–Richter law is violated, whereas the indicator earthquakes, governed by the Poisson distribution within the scope of the entropy production law, play a crucial role in the preparation of the strong earthquakes. Some results are illustrated by the example of the recent strong earthquakes in Central Kuriles.

Cumulative ratio information based on general cumulative entropy

ABSTRACTIn this paper, we suggest an extension of the cumulative residual entropy (CRE) and call it generalized cumulative entropy. The proposed entropy not only retains attributes of the existing uncertainty measures but also possesses the absolute homogeneous property with unbounded support, which the CRE does not have. We demonstrate its mathematical properties including the entropy of order statistics and the principle of maximum general cumulative entropy. We also introduce the cumulative ratio information as a measure of discrepancy between two distributions and examine its application to a goodness-of-fit test of the logistic distribution. Simulation study shows that the test statistics based on the cumulative ratio information have comparable statistical power with competing test statistics.

Cumulative Paired φ-Entropy

A new kind of entropy will be introduced which generalizes both the differential entropy and the cumulative (residual) entropy. The generalization is twofold. First, we simultaneously define the entropy for cumulative distribution functions (cdfs) and survivor functions (sfs), instead of defining it separately for densities, cdfs, or sfs. Secondly, we consider a general “entropy generating function” φ, the same way Burbea et al. (IEEE Trans. Inf. Theory 1982, 28, 489–495) and Liese et al. (Convex Statistical Distances; Teubner-Verlag, 1987) did in the context of φ-divergences. Combining the ideas of φ-entropy and cumulative entropy leads to the new “cumulative paired φ-entropy” ( C P E φ ). This new entropy has already been discussed in at least four scientific disciplines, be it with certain modifications or simplifications. In the fuzzy set theory, for example, cumulative paired φ-entropies were defined for membership functions, whereas in uncertainty and reliability theories some variations of C P E φ were recently considered as measures of information. With a single exception, the discussions in the scientific disciplines appear to be held independently of each other. We consider C P E φ for continuous cdfs and show that C P E φ is rather a measure of dispersion than a measure of information. In the first place, this will be demonstrated by deriving an upper bound which is determined by the standard deviation and by solving the maximum entropy problem under the restriction of a fixed variance. Next, this paper specifically shows that C P E φ satisfies the axioms of a dispersion measure. The corresponding dispersion functional can easily be estimated by an L-estimator, containing all its known asymptotic properties. C P E φ is the basis for several related concepts like mutual φ-information, φ-correlation, and φ-regression, which generalize Gini correlation and Gini regression. In addition, linear rank tests for scale that are based on the new entropy have been developed. We show that almost all known linear rank tests are special cases, and we introduce certain new tests. Moreover, formulas for different distributions and entropy calculations are presented for C P E φ if the cdf is available in a closed form.

ON GENERALIZED CUMULATIVE ENTROPIES

In the present paper, we introduce a generalization of the cumulative entropy proposed by Di Crescenzo and Longobardi [8]. This new notion is related to the lower records and the reversed relevation transform. Dynamic version of the newly proposed measure is considered. Several properties including the effect of linear transformations, a two-dimensional version, a normalized version, bounds, stochastic ordering, etc. are studied for the generalized cumulative entropy (GCE). Similar results are obtained for the dynamic GCE. Various relationships with other functions are derived. A class of distributions is introduced and several properties are studied. Finally, empirical GCE is proposed to estimate the newly proposed information measure.

Bivariate extension of (dynamic) cumulative past entropy

ABSTRACTRecently, the concept of cumulative residual entropy (CRE) has been studied by many researchers in higher dimensions. In this article, we extend the definition of (dynamic) cumulative past entropy (DCPE), a dual measure of (dynamic) CRE, to bivariate setup and obtain some of its properties including bounds. We also look into the problem of extending DCPE for conditionally specified models. Several properties, including monotonicity, and bounds of DCPE are obtained for conditional distributions. It is shown that the proposed measure uniquely determines the distribution function. Moreover, we also propose a stochastic order based on this measure.

On generalized dynamic cumulative past entropy measure

ABSTRACTRecently, the concept of dynamic cumulative residual entropy and its generalizations has gained much attention among researchers. In this work, a new generalized dynamic cumulative measure in the past lifetime is proposed. Further, some characterization results connecting this new generalized dynamic entropy measure and other reversed measures are obtained.

Quantile-based cumulative entropies

ABSTRACTIt is well known that Shannon’s entropy plays an important role in the measurement of uncertainty of probability distributions. However, in certain situations Shannon entropy is not appropriate to measure the uncertainty and therefore an alternative measure has been introduced called cumulative residual entropy, based on the survival function (sf) instead of the probability density function (pdf) f(x) used in Shannon’s entropy. In the present paper, we introduce and study quantile versions of the cumulative entropy functions in the residual and past lifetimes. Unlike the cumulative entropies based on sf, the quantile-based cumulative entropy measures uniquely determine the underlying probability distribution.

On Shift-Dependent Cumulative Entropy Measures

Measures of cumulative residual entropy (CRE) and cumulative entropy (CE) about predictability of failure time of a system have been introduced in the studies of reliability and life testing. In this paper, cumulative distribution and survival function are used to develop weighted forms of CRE and CE. These new measures are denominated as weighted cumulative residual entropy (WCRE) and weighted cumulative entropy (WCE) and the connections of these new measures with hazard and reversed hazard rates are assessed. These information-theoretic uncertainty measures are shift-dependent and various properties of these measures are studied, including their connections with CRE, CE, mean residual lifetime, and mean inactivity time. The notions of weighted mean residual lifetime (WMRL) and weighted mean inactivity time (WMIT) are defined. The connections of weighted cumulative uncertainties with WMRL and WMIT are used to calculate the cumulative entropies of some well-known distributions. The joint versions of WCE and WCRE are defined which have the additive properties similar to those of Shannon entropy for two independent random lifetimes. The upper boundaries of newly introduced measures and the effect of linear transformations on them are considered. Finally, empirical WCRE and WCE are proposed by virtue of sample mean, sample variance, and order statistics to estimate the new measures of uncertainty. The consistency of these estimators is studied under specific choices of distributions.

Nonuniform Sparse Data Clustering Cascade Algorithm Based on Dynamic Cumulative Entropy

A small amount of prior knowledge and randomly chosen initial cluster centers have a direct impact on the accuracy of the performance of iterative clustering algorithm. In this paper we propose a new algorithm to compute initial cluster centers for k-means clustering and the best number of the clusters with little prior knowledge and optimize clustering result. It constructs the Euclidean distance control factor based on aggregation density sparse degree to select the initial cluster center of nonuniform sparse data and obtains initial data clusters by multidimensional diffusion density distribution. Multiobjective clustering approach based on dynamic cumulative entropy is adopted to optimize the initial data clusters and the best number of the clusters. The experimental results show that the newly proposed algorithm has good performance to obtain the initial cluster centers for the k-means algorithm and it effectively improves the clustering accuracy of nonuniform sparse data by about 5%.

Extension of the past lifetime and its connection to the cumulative entropy

Given two absolutely continuous nonnegative independent random variables, we define the reversed relevation transform as dual to the relevation transform. We first apply such transforms to the lifetimes of the components of parallel and series systems under suitably proportionality assumptions on the hazard rates. Furthermore, we prove that the (reversed) relevation transform is commutative if and only if the proportional (reversed) hazard rate model holds. By repeated application of the reversed relevation transform we construct a decreasing sequence of random variables which leads to new weighted probability densities. We obtain various relations involving ageing notions and stochastic orders. We also exploit the connection of such a sequence to the cumulative entropy and to an operator that is dual to the Dickson-Hipp operator. Iterative formulae for computing the mean and the cumulative entropy of the random variables of the sequence are finally investigated.