Entropy-based Algorithm Research Articles

Modification of entropy-based algorithm for determining words boundaries in noisy conditions

A modification of the algorithm for finding word boundaries in continuous speech based on an analysis of the entropy of a speech signal is described. At the preprocessing stage, the signal is divided into frames; the entropy value is calculated for each frame and compared with the threshold value of the entropy γ. The algorithm is carried out in two stages, first a rough and then more precise definition of the word boundaries is made. At each stage, its own minimum distance between words is used. A modification is proposed, which consists in the fact that the entropy threshold γ is determined on the basis of the calculated SNR coefficient taking into account the experimental selection of signal parameters at the stage of precise determination of word boundaries in a noisy speech signal. Checking the functionality of the modified algorithm on noisy signals has shown that taking into account the effect of SNR on the selection of signal parameters increases the accuracy of word boundary determination.

Land-Cover Classification Using MaxEnt: Can We Trust in Model Quality Metrics for Estimating Classification Accuracy?

MaxEnt is a popular maximum entropy-based algorithm originally developed for modelling species distribution, but increasingly used for land-cover classification. In this article, we used MaxEnt as a single-class land-cover classification and explored if recommended procedures for generating high-quality species distribution models also apply for generating high-accuracy land-cover classification. We used remote sensing imagery and randomly selected ground-true points for four types of land covers (built, grass, deciduous, evergreen) to generate 1980 classification maps using MaxEnt. We calculated different accuracy discrimination and quality model metrics to determine if these metrics were suitable proxies for estimating the accuracy of land-cover classification outcomes. Correlation analysis between model quality metrics showed consistent patterns for the relationships between metrics, but not for all land-covers. Relationship between model quality metrics and land-cover classification accuracy were land-cover-dependent. While for built cover there was no consistent patterns of correlations for any quality metrics; for grass, evergreen and deciduous, there was a consistent association between quality metrics and classification accuracy. We recommend evaluating the accuracy of land-cover classification results by using proper discrimination accuracy coefficients (e.g., Kappa, Overall Accuracy), and not placing all the confidence in model’s quality metrics as a reliable indicator of land-cover classification results.

An Entropy-Based Machine Learning Algorithm for Combining Macroeconomic Forecasts

This paper applies a Machine Learning approach with the aim of providing a single aggregated prediction from a set of individual predictions. Departing from the well-known maximum-entropy inference methodology, a new factor capturing the distance between the true and the estimated aggregated predictions presents a new problem. Algorithms such as ridge, lasso or elastic net help in finding a new methodology to tackle this issue. We carry out a simulation study to evaluate the performance of such a procedure and apply it in order to forecast and measure predictive ability using a dataset of predictions on Spanish gross domestic product.

P1880Entropy-based algorithm for atrial fibrillation detection using photoplethysomgraphic signal recorded by a smart watch

Abstract Background Atrial fibrillation (AF) is the most common arrhythmia, and its paroxysmal and short duration nature makes its detection challenging. The most important limitation of current smartwatches is that patients need to touch to the sensor of the watch to record signals when patients feel discomfort. We developed a wearable smart watch and evaluated its accuracy to differentiate AF from sinus rhythm, which can continuously detecting heart rhythm without hand touching the device. Methods and results A wearable smart watch with PPG sensor and electrocardiogram (ECG) recording function was used for signal acquisition. A total 399 patients with a mean age of 67 years old were enrolled in the study, of whom 237 (81.5%) were male, and 101 have been diagnosed with AF. Pulse wave extracted from the green light spectrum of the signal and ECG were recorded for about 10 minutes for each patient. Pulse-to-pulse intervals (PPI) were automatically identified. All ECG signals were verified by two cardiologists. The correlation between R-to-R interval on ECG and PPI were excellent, with a correlation coefficient R >0.99 (p<0.05). An entropy-based algorithm which combined Shannon entropy of successive difference of PPI and sample entropy of PPI was used to discriminate between AF and sinus rhythm. This method had high sensitivity and specificity (96% and 98%, respectively), the area under receiver operating characteristic curve reached 0.98. Conclusions We developed an entropy-based algorithm for AF detection with PPG signal recorded by a wearable smart watch. This algorithm discriminates AF from sinus rhythm accurately. This advance in technology overcomes an important clinical obstacle and can increase the AF detection rate tremendously.

An Entropy-Based Algorithm with Nonlocal Residual Learning for Image Compressive Sensing Recovery

Image recovery from compressive sensing (CS) measurement data, especially noisy data has always been challenging due to its implicit ill-posed nature, thus, to seek a domain where a signal can exhibit a high degree of sparsity and to design an effective algorithm have drawn increasingly more attention. Among various sparsity-based models, structured or group sparsity often leads to more powerful signal reconstruction techniques. In this paper, we propose a novel entropy-based algorithm for CS recovery to enhance image sparsity through learning the group sparsity of residual. To reduce the residual of similar packed patches, the group sparsity of residual is described by a Laplacian scale mixture (LSM) model, therefore, each singular value of the residual of similar packed patches is modeled as a Laplacian distribution with a variable scale parameter, to exploit the benefits of high-order dependency among sparse coefficients. Due to the latent variables, the maximum a posteriori (MAP) estimation of the sparse coefficients cannot be obtained, thus, we design a loss function for expectation–maximization (EM) method based on relative entropy. In the frame of EM iteration, the sparse coefficients can be estimated with the denoising-based approximate message passing (D-AMP) algorithm. Experimental results have shown that the proposed algorithm can significantly outperform existing CS techniques for image recovery.

Naturally occurring mutations in PB1 affect influenza A virus replication fidelity, virulence, and adaptability

BackgroundMutations in the PB1 subunit of RNA-dependent RNA polymerase (RdRp) of influenza A virus can affect replication fidelity. Before the influenza A/H1N1 pandemic in 2009, most human influenza A/H1N1 viruses contained the avian-associated residue, serine, at position 216 in PB1. However, near the onset of the 2009 pandemic, human viruses began to acquire the mammalian-associated residue, glycine, at PB1–216, and PB1–216G became predominant in human viruses thereafter.MethodsUsing entropy-based analysis algorithm, we have previously identified several host-specific amino-acid signatures that separated avian and swine viruses from human influenza viruses. The presence of these host-specific signatures in human influenza A/H1N1 viruses suggested that these mutations were the result of adaptive genetic evolution that enabled these influenza viruses to circumvent host barriers, which resulted in cross-species transmission. We investigated the biological impact of this natural avian-to-mammalian signature substitution at PB1–216 in human influenza A/H1N1 viruses.ResultsWe found that PB1–216G viruses had greater mutation potential, and were more sensitive to ribavirin than PB1–216S viruses. In oseltamivir-treated HEK293 cells, PB1–216G viruses generated mutations in viral neuraminidase at a higher rate than PB1–216S viruses. By contrast, PB1–216S viruses were more virulent in mice than PB1–216G viruses. These results suggest that the PB1-S216G substitution enhances viral epidemiological fitness by increasing the frequency of adaptive mutations in human influenza A/H1N1 viruses.ConclusionsOur results thus suggest that the increased adaptability and epidemiological fitness of naturally arising human PB1–216G viruses, which have a canonical low-fidelity replicase, were the biological mechanisms underlying the replacement of PB1–216S viruses with a high-fidelity replicase following the emergence of pdmH1N1. We think that continued surveillance of such naturally occurring PB1–216 variants among others is warranted to assess the potential impact of changes in RdRp fidelity on the adaptability and epidemiological fitness of human A/H1N1 influenza viruses.

Optimization of concrete cable-stayed bridges under seismic action

A computational tool is presented for the optimum design of concrete cable-stayed bridges under seismic action. The finite element method is used for the three-dimensional analysis of the structure under dead and live loads, time-dependent effects and geometrical nonlinearities are considered. The dynamic analysis is solved by the modal/spectral approach. The design problem is posed as a multi-criteria optimization. The design variables are the cable forces and the cross-sectional dimensions of cables, deck and towers. Design objectives of minimum cost, deflections, natural frequencies and stresses considering both, serviceability and ultimate limit states are considered. Given that a gradient-based algorithm is applied for the optimization the discrete direct method is used for sensitivity analysis. An entropy-based algorithm finds economically and structurally efficient solutions by rearranging the stiffness and mass distribution to enhance the structural response. Numerical examples illustrate the features of the proposed computational tool.

Leak detection of gas pipelines using acoustic signals based on wavelet transform and Support Vector Machine

Leak detection of gas pipelines has attracted extensive attention in recent years because such a leak could result in significant damage to society. This paper proposes an integrated leak detection method using acoustic signals based on wavelet transform and Support Vector Machine (SVM). Specifically, the optimal wavelet basis is selected by the entropy-based algorithm adaptively, with which acoustic signals gathered by acoustic sensors are first pre-processed by wavelet transform. Then useful features containing leak severity information are extracted from multi-domain components of the acoustic signals. Moreover, for leak detection and severity classification, the Relief-F algorithm is applied to select the most discriminative features. Furthermore, selected features are used as the input of SVM classifiers to identify the leak severity of gas pipelines. The effectiveness of the proposed method is validated using laboratory experiments. The results demonstrate that the proposed method achieves high accuracy of 99.4% to determine the leak state and non-leak state by using the first three most discriminative features and 95.6% to classify the normal and several leak severity conditions by using the first five most discriminative features. Therefore, it is effective for leak detection and promising for the development of a real-time monitoring system.

Tsalli’s Entropy-based Segmentation Method for Accurate Pigmented Skin Lesion Identification

Skin disease causes serious health hazards like cancer, leprosy, scleroderma, etc. Early and precise diagnosis can overcome the long-term effect of these diseases. Computer vision brings this precision with automated diagnostic system. The most crucial part of such a system is the segmentation of diseased regions. A novel segmentation algorithm is proposed in this study by comparing Tsalli’s and Sharma–Mittal entropy. Five well-known state-of-the-art segmentation techniques are compared with the selected one using Hubert and Adjusted Rand Index. The result shows that Tsalli’s entropy-based algorithm is significantly better for the pigmented lesion segmentation than the compared techniques.

Is Modern English becoming less inflectionally diversified? Evidence from entropy-based algorithm

Researchers have long noticed the phenomenon of inflection decay in the evolution of English, which may be driven by the Principle of Least Effort (Millward and Hayes, 2010; Zipf, 1949). However, most studies in this line of research were based on exemplar-oriented observations while few explored the phenomenon from a quantitative perspective. In addition, it remains unclear whether the trend of inflection decay has continued in Modern English. In the present study, an index named Inflectional Entropy was developed to measure the diachronic change of the inflectional trend of Modern English in the 20th century. A trend of inflection decay in Modern English was found based on experiments on several large-scale corpora. The findings provide quantitative evidence to linguistic theories such as Principle of Least Effort, Language Niche Hypothesis (Lupyan and Dale, 2010), and Principle of Entropy Minimization (Ferrer-i-Cancho, 2018).

Entropy evaluation sheds light on ecosystem complexity

Preserving biodiversity and ecosystem stability is a challenge that can be pursued through modern statistical mechanics modeling. Here we introduce a variational maximum entropy-based algorithm to evaluate the entropy in a minimal ecosystem on a lattice in which two species struggle for survival. The method quantitatively reproduces the scale-free law of the prey shoals size, where the simpler mean-field approach fails: the direct near neighbor correlations are found to be the fundamental ingredient describing the system self-organized behavior. Furthermore, entropy allows the measurement of structural ordering, that is found to be a key ingredient in characterizing two different coexistence behaviors, one where predators form localized patches in a sea of preys and another where species display more complex patterns. The general nature of the introduced method paves the way for its application in many other systems of interest.

New maximum entropy-based algorithm for structural design optimization

• A new algorithm based on nonlinear transformation is proposed to solve practical problems of reliability analysis. • Finite boundaries are obtained for numerical integration of the maximum entropy method through the transformation. • Typical examples are studied to confirm the efficiency of the proposed method. A new algorithm based on nonlinear transformation is proposed to improve the classical maximum entropy method and solve practical problems of reliability analysis. There are three steps in the new algorithm. Firstly, the performance function of reliability analysis is normalized, dividing by its value when each input is the mean value of the corresponding random variable. Then the nonlinear transformation of such normalized performance function is completed by using a monotonic nonlinear function with an adjustable parameter. Finally, the predictions of probability density function and/or the failure probability in reliability analysis are achieved by looking the result of transformation as a new form of performance function in the classical procedure of maximum entropy method in which the statistic moments are given through the univariate dimension reduction method. In the proposed method, the uncontrollable error of integration on the infinite interval is removed by transforming it into a bounded one. Three typical nonlinear transformation functions are studied and compared in the numerical examples. Comparing with results from Monte Carlo simulation, it is found that a proper choice of the adjustable parameter can lead to a better result of the prediction of failure probability. It is confirmed in the examples that result from the proposed method with the arctangent transformation function is better than the other transformation functions. The error of prediction of failure probability is controllable if the adjustable parameter is chosen in a given interval, but the suggested value of the adjustable parameter can only be given empirically.

An application of sample entropy to precipitation in Paraíba State, Brazil

A climate system is characterized to be a complex non-linear system. In order to describe the complex characteristics of precipitation series in Paraiba State, Brazil, we aim the use of sample entropy, a kind of entropy-based algorithm, to evaluate the complexity of precipitation series. Sixty-nine meteorological stations are distributed over four macroregions: Zona da Mata, Agreste, Borborema, and Sertao. The results of the analysis show that intricacies of monthly average precipitation have differences in the macroregions. Sample entropy is able to reflect the dynamic change of precipitation series providing a new way to investigate complexity of hydrological series. The complexity exhibits areal variation of local water resource systems which can influence the basis for utilizing and developing resources in dry areas.

Entropy-Based Algorithm for Supply-Chain Complexity Assessment

This paper considers a graph model of hierarchical supply chains. The goal is to measure the complexity of links between different components of the chain, for instance, between the principal equipment manufacturer (a root node) and its suppliers (preceding supply nodes). The information entropy is used to serve as a measure of knowledge about the complexity of shortages and pitfalls in relationship between the supply chain components under uncertainty. The concept of conditional (relative) entropy is introduced which is a generalization of the conventional (non-relative) entropy. An entropy-based algorithm providing efficient assessment of the supply chain complexity as a function of the SC size is developed.

A Novel Entropy-Based Decoding Algorithm for a Generalized High-Order Discrete Hidden Markov Model

The optimal state sequence of a generalized High-Order Hidden Markov Model (HHMM) is tracked from a given observational sequence using the classical Viterbi algorithm. This classical algorithm is based on maximum likelihood criterion. We introduce an entropy-based Viterbi algorithm for tracking the optimal state sequence of a HHMM. The entropy of a state sequence is a useful quantity, providing a measure of the uncertainty of a HHMM. There will be no uncertainty if there is only one possible optimal state sequence for HHMM. This entropy-based decoding algorithm can be formulated in an extended or a reduction approach. We extend the entropy-based algorithm for computing the optimal state sequence that was developed from a first-order to a generalized HHMM with a single observational sequence. This extended algorithm performs the computation exponentially with respect to the order of HMM. The computational complexity of this extended algorithm is due to the growth of the model parameters. We introduce an efficient entropy-based decoding algorithm that used reduction approach, namely, entropy-based order-transformation forward algorithm (EOTFA) to compute the optimal state sequence of any generalized HHMM. This EOTFA algorithm involves a transformation of a generalized high-order HMM into an equivalent first-order HMM and an entropy-based decoding algorithm is developed based on the equivalent first-order HMM. This algorithm performs the computation based on the observational sequence and it requires OTN~2 calculations, where N~ is the number of states in an equivalent first-order model and T is the length of observational sequence.

An Improved Mutation Series Entropy-based Algorithm for Evaluation of Innovation Ability of Enterprises

An Improved Mutation Series Entropy-based Algorithm for Evaluation of Innovation Ability of Enterprises

Increased chlorophyll-a concentration in the South China Sea caused by occasional sea surface temperature fronts at peripheries of eddies

ABSTRACTThis study investigates the processes of occasional sea surface temperature (SST) fronts and their impacts on chlorophyll-a concentration (chl-a) in the South China Sea (SCS), based on satellite remote sensing and in situ observations in 2009–2013. The SST fronts were detected by an entropy-based edge detection algorithm method from satellite-derived SST images with a 0.011° grid size. Three offshore SST front case studies (S1, S2 and S3) at the peripheries of eddies in the northern SCS were studied. In case S1 in September 2013, two SST fronts were detected with gradient magnitudes (GMs) greater than 0.06°C km–1 in the cyclonic eddy and 0.08°C km–1 in the periphery waters, and the fronts only existed for one and two days, respectively. After three and seven days, the high chl-a was found in the strong SST front waters which were about 51 and 54% higher than the concentration in the surrounding waters. The depth of the maximum chl-a elevated from the subsurface (50 m) to the surface. In case S2 in August 2013, two SST fronts were detected at the periphery of an anti-cyclonic eddy with GM stronger than 0.06°C km–1 and only existed for one day. After two days, the chl-a in the SST front waters was about 40% higher than the levels in the surrounding waters. In case S3 in June 2009, the GM of the eddy-feature SST front was stronger than 0.12°C km–1 and existed for three days, which was generated by tropical cyclone Linfa. The chl-a in the eddy-feature phytoplankton bloom was 6 times higher than in the surrounding waters. The results show that, in general, occasional offshore SST fronts at peripheries of eddies have stronger influence on surface chl-a, comparing to those seasonal coastal and permanent offshore SST fronts, via ‘Wind Pump’ effects.

An Integrated Dictionary-Learning Entropy-Based Medical Image Fusion Framework

Image fusion is widely used in different areas and can integrate complementary and relevant information of source images captured by multiple sensors into a unitary synthetic image. Medical image fusion, as an important image fusion application, can extract the details of multiple images from different imaging modalities and combine them into an image that contains complete and non-redundant information for increasing the accuracy of medical diagnosis and assessment. The quality of the fused image directly affects medical diagnosis and assessment. However, existing solutions have some drawbacks in contrast, sharpness, brightness, blur and details. This paper proposes an integrated dictionary-learning and entropy-based medical image-fusion framework that consists of three steps. First, the input image information is decomposed into low-frequency and high-frequency components by using a Gaussian filter. Second, low-frequency components are fused by weighted average algorithm and high-frequency components are fused by the dictionary-learning based algorithm. In the dictionary-learning process of high-frequency components, an entropy-based algorithm is used for informative blocks selection. Third, the fused low-frequency and high-frequency components are combined to obtain the final fusion results. The results and analyses of comparative experiments demonstrate that the proposed medical image fusion framework has better performance than existing solutions.

Changes in Electroencephalography Complexity using a Brain Computer Interface-Motor Observation Training in Chronic Stroke Patients: A Fuzzy Approximate Entropy Analysis.

Entropy-based algorithms have been suggested as robust estimators of electroencephalography (EEG) predictability or regularity. This study aimed to examine possible disturbances in EEG complexity as a means to elucidate the pathophysiological mechanisms in chronic stroke, before and after a brain computer interface (BCI)-motor observation intervention. Eleven chronic stroke subjects and nine unimpaired subjects were recruited to examine the differences in their EEG complexity. The BCI-motor observation intervention was designed to promote functional recovery of the hand in stroke subjects. Fuzzy approximate entropy (fApEn), a novel entropy-based algorithm designed to evaluate complexity in physiological systems, was applied to assess the EEG signals acquired from unimpaired subjects and stroke subjects, both before and after training. The results showed that stroke subjects had significantly lower EEG fApEn than unimpaired subjects (p < 0.05) in the motor cortex area of the brain (C3, C4, FC3, FC4, CP3, and CP4) in both hemispheres before training. After training, motor function of the paretic upper limb, assessed by the Fugl-Meyer Assessment-Upper Limb (FMA-UL), Action Research Arm Test (ARAT), and Wolf Motor Function Test (WMFT) improved significantly (p < 0.05). Furthermore, the EEG fApEn in stroke subjects increased considerably in the central area of the contralesional hemisphere after training (p < 0.05). A significant correlation was noted between clinical scales (FMA-UL, ARAT, and WMFT) and EEG fApEn in C3/C4 in the contralesional hemisphere (p < 0.05). This finding suggests that the increase in EEG fApEn could be an estimator of the variance in upper limb motor function improvement. In summary, fApEn can be used to identify abnormal EEG complexity in chronic stroke, when used with BCI-motor observation training. Moreover, these findings based on the fApEn of EEG signals also expand the existing interpretation of training-induced functional improvement in stroke subjects. The entropy-based analysis might serve as a novel approach to understanding the abnormal cortical dynamics of stroke and the neurological changes induced by rehabilitation training.

A local entropy-based algorithm for information content extraction from time–frequency distributions of noisy signals

In this paper, an automatic adaptive method for identification and separation of the useful information content, from the background noise of time–frequency distributions (TFD) of multicomponent nonstationary signals, is presented. The method is based on an initial segmentation of the TFD information content by the K-means clustering algorithm, that partitions the initial data set in order to obtain K classes containing elements with similar amplitudes. It is shown that the local Rényi entropy (LRE) can accurately distinguish classes containing noise from classes with the useful information content, as a consequence of their basic structural differences in the time–frequency plane. Simulations are run to compare the performance of the proposed adaptive algorithm for blind separation of useful information from background noise (i.e. blind amplitude threshold) and non-adaptive (hard) amplitude TFD threshold procedures. Simulation results indicate that the proposed method performs better or closely to the best of five blindly chosen hard thresholds. The limitation of efficient hard-thresholding is the need of previous knowledge of the signal's structure and SNR or visual evaluation.