Joint Entropy Research Articles

Exploring the vulnerability of transportation networks by entropy: A case study of Asia–Europe maritime transportation network

With the rise of global trade, maritime transportation networks have become an indispensable element of logistics networks. Approximately 80% of the global trade volume is transported by sea with the maritime logistics network fueling global economic integration. Due to the uncertainty of the transportation process and the impact of accidents, the reliability analysis of the logistics network is a topic of immense interest. In this paper, we propose an original and novel model to quantitatively analyze the vulnerability of the maritime logistics network by considering the importance of each port in the network. Three centrality measures that consider different topology information of the network are used in this paper to identify the importance of ports. Different information about the network is considered through the joint entropy and the multiscale factor q to evaluate the vulnerability of the logistics network. The Asia–Europe maritime transportation network serves as a real-world example to demonstrate the effectiveness and applicability of our proposed model. The experimental results suggest that the performance of the maritime network is closely related to the heterogeneity of the connectivity pattern and the process of decentralization can reduce the vulnerability of the maritime network.

RF Fingerprint Extraction Method Based on CEEMDAN and Multidomain Joint Entropy

Specific emitter identification (SEI) can distinguish communication radio emitters with the fingerprint features carried by the received signal, and this technology has been widely used in military and civilian fields. However, in the real electromagnetic environment, the number of communication radio emitters is large and the signal-to-noise ratio (SNR) is low, which leads to poor nonlinear fingerprint analysis of SEI in a single domain. Therefore, combining the exploration of multiple domains of electromagnetic spatial information resources, this paper proposed a radio frequency (RF) fingerprint extraction method based on complementary ensemble empirical mode decomposition with adaptive noise (CEEMDAN) and multidomain joint entropy. The proposed method is an attempt and exploration further extraction of nonlinear fingerprint features in multiple domains. Firstly, considering the nonstationarity of the communication signal, this article adopts the CEEMDAN method to decompose the signal to multiple intrinsic mode functions (IMF). Then, the decomposed signal is represented in multiple spaces by a multidimensional phase space reconstruction technique. Nonlinear analysis of the original signal is performed in multiple spaces: multidimensional differential approximate entropy space, singular spectral entropy space, and power spectral entropy space. Finally, the support vector machine (SVM) is adopted in the classification stage. To demonstrate the robustness of the method, the method is verified on the universal software radio peripheral (USRP) dataset and the Northeastern University public dataset. In terms of the identification accuracy, the proposed method performs with 98.5% accuracy on the 5-class USRP dataset. It also performs with 94.7% accuracy on the 16-class public dataset. The experimental results show that the proposed method has a stable identification performance and has a more than 85% recognition rate in the SNR above 5dB.

Discrimination of secondary hypsarrhythmias to Zika virus congenital syndrome and west syndrome based on joint moments and entropy measurements.

Hypsarrhythmia is a specific chaotic morphology, present in the interictal period of the electroencephalogram (EEG) signal in patients with West Syndrome (WS), a severe form of childhood epilepsy and that, recently, was also identified in the examinations of patients with Zika Virus Congenital Syndrome (ZVCS). This innovative work proposes the development of a computational methodology for analysis and differentiation, based on the time-frequency domain, between the chaotic pattern of WS and ZVCS hypsarrhythmia. The EEG signal time-frequency analysis is carried out from the Continuous Wavelet Transform (CWT). Four joint moments—joint mean—mu _{(t,f)}, joint variance—sigma _{(t,f)}^2, joint skewness—lambda _{(t,f)}, and joint kurtosis—kappa _{(t,f)}—and four entropy measurements—Shannon, Log Energy, Norm, and Sure—are obtained from the CWT to compose the representative feature vector of the EEG hypsarrhythmic signals under analysis. The performance of eight classical types of machine learning algorithms are verified in classification using the k-fold cross validation and leave-one-patient-out cross validation methods. Discrimination results provided 78.08% accuracy, 85.55% sensitivity, 73.21% specificity, and AUC = 0.89 for the ANN classifier.

Synergistic impact of motion and acquisition/reconstruction parameters on 18 F-FDG PET radiomic features in non-small cell lung cancer: Phantom and clinical studies.

ObjectivesThis study is aimed at examining the synergistic impact of motion and acquisition/reconstruction parameters on 18F‐FDG PET image radiomic features in non‐small cell lung cancer (NSCLC) patients, and investigating the robustness of features performance in differentiating NSCLC histopathology subtypes.MethodsAn in‐house developed thoracic phantom incorporating lesions with different sizes was used with different reconstruction settings, including various reconstruction algorithms, number of subsets and iterations, full‐width at half‐maximum of post‐reconstruction smoothing filter and acquisition parameters, including injected activity and test–retest with and without motion simulation. To simulate motion, a special motor was manufactured to simulate respiratory motion based on a normal patient in two directions. The lesions were delineated semi‐automatically to extract 174 radiomic features. All radiomic features were categorized according to the coefficient of variation (COV) to select robust features. A cohort consisting of 40 NSCLC patients with adenocarcinoma (n = 20) and squamous cell carcinoma (n = 20) was retrospectively analyzed. Statistical analysis was performed to discriminate robust features in differentiating histopathology subtypes of NSCLC lesions.ResultsOverall, 29% of radiomic features showed a COV ≤5% against motion. Forty‐five percent and 76% of the features showed a COV ≤ 5% against the test–retest with and without motion in large lesions, respectively. Thirty‐three percent and 45% of the features showed a COV ≤ 5% against different reconstruction parameters with and without motion, respectively. For NSCLC histopathological subtype differentiation, statistical analysis showed that 31 features were significant (p‐value < 0.05). Two out of the 31 significant features, namely, the joint entropy of GLCM (AUC = 0.71, COV = 0.019) and median absolute deviation of intensity histogram (AUC = 0.7, COV = 0.046), were robust against the motion (same reconstruction setting).ConclusionsMotion, acquisition, and reconstruction parameters significantly impact radiomic features, just as their synergies. Radiomic features with high predictive performance (statistically significant) in differentiating histopathological subtype of NSCLC may be eliminated due to non‐reproducibility.

Sensor configuration optimization based on the entropy of adjoint concentration distribution for stochastic source term estimation in urban environment

Sensor monitoring is foundational to source term estimation (STE). Recently, stochastic STE methods based mainly on Bayesian inference have been attracting attention. However, few studies have developed sensor configuration optimization (SCO) methods aiming to ensure good STE performance in the monitoring area, such that the posterior probability could aggregate around the truths while addressing most sources. This research proposes a deployment method by designing an objective function and applying a simulated annealing (SA) algorithm. The objective function is set as the information joint entropy of the adjoint concentration. The performance of the proposed method was assessed by Bayesian inference STE for 25 unknown sources based on the obtained optimal configuration in a regular block-arrayed building group model. The STE results were compared with those of uniform and random configurations. According to the results, because the optimal configuration can provide the most informative measurements, it yields the best estimations. If we quantize the STE errors and take the average for all unknown sources, the location and strength estimation errors of the optimal configuration significantly decrease by approximately 45% and 99%, respectively, when compared with the random configuration and 39% and 96%, respectively, when compared with the uniform configuration.

Transfer Entropy of West Pacific Earthquakes to Inner Van Allen Belt Electron Bursts

Lithosphere-ionosphere non-linear interactions create a complex system where links between different phenomena can remain hidden. The statistical correlation between West Pacific strong earthquakes and high-energy electron bursts escaping trapped conditions was demonstrated in past works. Here, it is investigated from the point of view of information. Starting from the conditional probability statistical model, which was deduced from the correlation, the Shannon entropy, the joint entropy, and the conditional entropy are calculated. Time-delayed mutual information and transfer entropy have also been calculated analytically here for binary events: by including correlations between consecutive earthquake events, and between consecutive earthquakes and electron bursts. These quantities have been evaluated for the complex dynamical system of lithosphere-ionosphere; although the expressions calculated by probabilities resulted in being valid for each pair of binary events. Peaks occurred for the same time delay as in the correlations, Δt = 1.5–3.5 h, and as well as for a new time delay, Δt = −58.5–−56.5 h, for the transfer entropy; this last is linked to EQ self-correlations from the analysis. Even if the low number of self-correlated EQs makes this second peak insignificant in this case, it is of interest to separate the non-linear contribution of the transfer entropy of binary events in the study of a complex system.

Anticipating synchrony in dynamical systems using information theory.

Synchronization in coupled dynamical systems has been a well-known phenomenon in the field of nonlinear dynamics for a long time. This phenomenon has been investigated extensively both analytically and experimentally. Although synchronization is observed in different areas of our real life, in some cases, this phenomenon is harmful; consequently, an early warning of synchronization becomes an unavoidable requirement. This paper focuses on this issue and proposes a reliable measure ( R), from the perspective of the information theory, to detect complete and generalized synchronizations early in the context of interacting oscillators. The proposed measure R is an explicit function of the joint entropy and mutual information of the coupled oscillators. The applicability of R to anticipate generalized and complete synchronizations is justified using numerical analysis of mathematical models and experimental data. Mathematical models involve the interaction of two low-dimensional, autonomous, chaotic oscillators and a network of coupled Rössler and van der Pol oscillators. The experimental data are generated from laboratory-scale turbulent thermoacoustic systems.

Mutual Information between Order Book Layers.

The order book is a list of all current buy or sell orders for a given financial security. The rise of electronic stock exchanges introduced a debate about the relevance of the information it encapsulates of the activity of traders. Here, we approach this topic from a theoretical perspective, estimating the amount of mutual information between order book layers, i.e., different buy/sell layers, which are aggregated by buy/sell orders. We show that (i) layers are not independent (in the sense that the mutual information is statistically larger than zero), (ii) the mutual information between layers is small (compared to the joint entropy), and (iii) the mutual information between layers increases when comparing the uppermost layers to the deepest layers analyzed (i.e., further away from the market price). Our findings, and our method for estimating mutual information, are relevant to developing trading strategies that attempt to utilize the information content of the limit order book.

New mathematical measures for apprehending complexity of chiral molecules using information entropy.

In this paper, we present several new theoretical measures based on information entropy that can be used to analyze the information content of a chiral molecule. Starting from a differentiation between "chiral" and "achiral" portions in a chiral molecule, we define a new concept that allows us to quantify the complexity of chiral constitutional 2D-isomers of C10 to C20 alkanes. Various new chiral and achiral information measures founded on joint entropy, mutual information, and conditional entropy are presented providing an access to a set of regression equations. Then, introducing a case-based measure of entropy, we demonstrate that the distribution of the chiral complexity in these molecules is mostly skewed-right: 60% of the chiral isomers follow a 60/40 distribution rule, which indicates a concentration of chiral complexity in a small number of topological features. Furthermore, by replacing 2D topological distances by 3D distances, the application of these new information measures goes from conformational to racemization and deracemization studies. Interestingly, when the geometrical distances between atoms and the chiral center(s) are taken into account when determining the chiral information entropy, one can observe a significative Pearson correlation coefficient (R= 0.70) between the chiral entropy of 3D molecules and the continuous chirality measure. Finally, we show that our approach is applicable to almost any type of chiral organic chemical structures if in the entropy equation, atoms are represented by their electrotopological state (E-state) index instead of connectivity.

Intelligent fault diagnosis of wind turbine gearboxes based on refined generalized multi-scale state joint entropy and robust spectral feature selection

Intelligent fault diagnosis of wind turbine gearboxes based on refined generalized multi-scale state joint entropy and robust spectral feature selection

Scenario-based analysis for discovering relations among interestingness measures

Many interestingness measures have been proposed for mining meaningful association rules among two events in the form of A→B, but their characteristics and semantic similarity relations have not been comprehensively investigated. This paper presents a scenario-based approach for characterizing sixty-one commonly used measures and revealing their relationships in three steps. The first step generates a set of 969 three-probability scenarios, S={s|s=(p(A),p(B),p(A,B))∧p(A),p(B),p(A|B)∈[0,1]∧p(A,B)⩽min(p(A),p(B))}, in consideration of all possible situations in the range of 0.0 to 1.0 with a step of 0.05, excluding infinity and not-a-number cases. In the second step, 937,992 pairs of scenarios are enumerated, and for each scenario pair s1 and s2, the values of a measure (M) of s1 and s2, i.e., M(s1) and M(s2), are compared with the result of greater-than (M(s1)>M(s2)), smaller-than (M(s1)<M(s2)), or equal-to (M(s1)=M(s2)) for characterizing the measure. The final step is based on three types of relations: (1) behavior-based, (2) correlation-based, and (3) association-based similarity relations. The behavior of measures is depicted using nine common algebraic/statistical properties and four special condition properties, i.e., zero, min–max, infinity, and not-a-number of the measures. Similarities among the measures can be examined by grouping measures based on their properties. With three correlation functions, i.e., correlation coefficient, joint entropy, and mutual information, a correlation analysis was performed to discover relations among interestingness measures in the form of dendrograms and clusters with thresholding. Finally, the details of the relations among these interestingness measures are explored with association rule mining. Besides support, confidence, and lift, we propose five types of rules, i.e. same-directionrule (S-rule), opposite-direction rule (O-rule), equal-both rule (E-rule), equal-left rule (EL-rule),and equal-right rule (ER-rule) for a five-gradient comparison of any two measures to outline their similarities and dissimilarities in five directions.

Ensemble Entropy Metric for Hyperspectral Anomaly Detection

In hyperspectral anomaly detection, anomalies are rare targets that exhibit distinct spectral signatures from the background. Thus, anomalies are with low probabilities of occurrence in hyperspectral images. In this article, we develop a new technique for hyperspectral anomaly detection that adopts a new information theory perspective, to fully utilize the aforementioned concepts. Our goal is to transform system entropy into quantitative metrics of anomaly conspicuousness of pixels. To do so, two tasks are first completed: first, the construction of occurrence probability of pixels based on the density peak clustering algorithm, and second, the valid system definitions for pixels in specific anomaly detection problems with multiviews. Specifically, three types of systems are separately established by pixel pairs to conform to the definitions of three entropy definitions in information theory, i.e., Shannon entropy, joint entropy, and relative entropy. Then, three individual entropy-based metrics that assess the anomaly conspicuousness are defined. In addition, we design a standard deviation-based ensemble strategy for the integrated representation of the three individual metrics, which considers both logic &#x201C;OR&#x201D; and &#x201C;AND&#x201D; operations to simultaneously improve the detection rate and reduce the false alarm rate. Our experimental results obtained on two publicly available datasets with anomalies of different sizes and shapes demonstrate the superiority of our newly proposed anomaly detection method.

Introduction to logical entropy and its relationship to Shannon entropy

We live in the information age. Claude Shannon, as the father of the information age, gave us a theory of communications that quantified an “amount of information,” but, as he pointed out, “no concept of information itself was defined.” Logical entropy provides that definition. Logical entropy is the natural measure of the notion of information based on distinctions, differences, distinguishability, and diversity. It is the (normalized) quantitative measure of the distinctions of a partition on a set-just as the Boole–Laplace logical probability is the normalized quantitative measure of the elements of a subset of a set. And partitions and subsets are mathematically dual concepts – so the logic of partitions is dual in that sense to the usual Boolean logic of subsets, and hence the name “logical entropy.” The logical entropy of a partition has a simple interpretation as the probability that a distinction or dit (elements in different blocks) is obtained in two independent draws from the underlying set. The Shannon entropy is shown to also be based on this notion of information-as-distinctions; it is the average minimum number of binary partitions (bits) that need to be joined to make all the same distinctions of the given partition. Hence all the concepts of simple, joint, conditional, and mutual logical entropy can be transformed into the corresponding concepts of Shannon entropy by a uniform non-linear dit-bit transform. And finally logical entropy linearizes naturally to the corresponding quantum concept. The quantum logical entropy of an observable applied to a state is the probability that two different eigenvalues are obtained in two independent projective measurements of that observable on that state.

Feature Extraction and Classification of MRI Using Hybrid RBF Kernel and SVM

An enhanced classification system for classification of MR images using association of kernels with support vector machine is developed and presented in this paper along with the design and development of content-based image retrieval (CBIR) system. Content of image retrieval is the process of finding relevant image from large collection of image database using visual queries. Medical images have led to growth in large image collection. Oriented Rician Noise Reduction Anisotropic Diffusion filter is used for image denoising. A modified hybrid Otsu algorithm termed is used for image segmentation. The texture features are extracted using GLCM method. Genetic algorithm with Joint entropy is adopted for feature selection. The classification is done by support vector machine along with various kernels and the performance is validated. A classification accuracy of 98.83% is obtained using SVM with GRBF kernel. Various features have been extracted and these features are used to classify MR images into five different categories. Performance of the MC-SVM classifier is compared with different kernel functions. From the analysis and performance measures like classification accuracy, it is inferred that the brain and spinal cord MRI classification is best done using MC- SVM with Gaussian RBF kernel function than linear and polynomial kernel functions. The proposed system can provide best classification performance with high accuracy and low error rate.

Diagnostic value of baseline 18FDG PET/CT skeletal textural features in follicular lymphoma

At present, 18F-fluorodesoxyglucose (18FDG) positron emission tomography (PET)/computed tomography (CT) cannot be used to omit a bone marrow biopsy (BMB) among initial staging procedures in follicular lymphoma (FL). The additional diagnostic value of skeletal textural features on baseline 18FDG-PET/CT in diffuse large B-cell lymphoma (DLBCL) patients has given promising results. The aim of this study is to evaluate the value of 18FDG-PET/CT radiomics for the diagnosis of bone marrow involvement (BMI) in FL patients. This retrospective bicentric study enrolled newly diagnosed FL patients addressed for baseline 18FDG PET/CT. For visual assessment, examinations were considered positive in cases of obvious bone focal uptakes. For textural analysis, the skeleton volumes of interest (VOIs) were automatically extracted from segmented CT images and analysed using LifeX software. BMB and visual assessment were taken as the gold standard: BMB −/PET − patients were considered as bone-NEGATIVE patients, whereas BMB +/PET −, BMB −/PET + and BMB +/PET + patients were considered bone-POSITIVE patients. A LASSO regression algorithm was used to select features of interest and to build a prediction model. Sixty-six consecutive patients were included: 36 bone-NEGATIVE (54.5%) and 30 bone-POSITIVE (45.5%). The LASSO regression found variance_GLCM, correlation_GLCM, joint entropy_GLCM and busyness_NGLDM to have nonzero regression coefficients. Based on ROC analysis, a cut-off equal to − 0.190 was found to be optimal for the diagnosis of BMI using PET pred.score. The corresponding sensitivity, specificity, PPV and NPV values were equal to 70.0%, 83.3%, 77.8% and 76.9%, respectively. When comparing the ROC AUCs with using BMB alone, visual PET assessment or PET pred.score, a significant difference was found between BMB versus visual PET assessments (p = 0.010) but not between BMB and PET pred.score assessments (p = 0.097). Skeleton texture analysis is worth exploring to improve the performance of 18FDG-PET/CT for the diagnosis of BMI at baseline in FL patients.

Empirical assessment of the relation between ecological connectivity and land complexity based on information-theoretic metrics

Habitat fragmentation and connectivity loss pose significant threats to biodiversity at both local and landscape levels. Strategies to increase ecological connectivity and preserve strong connectivity are important for dealing with the potential threat of habitat degradation. Various metrics have been used to measure (i.e., quantify) landscape composition and configuration in landscape ecology. However, their relationship with ecological connectivity must be understood to interpret landscape patterns comprehensively. In the present study, correlations between ecological connectivity and land complexity are examined based on information-theory metrics. Two primary questions are explored: (1) to what extent are landscape mosaic measures of entropy correlated with ecological connectivity, with landscape gradient-based measures, and with each other? (2) are landscape gradient-based entropy measures correlated with ecological connectivity more than discrete entropy measures? Results show that all information theoretic metrics are statistically significant (p < 0.05) for modelling ecological connectivity. Among categorically-based indices, the relationship between ECI and joint entropy was the most significant, while a generalized additive model indicated that Boltzmann entropy could predict the ecological connectivity index, explaining ∼60% of the variance. Therefore, configurational entropy can be used for improving ecological connectivity models.

Global Joint Entropy Algorithm-Based Evaluation of Dexmedetomidine Combined Local Anesthetics in Brachial Plexus Block from Ultrasound Images

This paper aimed to explore dexmedetomidine combined local anesthetics in brachial plexus block through ultrasound imaging (UI) under global joint entropy algorithm. Patients who underwent upper limb surgery and brachial plexus block were selected as research objects. Patients in group A were given 0.375% ropivacaine and normal saline, and patients in group B were given 0.375% ropivacaine and 1.0 μg/kg dexmedetomidine. The results of UI were analyzed by global joint entropy-based K-means clustering (GKC) algorithm, and the use effects of the two methods were compared in combination with other postanesthesia manifestations. The results were as follows. The segmentation accuracy (96.21% and 83.52%) of GKC was higher than 82.21% and 70.52% of the local joint entropy-based K-means clustering (LKC) ( P &lt; 0.05 ). The duration of sensory and motor block (352.78 ± 45.89 min and 324.38 ± 41.29 min) in group B was significantly longer than 292.28 ± 35.69 min and 256.58 ± 42.76 min in group A ( P &lt; 0.05 ). Compared with 84.91 ± 8.77 beats/min and 89.58 ± 7.62 beats/min in group A, mean arterial pressure (70.24 ± 9.77 beats/min and 69.89 ± 8.97 beats/min) in group B was lower at T1 and T2 ( P &lt; 0.05 ). The duration of postoperative pain (582.70 ± 51.89 min) in group B was longer than 372.89 ± 49.89 min in group A ( P &lt; 0.05 ). The postoperative pain score (2.98 ± 1.08) in group B was significantly lower than 4.48 ± 2.19 in group A ( P &lt; 0.05 ). Therefore, dexmedetomidine combined local anesthetics could prolong the duration of sensory and motor nerve block. Besides, dexmedetomidine had sedative and analgesic effects, so as to prolong postoperative pain time and reduce pain degree of patients.

Accelerating Causal Inference and Feature Selection Methods through G-Test Computation Reuse.

This article presents a novel and remarkably efficient method of computing the statistical G-test made possible by exploiting a connection with the fundamental elements of information theory: by writing the G statistic as a sum of joint entropy terms, its computation is decomposed into easily reusable partial results with no change in the resulting value. This method greatly improves the efficiency of applications that perform a series of G-tests on permutations of the same features, such as feature selection and causal inference applications because this decomposition allows for an intensive reuse of these partial results. The efficiency of this method is demonstrated by implementing it as part of an experiment involving IPC–MB, an efficient Markov blanket discovery algorithm, applicable both as a feature selection algorithm and as a causal inference method. The results show outstanding efficiency gains for IPC–MB when the G-test is computed with the proposed method, compared to the unoptimized G-test, but also when compared to IPC–MB++, a variant of IPC–MB which is enhanced with an AD–tree, both static and dynamic. Even if this proposed method of computing the G-test is presented here in the context of IPC–MB, it is in fact bound neither to IPC–MB in particular, nor to feature selection or causal inference applications in general, because this method targets the information-theoretic concept that underlies the G-test, namely conditional mutual information. This aspect grants it wide applicability in data sciences.

Prediction of Malignant Cell Infiltration Patterns with Texture Features of Biopsy-Correlated Positron Emission Tomography of Osteolytic Lesions in Multiple Myeloma

Prediction of Malignant Cell Infiltration Patterns with Texture Features of Biopsy-Correlated Positron Emission Tomography of Osteolytic Lesions in Multiple Myeloma

Robust imaging habitat computation using voxel-wise radiomics features

Tumor heterogeneity has been postulated as a hallmark of treatment resistance and a cure constraint in cancer patients. Conventional quantitative medical imaging (radiomics) can be extended to computing voxel-wise features and aggregating tumor subregions with similar radiological phenotypes (imaging habitats) to elucidate the distribution of tumor heterogeneity within and among tumors. Despite the promising applications of imaging habitats, they may be affected by variability of radiomics features, preventing comparison and generalization of imaging habitats techniques. We performed a comprehensive repeatability and reproducibility analysis of voxel-wise radiomics features in more than 500 lung cancer patients with computed tomography (CT) images and demonstrated the effect of voxel-wise radiomics variability on imaging habitats computation in 30 lung cancer patients with test–retest images. Repeatable voxel-wise features characterized texture heterogeneity and were reproducible regardless of the applied feature extraction parameters. Imaging habitats computed using robust radiomics features were more stable than those computed using all features in test–retest CTs from the same patient. Nine voxel-wise radiomics features (joint energy, joint entropy, sum entropy, maximum probability, difference entropy, Imc1, Imc2, Idn and Idmn) were repeatable and reproducible. This supports their application for computing imaging habitats in lung tumors towards the discovery of previously unseen tumor heterogeneity and the development of novel non-invasive imaging biomarkers for precision medicine.