Entropy Distance Research Articles

Exploring structural variances in monatomic metallic glasses using machine learning and molecular dynamics simulation.

BCC and FCC metals have different glass-forming abilities (GFA) and exhibit different characteristics during the glass transition. However, the structural origin of their different GFAs is still not clear. Here, we explored the structures of eight monatomic metallic glasses by combining molecular dynamics (MD) simulations and machine learning (ML). Our findings reveal that, despite their common long-range disordered atomic structure, metallic glasses can be further classified into two distinct categories indicating an underlying structural order within the disorder. Using machine learning, we found that BCC liquids can sample more diverse glass states than FCC liquids. Furthermore, glasses formed from BCC metals (GFFBs) exhibit a higher degree of disorder than glasses formed from FCC metals (GFFFs). These findings highlight the inherent differences between GFFFs and GFFBs, which help explain the different glass-forming abilities of FCC and BCC metals. Additionally, our results demonstrate the promising potential of computer vision and ML methods in exploring material structures. Classical molecular dynamics simulations were employed to generate configurations of GFFBs and GFFFs, and the simulations were performed using the LAMMPS code. Inter-atomic interactions were described using a classical embedded atom model (EAM) potential. The initial configuration of the model consists of 32,000 atoms in a three-dimensional (3D) cubic box with periodic boundary conditions applied in all three directions. For machine learning, we utilized an unsupervised machine learning method along with MobileNetV2 for classifying glass structures. Image entropy and image distances were used to measure the structural differences of the metallic glasses.

Feasibility of Using 18F-FDG PET/CT Radiomics and Machine Learning to Detect Drug-Induced Interstitial Lung Disease

Background: Bleomycin is an oncolytic and antibiotic agent used to treat various human cancers because of its antitumor activity. Unfortunately, up to 46% of the patients treated with bleomycin develop drug-induced interstitial lung disease (DIILD) and potentially life-threatening interstitial pulmonary fibrosis. Tools and biomarkers for predicting and detecting DIILD are limited. Therefore, we aimed to evaluate the feasibility of 18F-FDG PET/CT, PET radiomics, and machine learning in distinguishing DIILD in an explorative pilot study. Methods: Eighteen Hodgkin’s lymphoma (HL) patients, of whom 10 developed DIILD after treatment with bleomycin, were retrospectively included. Five diffuse large B-cell lymphoma (DLBCL) patients were included as a control group since they were not treated with bleomycin. All patients underwent 18F-FDG PET/CT scans before (baseline) and during treatment (interim). Structural changes were assessed by changes in Hounsfield Units (HUs). The 18F-FDG PET scans were used to assess metabolic changes by examining the feasibility of 504 radiomics features, including the mean activity of the lungs (SUVmean). A Random Forest (RF) classifier evaluated the identification and prediction of DIILD based on PET radiomics features. Results: HL patients who developed DIILD showed a significant increase in standard SUV metrics (SUVmean; p = 0.012, median increase 37.4%), and in some regional PET radiomics features (texture strength; p = 0.009, median increase 101.6% and zone distance entropy; p = 0.019, median increase 18.5%), while this was not found in HL patients who did not develop DIILD and DLBCL patients. The RF classifier correctly identified DIILD in 72.2% of the patients and predicted the development of DIILD correctly in 50% of the patients. There were no significant differences in HUs over time within all three patient groups. Conclusions: Our explorative longitudinal pilot study suggests that certain regional 18F-FDG PET radiomics features can effectively identify DIILD in HL patients treated with bleomycin, as significant longitudinal increases were observed in SUVmean, texture strength, and zone distance entropy after the development of DIILD. The metabolic activity of these features did not significantly increase over time in DLBCL patients and HL patients who did not develop DIILD. This indicates that 18F-FDG PET radiomics, with and without machine learning, might serve as potential biomarkers for detecting DIILD.

An appropriate artificial intelligence technique for plastic materials recycling using bipolar dual hesitant fuzzy set

Plastic recycling has become more important than ever as the globe struggles with growing environmental issues. This research explores the significant environmental impact of recycling plastic and its growing relevance. The pervasive material known as plastic presents a complex risk to both human health and ecosystems in contemporary life. It exacerbates problems including marine pollution, habitat damage, and wildlife entanglement because of its persistence in landfills and seas, which leads to serious ecological deterioration. In addition, producing plastic uses a lot of energy and produces a lot of greenhouse gas emissions, which exacerbate climate change. Through the use of multi-criteria decision making (MCDM), this study emphasizes how vital it is to support recycling activities in order to protect the environment and promote a sustainable future. The elimination and choice ex-pressing reality (ELECTRE) approach is used to rank the alternatives in this proposed research study that employs bipolar dual hesitant fuzzy sets (BDHFs). The most efficient and versatile outranking method for making decisions is the BDHF-ELECTRE approach. The weights of environment, economic, social, technical, and finally safety is computed using the entropy distance metric. The economic factor received the highest score of 0.2945 among the other factors since economic considerations are crucial in choosing the most efficient plastic recycling method, as they ensure sustainability, cost-effectiveness, resource allocation, and overall feasibility in managing plastic waste. The decision-makers determined that the mechanical recycling approach ought to be prioritized over all others for the efficient recycling of plastic waste. The robustness of the system is examined in the sensitive and comparative analyses. The proposed MCDM technique thus presents a viable solution, mitigating the adverse effects of plastic waste by conserving resources, reducing energy consumption, and curbing pollution.

A belief interval euclidean distance entropy of the mass function and its application in multi-sensor data fusion

A belief interval euclidean distance entropy of the mass function and its application in multi-sensor data fusion

Continuity of the relative entropy of resource

We present a criterion to establish the local continuity of the relative entropy of resource, i.e. the relative entropy distance to the set of free states, in any quantum resource theory. Several basic corollaries of this criterion are presented. Applications to the relative entropy of entanglement in multipartite quantum systems are considered. It is shown, in particular, that local continuity of any relative entropy of multipartite entanglement follows from local continuity of the quantum mutual information.

An entropy-based study on the mutational landscape of SARS-CoV-2 in USA: Comparing different variants and revealing co-mutational behavior of proteins

COVID-19 emergency has pushed the international scientific community to use every resource to combat the spread of the virus, to understand its biology and predict its possible evolution in terms of new variants. Since the first SARS-CoV-2 virus nucleotide and amino acid sequences were made available, information theory was used to study how viral information content was changing over time and then trace the evolution of its mutational landscape. In this work we analyzed SARS-CoV-2 sequences collected mainly in the USA in a period from March 2020 until December 2022 and computed mutation profiles of viral proteins over time through an entropy-based approach using Shannon Entropy and Hellinger distance. This representation allows an at-a-glance view of the mutational landscape of viral proteins over time and can provide new insights on the evolution of the virus from different points of view. Non-structural proteins typically showed flat mutation profiles, characterized by a very low Average mutation Entropy, while accessory and structural proteins showed mostly non uniform and high mutation profiles, often coupled with the predominance of variants. Interestingly NSP2 protein, whose function is currently still debated, falls in the same branch of NSP14 and NSP10 in the phylogenetic tree of mutations constructed through correlations of mutation profiles, suggesting a co-evolution of those proteins and a possible functional link with each other. To the best of our knowledge this is the first study based on a massive amount of data (n = 107,939,973) that analyzes from an entropy point of view the mutational landscape of SARS-CoV-2 over time and depicts a mutational temporal profile of each protein of the virus.

Be My Guesses: The interplay between side-channel leakage metrics

In a theoretical context of side-channel attacks, optimal bounds between success rate, guessing entropy and statistical distance are derived with a simple majorization (Schur-concavity) argument. They are further theoretically refined for different versions of the classical Hamming weight leakage model, in particular assuming a priori equiprobable secret keys and additive white Gaussian measurement noise. Closed-form expressions and numerical computation are given. A study of the impact of the choice of the substitution box with respect to side-channel resistance reveals that its nonlinearity tends to homogenize the expressivity of success rate, guessing entropy and statistical distance. The intriguing approximate relation between guessing entropy and success rate GE=1/SR is observed in the case of 8-bit bytes and low noise. The exact relation between guessing entropy, statistical distance and alphabet size GE=M+12−M2SD for deterministic leakages and equiprobable keys is proved.

Scribble Hides Class: Promoting Scribble-Based Weakly-Supervised Semantic Segmentation with Its Class Label

Scribble-based weakly-supervised semantic segmentation using sparse scribble supervision is gaining traction as it reduces annotation costs when compared to fully annotated alternatives. Existing methods primarily generate pseudo-labels by diffusing labeled pixels to unlabeled ones with local cues for supervision. However, this diffusion process fails to exploit global semantics and class-specific cues, which are important for semantic segmentation. In this study, we propose a class-driven scribble promotion network, which utilizes both scribble annotations and pseudo-labels informed by image-level classes and global semantics for supervision. Directly adopting pseudo-labels might misguide the segmentation model, thus we design a localization rectification module to correct foreground representations in the feature space. To further combine the advantages of both supervisions, we also introduce a distance entropy loss for uncertainty reduction, which adapts per-pixel confidence weights according to the reliable region determined by the scribble and pseudo-label's boundary. Experiments on the ScribbleSup dataset with different qualities of scribble annotations outperform all the previous methods, demonstrating the superiority and robustness of our method. The code is available at https://github.com/Zxl19990529/Class-driven-Scribble-Promotion-Network.

Enhancing COPD classification using combined quantitative computed tomography and texture-based radiomics: a CanCOLD cohort study.

Recent advances in texture-based computed tomography (CT) radiomics have demonstrated its potential for classifying COPD. Participants from the Canadian Cohort Obstructive Lung Disease (CanCOLD) study were evaluated. A total of 108 features were included: eight quantitative CT (qCT), 95 texture-based radiomic and five demographic features. Machine-learning models included demographics along with texture-based radiomics and/or qCT. Combinations of five feature selection and five classification methods were evaluated; a training dataset was used for feature selection and to train the models, and a testing dataset was used for model evaluation. Models for classifying COPD status and severity were evaluated using the area under the receiver operating characteristic curve (AUC) with DeLong's test for comparison. SHapely Additive exPlanations (SHAP) analysis was used to investigate the features selected. A total of 1204 participants were evaluated (n=602 no COPD; n=602 COPD). There were no differences between the groups for sex (p=0.77) or body mass index (p=0.21). For classifying COPD status, the combination of demographics, texture-based radiomics and qCT performed better (AUC=0.87) than the combination of demographics and texture-based radiomics (AUC=0.81, p<0.05) or qCT alone (AUC=0.84, p<0.05). Similarly, for classifying COPD severity, the combination of demographics, texture-based radiomics and qCT performed better (AUC=0.81) than demographics and texture-based radiomics (AUC=0.72, p<0.05) or qCT alone (AUC=0.79, p<0.05). Texture-based radiomics and qCT features were among the top five features selected (15th percentile of the CT density histogram, CT total airway count, pack-years, CT grey-level distance zone matrix zone distance entropy, CT low-attenuation clusters) for classifying COPD status. Texture-based radiomics and conventional qCT features in combination improve machine‑learning models for classification of COPD status and severity.

Petz recovery from subsystems in conformal field theory

We probe the multipartite entanglement structure of the vacuum state of a CFT in 1+1 dimensions, using recovery operations that attempt to reconstruct the density matrix in some region from its reduced density matrices on smaller subregions. We use an explicit recovery channel known as the twirled Petz map, and study distance measures such as the fidelity, relative entropy, and trace distance between the original state and the recovered state. One setup we study in detail involves three contiguous intervals A, B and C on a spatial slice, where we can view these quantities as measuring correlations between A and C that are not mediated by the region B that lies between them. We show that each of the distance measures is both UV finite and independent of the operator content of the CFT, and hence depends only on the central charge and the cross-ratio of the intervals. We evaluate these universal quantities numerically using lattice simulations in critical spin chain models, and derive their analytic forms in the limit where A and C are close using the OPE expansion. In the case where A and C are far apart, we find a surprising non-commutativity of the replica trick with the OPE limit. For all values of the cross-ratio, the fidelity is strictly better than a general information-theoretic lower bound in terms of the conditional mutual information. We also compare the mutual information between various subsystems in the original and recovered states, which leads to a more qualitative understanding of the differences between them. Further, we introduce generalizations of the recovery operation to more than three adjacent intervals, for which the fidelity is again universal with respect to the operator content.

Fault Diagnosis Method for Rolling Bearings Based on Grey Relation Degree.

Aiming at the difficult problem of extracting fault characteristics and the low accuracy of fault diagnosis throughout the full life cycle of rolling bearings, a fault diagnosis method for rolling bearings based on grey relation degree is proposed in this paper. Firstly, the subtraction-average-based optimizer is used to optimize the parameters of the variational mode decomposition algorithm. Secondly, the vibration signals of bearings are decomposed by using the optimized results, and the feature vector of the intrinsic mode function component corresponding to the minimum envelope entropy is extracted. Finally, the grey proximity and similarity relation degree based on standard distance entropy are weighted to calculate the grey comprehensive relation degree between the feature vector of vibration signals and each standard state. By comparing the results, the diagnosis of different fault states and degrees of rolling bearings is realized. The XJTU-SY dataset was used for experimentation, and the results show that the proposed method achieves a diagnostic accuracy of 95.24% and has better diagnosis performance compared to various algorithms. It provides a reference for the fault diagnosis of rolling bearings throughout the full life cycle.

Weighting Low-Intensity MS/MS Ions and m/z Frequency for Spectral Library Annotation.

Calculating spectral similarity is a fundamental step in MS/MS data analysis in untargeted metabolomics experiments, as it facilitates the identification of related spectra and the annotation of compounds. To improve matching accuracy when querying an experimental mass spectrum against a spectral library, previous approaches have proposed increasing peak intensities for high m/z ranges. These high m/z values tend to be smaller in magnitude, yet they offer more crucial information for identifying the chemical structure. Here, we evaluate the impact of using these weights for identifying structurally related compounds and mass spectral library searches. Additionally, we propose a weighting approach that (i) takes into account the frequency of the m/z values within a spectral library in order to assign higher importance to the most common peaks and (ii) increases the intensity of lower peaks, similar to previous approaches. To demonstrate our approach, we applied weighting preprocessing to modified cosine, entropy, and fidelity distance metrics and benchmarked it against previously reported weights. Our results demonstrate how weighting-based preprocessing can assist in annotating the structure of unknown spectra as well as identifying structurally similar compounds. Finally, we examined scenarios in which the utilization of weights resulted in diminished performance, pinpointing spectral features where the application of weights might be detrimental.

An Information Theoretic Condition for Perfect Reconstruction.

A new information theoretic condition is presented for reconstructing a discrete random variable X based on the knowledge of a set of discrete functions of X. The reconstruction condition is derived from Shannon's 1953 lattice theory with two entropic metrics of Shannon and Rajski. Because such a theoretical material is relatively unknown and appears quite dispersed in different references, we first provide a synthetic description (with complete proofs) of its concepts, such as total, common, and complementary information. The definitions and properties of the two entropic metrics are also fully detailed and shown to be compatible with the lattice structure. A new geometric interpretation of such a lattice structure is then investigated, which leads to a necessary (and sometimes sufficient) condition for reconstructing the discrete random variable X given a set {X1,…,Xn} of elements in the lattice generated by X. Intuitively, the components X1,…,Xn of the original source of information X should not be globally "too far away" from X in the entropic distance in order that X is reconstructable. In other words, these components should not overall have too low of a dependence on X; otherwise, reconstruction is impossible. These geometric considerations constitute a starting point for a possible novel "perfect reconstruction theory", which needs to be further investigated and improved along these lines. Finally, this condition is illustrated in five specific examples of perfect reconstruction problems: the reconstruction of a symmetric random variable from the knowledge of its sign and absolute value, the reconstruction of a word from a set of linear combinations, the reconstruction of an integer from its prime signature (fundamental theorem of arithmetic) and from its remainders modulo a set of coprime integers (Chinese remainder theorem), and the reconstruction of the sorting permutation of a list from a minimal set of pairwise comparisons.

Multidimensional Measure Matching for Crowd Counting.

This article addresses the challenge of scale variations in crowd-counting problems from a multidimensional measure-theoretic perspective. We start by formulating crowd counting as a measure-matching problem, based on the assumption that discrete measures can express the scattered ground truth and the predicted density map. In this context, we introduce the Sinkhorn counting loss and extend it to the semi-balanced form, which alleviates the problems including entropic bias, distance destruction, and amount constraints. We then model the measure matching under the multidimensional space, in order to learn the counting from both location and scale. To achieve this, we extend the traditional 2-D coordinate support to 3-D, incorporating an additional axis to represent scale information, where a pyramid-based structure will be leveraged to learn the scale value for the predicted density. Extensive experiments on four challenging crowd-counting datasets, namely, ShanghaiTech A, UCF-QNRF, JHU ++ , and NWPU have validated the proposed method. Code is released at https://github.com/LoraLinH/Multidimensional-Measure-Matching-for-Crowd-Counting.

Research on Medical Image Enhancement Method Based on Conditional Entropy Generative Adversarial Networks

Abstract This study proposes an image enhancement method combining conditional entropy and generative adversarial network, aiming to improve the image quality while avoiding overfitting through the negative training of dependent generative adversarial network and introducing dependent entropy distance loss. Through NIQMC, NIQE and BTMQI evaluation indexes, this paper evaluates the effects of different parameter combinations and image chunk sizes on the enhancement results. It utilizes information entropy as an evaluation index to measure the impact of conditional entropy distance loss. The effectiveness of adversarial learning and conditional entropy in image enhancement is verified by comparing the experimental results. The experiments show that the system can achieve the best image quality of SSIM=0.9852, PSNR=27.58, and SNROI=21.34 with the parameters S=50 and R=4.0%, indicating that the method can effectively retain the detailed information and realism of the Image while enhancing the clarity of the Image, demonstrating a significant performance advantage.

DEA cross-efficiency models with prospect theory and distance entropy: An empirical study on high-tech industries

Cross-efficiency evaluation with the data envelopment analysis (DEA) model is an effective way to assess performance and provide a complete ranking of decision-making units. However, it is generally assumed that decision makers are perfectly rational in the cross-efficiency model, which fails to consider the subjective preferences of decision-makers. Moreover, the arithmetic average method is usually adopted to aggregate efficiency scores in traditional cross-efficiency methods, which underestimates the importance of self-evaluation. To address these issues, we extend cross-efficiency with the DEA model by incorporating prospect theory and the distance entropy function. First, we calculate the prospect values of decision-making units to describe the non-rational subjective preferences under the risk of decision-makers. Second, based on prospect cross-efficiency, a new distance entropy function is developed to aggregate the ultimate prospect cross-efficiency values. More specifically, some traditional cross-efficiency evaluation models can be considered special cases of prospect cross-efficiency models with appropriate adjustments to the parameters. Finally, an empirical example is used to evaluate the prospect cross-efficiency results with the high-tech industries of 29 provinces in China to illustrate the applicability and effectiveness of the proposed model in ranking observations.

Conditioning in Tropical Probability Theory.

We define a natural operation of conditioning of tropical diagrams of probability spaces and show that it is Lipschitz continuous with respect to the asymptotic entropy distance.

Distance correlation entropy and ordinal distance complexity measure: efficient tools for complex systems

Distance correlation entropy and ordinal distance complexity measure: efficient tools for complex systems

Reconstruction of Viral Variants via Monte Carlo Clustering.

Identifying viral variants through clustering is essential for understanding the composition and structure of viral populations within and between hosts, which play a crucial role in disease progression and epidemic spread. This article proposes and validates novel Monte Carlo (MC) methods for clustering aligned viral sequences by minimizing either entropy or Hamming distance from consensuses. We validate these methods on four benchmarks: two SARS-CoV-2 interhost data sets and two HIV intrahost data sets. A parallelized version of our tool is scalable to very large data sets. We show that both entropy and Hamming distance-based MC clusterings discern the meaningful information from sequencing data. The proposed clustering methods consistently converge to similar clusterings across different runs. Finally, we show that MC clustering improves reconstruction of intrahost viral population from sequencing data.

Continuity bounds on observational entropy and measured relative entropies

We derive a measurement-independent asymptotic continuity bound on the observational entropy for general positive operator valued measures measurements, making essential use of its property of bounded concavity. The same insight is used to obtain continuity bounds for other entropic quantities, including the measured relative entropy distance to a convex set of states under a general set of measurements. As a special case, we define and study conditional observational entropy, which is an observational entropy in one (measured) subsystem conditioned on the quantum state in another (unmeasured) subsystem. We also study continuity of relative entropy with respect to a jointly applied channel, finding that observational entropy is uniformly continuous as a function of the measurement. But we show by means of an example that this continuity under measurements cannot have the form of a concrete asymptotic bound.