Entropy Measures Research Articles

FunSpace: A functional and spatial analytic approach to cell imaging data using entropy measures.

Spatial heterogeneity in the tumor microenvironment (TME) plays a critical role in gaining insights into tumor development and progression. Conventional metrics typically capture the spatial differential between TME cellular patterns by either exploring the cell distributions in a pairwise fashion or aggregating the heterogeneity across multiple cell distributions without considering the spatial contribution. As such, none of the existing approaches has fully accounted for the simultaneous heterogeneity caused by both cellular diversity and spatial configurations of multiple cell categories. In this article, we propose an approach to leverage spatial entropy measures at multiple distance ranges to account for the spatial heterogeneity across different cellular organizations. Functional principal component analysis (FPCA) is applied to estimate FPC scores which are then served as predictors in a Cox regression model to investigate the impact of spatial heterogeneity in the TME on survival outcome, potentially adjusting for other confounders. Using a non-small cell lung cancer dataset (n = 153) as a case study, we found that the spatial heterogeneity in the TME cellular composition of CD14+ cells, CD19+ B cells, CD4+ and CD8+ T cells, and CK+ tumor cells, had a significant non-zero effect on the overall survival (p = 0.027). Furthermore, using a publicly available multiplexed ion beam imaging (MIBI) triple-negative breast cancer dataset (n = 33), our proposed method identified a significant impact of cellular interactions between tumor and immune cells on the overall survival (p = 0.046). In simulation studies under different spatial configurations, the proposed method demonstrated a high predictive power by accounting for both clinical effect and the impact of spatial heterogeneity.

Unsupervised Domain Adaptation with Asymmetrical Margin Disparity loss and Outlier Sample Extraction

Unsupervised domain adaptation (UDA) trains models using labeled data from a specific source domain and then transferring the knowledge to certain target domains that have few or no labels. Many prior measurement-based works achieve lots of progress, but their feature distinguishing abilities to classify target samples with similar features are not enough; they do not adequately consider the confusing samples in the target domain that are similar to the source domain; and they don't consider negative transfer of the outlier sample in source domain. We address these issues in our work and propose an UDA method with asymmetrical margin disparity loss and outlier sample extraction, called AMD-Net with OSE. We propose an Asymmetrical Margin Disparity Discrepancy (AMD) method and a training strategy based on sample selection mechanism to make the network have better feature extraction ability and the network gets rid of local optimal. Firstly, in the AMD method, we design a multi-label entropy metric to evaluate the marginal disparity loss of the confusing samples in the target domain. This asymmetric marginal disparity loss designment uses the different entropy measurement algorithms of the two domains to excavate the differences of the two domains as much as possible, so as to find the common features of the two domains. Secondly, A sample selection mechanism is designed to evaluate which part of the sample in target domain is confusable. We define the certainty of the sample in the target domain, adopt a progressive learning scheme, and adopt one-hot marginal disparity loss for most of the samples in the target domain with low uncertainty and easy to distinguish. The multi-label marginal calculation method is used only for the uncertainty samples in the target domain whose certainty is less than the threshold value, so that the network can get rid of the local optimal as much as possible. At last, we further propose an outlier sample extraction algorithm (OSE) based on weighted cosine similarity distance for source domain to reduce the negative migration effect caused by outlier samples in the source domain. Extensive experiments on four datasets Office-31, Office-Home, VisDA-2017 and DomainNet demonstrate that our method works well in various UDA settings and outperforms the state-of-the-art methods.

A New Separation Index and Classification Techniques Based on Shannon Entropy

The purpose is to use Shannon entropy measures to develop classification techniques and an index which estimates the separation of the groups in a finite mixture model. These measures can be applied to machine learning techniques such as discriminant analysis, cluster analysis, exploratory data analysis, etc. If we know the number of groups and we have training samples from each group (supervised learning) the index is used to measure the separation of the groups. Here some entropy measures are used to classify new individuals in one of these groups. If we are not sure about the number of groups (unsupervised learning), the index can be used to determine the optimal number of groups from an entropy (information/uncertainty) criterion. It can also be used to determine the best variables in order to separate the groups. In all the cases we assume that we have absolutely continuous random variables and we use the Shannon entropy based on the probability density function. Theoretical, parametric and non-parametric techniques are proposed to get approximations of these entropy measures in practice. An application to gene selection in a colon cancer discrimination study with a lot of variables is provided as well.

Agricultural Land Cover Mapping through Two Deep Learning Models in the Framework of EU’s CAP Activities Using Sentinel-2 Multitemporal Imagery

The images of the Sentinel-2 constellation can help the verification process of farmers’ declarations, providing, among other things, accurate spatial explicit maps of the agricultural land cover. The aim of the study is to design, develop, and evaluate two deep learning (DL) architectures tailored for agricultural land cover and crop type mapping. The focus is on a detailed class scheme encompassing fifteen distinct classes, utilizing Sentinel-2 imagery acquired on a monthly basis throughout the year. The study’s geographical scope covers a diverse rural area in North Greece, situated within southeast Europe. These architectures are a Temporal Convolutional Neural Network (CNN) and a combination of a Recurrent and a 2D Convolutional Neural Network (R-CNN), and their accuracy is compared to the well-established Random Forest (RF) machine learning algorithm. The comparative approach is not restricted to simply presenting the results given by classification metrics, but it also assesses the uncertainty of the classification results using an entropy measure and the spatial distribution of the classification errors. Furthermore, the issue of sampling strategy for the extraction of the training set is highlighted, targeting the efficient handling of both the imbalance of the dataset and the spectral variability of instances among classes. The two developed deep learning architectures performed equally well, presenting an overall accuracy of 90.13% (Temporal CNN) and 90.18% (R-CNN), higher than the 86.31% overall accuracy of the RF approach. Finally, the Temporal CNN method presented a lower entropy value (6.63%), compared both to R-CNN (7.76%) and RF (28.94%) methods, indicating that both DL approaches should be considered for developing operational EO processing workflows.

Non-linear oscillators with Kuramoto-like local coupling: Complexity analysis and spatiotemporal pattern generation

Can a simple oscillator system, as in cellular automata, sustain complex nature upon discretization in time and space? The answer is by no means trivial as even the most simple, two-state, nearest neighbours cellular automata can lead to Universal Turing Machine (UTM) computing power. This study analyses a recently proposed model consisting of a ring of identical excitable Adler-type oscillators with local Kuramoto-like coupling in terms of its complexity. Regions with non-trivial, complex behaviour have been identified, where spatiotemporal maps closely resemble those found in elementary cellular automata, not only from the visual perspective but also from entropic measures characterization. Also, the possibility of enhanced computation at the edge of chaos is explored by monitoring the effective complexity measure, entropy density and informational distance, following previous approaches. Distance matrix, and the corresponding dendrograms, show that in the complex regions, a non-trivial set of hierarchies emerges where local communities can appear and sustain themself in time. The fact that coupled oscillators can be realized through dedicated electronic circuitry or the result of natural processes can make finding such complex dynamics, with potential computational power, an important one in a broad scope of applications and implications.

Mild cognitive impairment detection with optimally selected EEG channels based on variational mode decomposition and supervised machine learning

Detecting mild cognitive impairment (MCI), which is typically the earliest stage of dementia, is essential for managing dementia. Recently, researchers have explored the use of electroencephalography (EEG) to detect MCI. EEG involves recording brain activity by placing a number of electrodes (channels) directly on the scalp. Here we present effective variational mode decomposition (VMD)-based methods for detection of MCI with optimally selected EEG channels. To achieve this, we first decompose the EEG signals from each channel into sub-signals called variational mode functions (VMFs) with different frequency bands. From each VMF, an entropy measure is used to extract a feature. Several supervised machine learning techniques are investigated to classify the resulting features and obtain full-channel classification accuracy. We assess the classification performance of the proposed methods on 19-channel EEGs recorded from 32 healthy subjects and 29 MCI patients in the resting state. In a second step, we explore the feasibility of reducing the number of EEG channels while maintaining classification accuracy by using several EEG channel selection methods. Specifically, we study the effectiveness of non-dominated sorting genetic algorithm (NSGA)-based methods (multi-objective optimization methods), in addition to greedy algorithms such as back-elimination and forward-addition. Results show that selecting a few suitable channels can lead to similar or even superior accuracy compared to a classifier that utilizes all EEG channels. The results also show that the multi-objective NSGA optimization methods have a greater ability than the greedy methods to select few compatible channels. For example, 99.51% is the maximum accuracy achieved using all 19 EEG channels when processed with a combination of VMD, log energy entropy, and a K-nearest neighbor (KNN) classifier. The same accuracy is achieved with only 9 channels selected by NSGA, compared to 13 channels for greedy algorithms. With an ensemble KNN classifier, the highest accuracy, 99.81%, is achieved using 11 channels selected by an NSGA-based method. The results are encouraging, demonstrating that this brain disorder can be accurately diagnosed using a minimal number of electrodes and paving the way for its adoption in clinical practice.

A New Generated Family of Distributions: Statistical Properties and Applications with Real-Life Data

Several standard distributions can be used to model lifetime data. Nevertheless, a number of these datasets from diverse fields such as engineering, finance, the environment, biological sciences, and others may not fit the standard distributions. As a result, there is a need to develop new distributions that incorporate a high degree of skewness and kurtosis while improving the degree of goodness-of-fit in empirical distributions. In this study, by applying the T-X method, we proposed a new flexible generated family, the Ramos-Louzada Generator (RL-G) with some relevant statistical properties such as quantile function, raw moments, incomplete moments, measures of inequality, entropy, mean and median deviations, and the reliability parameter. The RL-G family has the ability to model “right,” “left,” and “symmetric” data as well as different shapes of the hazard function. The maximum likelihood estimation (MLE) method has been used to estimate the parameters of the RL-G. The asymptotic performance of the MLE is assessed by simulation analysis. Finally, the flexibility of the RL-G family is demonstrated through the application of three real complete datasets from rainfall, breaking stress of carbon fibers, and survival times of hypertension patients, and it is evident that the RL-Weibull, which is a special case of the RL-G family, outperformed its submodels and other distributions.

Haralick texture analysis for microdosimetry: characterization of Monte Carlo generated 3D specific energy distributions

Objective. Explore the application of Haralick textural analysis to 3D distributions of specific energy (energy imparted per unit mass) scored in cell-scale targets considering varying mean specific energy (absorbed dose), target volume, and incident spectrum. Approach. Monte Carlo simulations are used to generate specific energy distributions in cell-scale water voxels ((1 μm)3–(15 μm)3) irradiated by photon sources (mean energies: 0.02–2 MeV) to varying mean specific energies (10–400 mGy). Five Haralick features (homogeneity, contrast, entropy, correlation, local homogeneity) are calculated using an implementation of Haralick analysis designed to reduce sensitivity to grey level quantization and are interpreted using fundamental radiation physics. Main results. Haralick measures quantify differences in 3D specific energy distributions observed with varying voxel volume, absorbed dose magnitude, and source spectrum. For example, specific energy distributions in small (1–3 μm) voxels with low magnitudes of absorbed dose (10 mGy) have relatively high measures of homogeneity and local homogeneity and relatively low measures of contrast and entropy (all relative to measures for larger voxels), reflecting the many voxels with zero specific energy in an otherwise sporadic distribution. With increasing target size, energy is shared across more target voxels, and trends in Haralick measures, such as decreasing homogeneity and increasing contrast and entropy, reflect characteristics of each 3D specific energy distribution. Specific energy distributions for sources of differing mean energy are characterized by Haralick measures, e.g. contrast generally decreases with increasing source energy, correlation and homogeneity are often (not always) higher for higher energy sources. Significance. Haralick texture analysis successfully quantifies spatial trends in 3D specific energy distributions characteristic of radiation source, target size, and absorbed dose magnitude, thus offering new avenues to quantify microdosimetric data beyond first order histogram features. Promising future directions include investigations of multiscale tissue models, targeted radiation therapy techniques, and biological response to radiation.

Construction of similarity measure for intuitionistic fuzzy sets and its application in face recognition and software quality evaluation

As a generalization of fuzzy sets, intuitionistic fuzzy sets (IFSs) are more capable of representing and addressing uncertainty in real-world problems. As a result, IFSs have been utilized in various areas of application. However, the distance and similarity measures between the two IFSs are still an open issue that has drawn much attention over the past few decades. Even though several intuitionistic fuzzy similarity measures (IFSMs) have been developed, a number of issues still exist, including counter-intuitive results, ‘the zero divisor problem,’ violation of similarity measure axioms, and being incompetent to detect minor changes in membership or non-membership. To overcome these shortcomings, a novel intuitionistic fuzzy similarity measure (IFSM) has been introduced in this study. Unlike existing measures, this method considered the global maximum and minimum of differences in memberships and differences in non-memberships, along with their individual differences, to construct an IFSM. Moreover, it has been shown that a convex combination of two similarity measures is also a similarity measure. Some numerical examples are employed to emphasize the advantages and strengths of the proposed method over existing ones. The suggested IFSM has been implemented on a few pattern classification issues to demonstrate its efficacy and a new face recognition method is also presented. Moreover, an entropy measure is introduced using the proposed IFSM, which is further used to construct the weights of attributes in the MADM method. Furthermore, the MADM technique, IF-E-TOPSIS, is constructed using the suggested IFSM and entropy measure. The effectiveness of this method is shown by utilizing it for software quality assessment, and the results are compared with existing ones to highlight the superiority of the proposed method.

New bounds on entropies based on order statistics and Gini’s mean difference

Entropy measures are fundamental tool in information theory and various areas of sciences. We establish various lower bounds to the cumulative residual entropy and cumulative entropy based on Gini’s mean difference. Applying new results on upper bounds in Balakrishnan, Buono, and Longobardi (2022), we obtain sharper upper bounds for the cumulative residual entropy and cumulative entropy. New generalized cumulative residual entropy and cumulative entropy are also proposed.

Topological analysis of entropy measure using regression models for silver iodide

Topological analysis of entropy measure using regression models for silver iodide

Quantifying expenditure hierarchies and the expansion of global consumption diversity

Economic growth stimulates fundamental changes in consumption patterns as consumers who get rich tend to spread their spending more evenly across a wider variety of goods and services. Comparing cross sectional spending patterns across rich and poor countries, we investigate how this diversification process enables more niche patterns of spending to emerge across the global population of consumers. We use entropy measures to quantify the dispersion of household spending across goods and study how it unfolds as GDP rises. Using a gravity model to study international differences in the relative order of income elasticities, i.e. expenditure hierarchies, we show how this diversification process on the national level is correlated with cultural norms, GDP and income inequality. We find that national expenditure hierarchies are relatively similar across countries among necessities, while they are increasingly unique among luxuries. We further consider how rising affluence tends to generate more niche consumption patterns by examining how rising income is positively correlated with demand heterogeneity and income inequality is negatively correlated with market depth.

Optimization assisted autoregressive technique with deep convolution neural network-based entropy filter for image demosaicing

ABSTRACT This paper presents an image demosaicing based on an optimization-driven deep learning model, namely the Autoregressive Water Wave Optimization algorithm (Autoregressive-WWO). The proposed method is devised by assimilating the Wave Optimization algorithm (WWO), and the Conditional autoregressive value at risk (CAViaR) model. Here, the input images are subjected to Autoregressive WWO-based local polynomial approximation and intersection of confidence intervals (LPA-ICI) filter, and Deep Convolution neural network (Deep CNN) in a concurrent manner. The filter coefficients are obtained from the proposed Autoregressive WWO-based LPA-ICI filter and the residual image is obtained from Deep CNN. In order to create the demosaiced image, these two outputs are combined using an entropy measure. The proposed method offered superior performance with the highest Peak signal to noise ratio (PSNR) of 40.049dB, the highest Second derivative measure of enhancement (SDME) of 50.168dB, and highest Structural Index Similarity (SSIM) of 0.9056.

Ordinal Conditional Entropy Displays Reveal Intrinsic Characteristics of the Rosenberg Self-Esteem Scale.

Individual subjects' ratings neither are metric nor have homogeneous meanings, consequently digital- labeled collections of subjects' ratings are intrinsically ordinal and categorical. However, in these situations, the literature privileges the use of measures conceived for numerical data. In this paper, we discuss the exploratory theme of employing conditional entropy to measure degrees of uncertainty in responding to self-rating questions and that of displaying the computed entropies along the ordinal axis for visible pattern recognition. We apply this theme to the study of an online dataset, which contains responses to the Rosenberg Self-Esteem Scale. We report three major findings. First, at the fine scale level, the resultant multiple ordinal-display of response-vs-covariate entropy measures reveals that the subjects on both extreme labels (high self-esteem and low self-esteem) show distinct degrees of uncertainty. Secondly, at the global scale level, in responding to positively posed questions, the degree of uncertainty decreases for increasing levels of self-esteem, while, in responding to negative questions, the degree of uncertainty increases. Thirdly, such entropy-based computed patterns are preserved across age groups. We provide a set of tools developed in R that are ready to implement for the analysis of rating data and for exploring pattern-based knowledge in related research.

Multi-attribute decision-making based on comprehensive hesitant fuzzy entropy

It is unquestionable that hesitant fuzzy set has developed into an important tool for handling uncertain data. As a measure between data, hesitant fuzzy entropy is significant in the decision-making process. However, existing entropy measures have a series of problems such as anti intuition. In the decision-making problem under hesitant fuzzy frameworks, traditional decision-making methods such as TOPSIS method assume that the decision-maker is absolutely rational, which also makes the calculation results unreasonable. Therefore, the present study provides a comprehensive hesitant fuzzy entropy. Additionally, the traditional TOPSIS method is enhanced based on the cumulative prospect theory and the entropy method, thereby generating a multi-attribute decision-making model with unknown attribute weights. First, a new axiomatic definition of hesitant fuzzy entropy is given, then the comprehensive hesitant fuzzy entropy is defined based on the two features of fuzziness and unclarity. Besides, we prove that our method meets the new axiomatic definition. Finally, the improved TOPSIS method is applied to evaluate the comprehensive entropy measure, while simulations verify the effectiveness of the proposed method. The results showed that compared to existing entropy measures, the comprehensive hesitant fuzzy entropy more accurately reflects the uncertainty of hesitant fuzzy elements, and also has the advantages of simple calculation and easy understanding. Meanwhile, the improved TOPSIS method takes into account the psychological preferences of decision-makers, which is more reasonable than traditional methods.

Algebraic Structure Graphs over the Commutative Ring Zm: Exploring Topological Indices and Entropies Using M-Polynomials

The field of mathematics that studies the relationship between algebraic structures and graphs is known as algebraic graph theory. It incorporates concepts from graph theory, which examines the characteristics and topology of graphs, with those from abstract algebra, which deals with algebraic structures such as groups, rings, and fields. If the vertex set of a graph G^ is fully made up of the zero divisors of the modular ring Zn, the graph is said to be a zero-divisor graph. If the products of two vertices are equal to zero under (modn), they are regarded as neighbors. Entropy, a notion taken from information theory and used in graph theory, measures the degree of uncertainty or unpredictability associated with a graph or its constituent elements. Entropy measurements may be used to calculate the structural complexity and information complexity of graphs. The first, second and second modified Zagrebs, general and inverse general Randics, third and fifth symmetric divisions, harmonic and inverse sum indices, and forgotten topological indices are a few topological indices that are examined in this article for particular families of zero-divisor graphs. A numerical and graphical comparison of computed topological indices over a proposed structure has been studied. Furthermore, different kinds of entropies, such as the first, second, and third redefined Zagreb, are also investigated for a number of families of zero-divisor graphs.

Gait Monitoring and Analysis: A Mathematical Approach.

Gait abnormalities are common in the elderly and individuals diagnosed with Parkinson's, often leading to reduced mobility and increased fall risk. Monitoring and assessing gait patterns in these populations play a crucial role in understanding disease progression, early detection of motor impairments, and developing personalized rehabilitation strategies. In particular, by identifying gait irregularities at an early stage, healthcare professionals can implement timely interventions and personalized therapeutic approaches, potentially delaying the onset of severe motor symptoms and improving overall patient outcomes. In this paper, we studied older adults affected by chronic diseases and/or Parkinson's disease by monitoring their gait due to wearable devices that can accurately detect a person's movements. In our study, about 50 people were involved in the trial (20 with Parkinson's disease and 30 people with chronic diseases) who have worn our device for at least 6 months. During the experimentation, each device collected 25 samples from the accelerometer sensor for each second. By analyzing those data, we propose a metric for the "gait quality" based on the measure of entropy obtained by applying the Fourier transform.

Computational aspects of entropy measures for metal organic frameworks

ABSTRACT A form of porous polymeric material known as metal-organic frameworks is made up of metal ions connected by organic bridging ligands. Entropy is used as a complexity indicator for graphs and is regarded as the graph's structural information content. The graph entropies have evolved into information-theoretic variables for computing the structure information of chemical graphs. The potential use in chemistry of the graph entropy portion has gained the attention of the research community. In this article, we investigate the explicit expressions of graph entropies for two categories of metal organic frameworks (MOFs) named as, M 1 TPyP − M 2 (TPyP= 5, 10, 15, 20-tetrakis (4-pyridyl) porphyrin and M 1 , M 2 = Fe and Co) MOFs and butylated hydroxytoluene-(BHT-) based metal-organic (M= Co, Fe, Mn, Cr) (MBHT) frameworks. we compute some numeric values for certain graph entropies in Tables. In the last, we present a comprehensive graphic comparison of our new results.

Analysis of water conveying aluminum oxide/silver nanoparticles due to mixed convection through four square cavity's variable hot (cold) walled

The present work focuses on the measurement of the general entropy and the heat exchanges that occur inside a square-shaped cavity by changing the cold and hot wall in four cases filled with a hybrid nano-liquid (Al2O3-Ag/water) with a cylinder installed inside it. After validating the model, the parametric study has been conducted based on Rayleigh number (Ra), Hartman number (Ha), Darcy number (Da), Solid volume fraction (φ) and Porosity (ε) aspects. An effective finite element method was employed to solve the problem of flow in a dimensionless form of governing equations towards flow and thermal behaviors of nanofluid which is laminar and incompressible. Using the COMSOL Multiphysics® software computer suite, the equations for energy, motion, and continuity were resolved. Nusselt number calculations are presented to quantify heat transport via mixed convection. In the case studies, the entropy generation improves regardless of where the heated wall is located, except for the third case, where it is higher than the others. The findings are demonstrated with a higher Rayleigh number Ra, average Nusselt number, and entropy production. The heat transfer rate (HTR) can be effectively adjusted by making use of the magnetic field. The process of producing entropy in the third cavity, where the hot wall mediates the right wall, is the crucial observation made through this effort.

Dimensional measures of generalized entropy

Entropy is useful in statistical problems as a measure of irreversibility, randomness, mixing, dispersion, and number of microstates. However, there remains ambiguity over the precise mathematical formulation of entropy, generalized beyond the additive definition pioneered by Boltzmann, Gibbs, and Shannon (applicable to thermodynamic equilibria). For generalized entropies to be applied rigorously to nonequilibrium statistical mechanics, we suggest that there is a need for a physically interpretable (dimensional) framework that can be connected to dynamical processes operating in phase space. In this work, we introduce dimensional measures of entropy that admit arbitrary invertible weight functions (subject to curvature and convergence requirements). These ‘dimensional entropies’ have physical dimensions of phase-space volume and represent the extent of level sets of the distribution function. Dimensional entropies with power-law weight functions (related to Rényi and Tsallis entropies) are particularly robust, as they do not require any internal dimensional parameters due to their scale invariance. We also point out the existence of composite entropy measures that can be constructed from functionals of dimensional entropies. We calculate the response of the dimensional entropies to perturbations, showing that for a structured distribution, perturbations have the largest impact on entropies weighted at a similar phase-space scale. This elucidates the link between dynamics (perturbations) and statistics (entropies). Finally, we derive corresponding generalized maximum-entropy distributions. Dimensional entropies may be useful as a diagnostic (for irreversibility) and for theoretical modeling (if the underlying irreversible processes in phase space are understood) in chaotic and complex systems, such as collisionless systems of particles with long-range interactions.