Entropy-based Approach Research Articles

Bathymetry and discharge estimation in large and data-scarce rivers using an entropy-based approach

ABSTRACT This study implements an entropy theory-based approach to infer bathymetry for 29 selected cross-sections along a 1740 km reach of the Congo River. A genetic algorithm optimization approach is used based on an analysis of near-surface velocity measurements to generate a random sample of 1000 bathymetry profiles from which the analysis is carried out. The resulting simulated bathymetry shows good agreement compared to the measurements obtained via Accoustic Doppler Current Profiler (ADCP), with a correlation that varies from 0.49 to 0.88. The bathymetry results are subsequently used to estimate the two-dimensional cross-sectional flow velocity distribution and, consequently, to calculate the river discharge. The mean errors observed for flow area, discharge, and mean velocity are found to be 2.7%, 1.3%, and 1%, respectively. This study demonstrates, for the first time, the successful application of an entropy-based approach to estimate bathymetry and discharge in large rivers and has significant implications for remote sensing applications.

Exploring Colligation Diversity and Grammaticalization in Chinese: An Entropy-Based Approach

ABSTRACT As the process of grammaticalization unfolds, it remains to be determined whether a word could co-occur with more words in contexts or would be restricted to fewer words. Based on the Lancaster Corpus of Mandarin Chinese (LCMC), this study examines the differences in colligation diversity between lexical and grammatical words in Chinese by using entropy, aiming to explore how the colligational behaviour of the left and right sides of Chinese words changes accordingly with increasing grammaticalization. The comparisons of colligation diversity between two sides and across word categories reveal that lexical words show quite similar levels of colligation diversity on the left, which makes them significantly different from grammatical words. More category-specific observations are disclosed by entropy-based approach. In the case of grammaticalization, an increase in entropy values denotes more types with a more uniform distribution, which is suggested to be the manifestation of semantic bleaching. Conversely, a decrease in entropy values may be an indicator of an increasing bondedness. The discussion on how grammaticalization affects the colligational behaviour of words should be based on the specific pathways of grammaticalization concerning word categories, as well as specific sides of words.

Model-agnostic variable importance for predictive uncertainty: an entropy-based approach

In order to trust the predictions of a machine learning algorithm, it is necessary to understand the factors that contribute to those predictions. In the case of probabilistic and uncertainty-aware models, it is necessary to understand not only the reasons for the predictions themselves, but also the reasons for the model’s level of confidence in those predictions. In this paper, we show how existing methods in explainability can be extended to uncertainty-aware models and how such extensions can be used to understand the sources of uncertainty in a model’s predictive distribution. In particular, by adapting permutation feature importance, partial dependence plots, and individual conditional expectation plots, we demonstrate that novel insights into model behaviour may be obtained and that these methods can be used to measure the impact of features on both the entropy of the predictive distribution and the log-likelihood of the ground truth labels under that distribution. With experiments using both synthetic and real-world data, we demonstrate the utility of these approaches to understand both the sources of uncertainty and their impact on model performance.

Entropy-Based Strategies for Multi-Bracket Pools.

Much work in the parimutuel betting literature has discussed estimating event outcome probabilities or developing optimal wagering strategies, particularly for horse race betting. Some betting pools, however, involve betting not just on a single event, but on a tuple of events. For example, pick six betting in horse racing, March Madness bracket challenges, and predicting a randomly drawn bitstring each involve making a series of individual forecasts. Although traditional optimal wagering strategies work well when the size of the tuple is very small (e.g., betting on the winner of a horse race), they are intractable for more general betting pools in higher dimensions (e.g., March Madness bracket challenges). Hence we pose the multi-brackets problem: supposing we wish to predict a tuple of events and that we know the true probabilities of each potential outcome of each event, what is the best way to tractably generate a set of n predicted tuples? The most general version of this problem is extremely difficult, so we begin with a simpler setting. In particular, we generate n independent predicted tuples according to a distribution having optimal entropy. This entropy-based approach is tractable, scalable, and performs well.

Spatial analysis of the digital economy’s influence on urban sustainable development: A decade-long study of Chinese prefecture-level cities

Urban Sustainable Development (USD) is crucial for social advancement, and the rise of the Digital Economy (DE) offers potential pathways to achieve it. This study has employed the DPSIR framework and an entropy-based approach to evaluate the DE and USD across 284 Chinese prefecture-level cities from 2011 to 2020. Using the Spatial Durbin Model, the results have revealed that DE significantly enhances USD and has a spatial spillover effect of up to 300 km. The eastern region, as well as the Pearl River Delta and Yangtze River Delta regions, show stronger spillover effects than the central and western regions and the Beijing-Tianjin-Hebei region. These findings have provided insights for policymakers to leverage digital advancements to promote urban sustainability.

Acoustic Emission Monitoring of Bond Deterioration in GFRP-Reinforced Concrete Members: An Entropy-Based Approach

The primary objective of acoustic emission (AE) monitoring is to accurately detect imminent critical damage, preventing premature failure by analysing the dynamic evolution of AE parameters. This study introduces AE entropy, a distinctive parameter derived from Shannon's entropy, employed to monitor the bond degradation between glass fibre-reinforced polymer (GFRP) rebars and the concrete interface in GFRP-reinforced concrete elements. Characteristic AE parameters have been synchronously collected during flexural bond tests, and information entropy and weighted information entropy, composed of typical AE characteristic parameters based on Shannon’s information entropy theory in the time domain, have been defined. Findings indicate that, with increasing bond stress, the specimens exhibit a trend characterized by "decreasing, stabilizing, and then increasing" in both AE characteristic parameters and weighted information entropy. The information entropy value shows fluctuations during any damage associated with GFRP-concrete bond deterioration. At peak bond stress values, information entropy values experience a rapid surge, indicating significant deterioration. The evolution process of debonding between GFRP rebar and concrete, switching between a chaotic state that develops randomly and an organized state that produces in order, follows specific laws (disordered–ordered–disordered), corresponding well to AE characteristic parameters weighted information entropy. Moreover, entropy values, both individual and weighted, when correlated with bond stress and slip variations in the test specimens, distinctly outline and predict various stress transfer mechanisms between the GFRP bar and concrete. In summary, this research underscores the utility of AE entropy as a valuable tool for damage monitoring in GFRP-reinforced structures. Its ability to capture microstructural deformations positions AE entropy as a promising candidate for improving critical damage identification and preventing premature failure

Investigation of an Inter-Compressor S-Duct Using the Lattice Boltzmann Method

Abstract This article presents the study of a subsonic inter-compressor S-duct. Numerical simulations are performed using large-eddy simulation (LES) based on a compressible hybrid thermal lattice Boltzmann method (LBM) implemented within the ProLB solver. Comparisons are made between the LES–LBM results, Reynolds-averaged Navier–Stokes (RANS) computations, and experimental measurements on a representative S-duct taken from the European project AIDA. Several cases with increasing complexity are addressed where the different rows surrounding the duct are gradually included in the computations. The effects of each row on the flow field development and loss levels are studied. The goal is to evaluate the ability of the LES–LBM to recover the aerodynamic behavior and the total pressure loss evolution within the duct. Results show that the LES–LBM retrieves the correct flow evolution inside the S-duct compared to the experiment and previous RANS results. The case where the upstream stator row or the low-pressure compressor stage is integrated shows an increase in total pressure loss, as previously observed in the literature, and a more developed flow field with complex flow features contributing to the loss generation. To further analyze the loss mechanism, an entropy-based approach is presented and highlights that most losses are generated close to the hub wall due to the migration of the upstream stator wakes.

An entropy-based study on the mutational landscape of SARS-CoV-2 in USA: Comparing different variants and revealing co-mutational behavior of proteins

COVID-19 emergency has pushed the international scientific community to use every resource to combat the spread of the virus, to understand its biology and predict its possible evolution in terms of new variants. Since the first SARS-CoV-2 virus nucleotide and amino acid sequences were made available, information theory was used to study how viral information content was changing over time and then trace the evolution of its mutational landscape. In this work we analyzed SARS-CoV-2 sequences collected mainly in the USA in a period from March 2020 until December 2022 and computed mutation profiles of viral proteins over time through an entropy-based approach using Shannon Entropy and Hellinger distance. This representation allows an at-a-glance view of the mutational landscape of viral proteins over time and can provide new insights on the evolution of the virus from different points of view. Non-structural proteins typically showed flat mutation profiles, characterized by a very low Average mutation Entropy, while accessory and structural proteins showed mostly non uniform and high mutation profiles, often coupled with the predominance of variants. Interestingly NSP2 protein, whose function is currently still debated, falls in the same branch of NSP14 and NSP10 in the phylogenetic tree of mutations constructed through correlations of mutation profiles, suggesting a co-evolution of those proteins and a possible functional link with each other. To the best of our knowledge this is the first study based on a massive amount of data (n = 107,939,973) that analyzes from an entropy point of view the mutational landscape of SARS-CoV-2 over time and depicts a mutational temporal profile of each protein of the virus.

New information measures for linear Diophantine fuzzy sets and their applications with LDF-ARAS on data storage system selection problem

In addition to membership and non-membership degrees, linear Diophantine fuzzy set (LDFS) incorporates two reference parameters in fuzzy set definition to represent human judgment more comprehensively. It also provides flexibility in updating the decision setting by changing the meaning of the reference parameters when required by the decision-maker. Information measures in fuzzy sets have many application areas in the literature. Especially, distance and entropy measures are utilized in multiple attribute decision-making (MADM) field. Distance measures are needed by distance-based methods like TOPSIS and EDAS because their alternative ranking procedure is based on measuring the distances between alternatives and some artificial ideal or average solutions. Entropy measure is a very important tool being used in objective attribute weighting procedures of MADM applications. Thus, new distance and entropy measure propositions for LDFS are the first contribution of the study. Also, Additive Ratio Assessment (ARAS) method is extended into LDFS (LDF-ARAS) for the first time in the literature as a second contribution. In this extension, we selected ARAS as one of the most cited tools because it is found flexible and easily adaptable by researchers who do not have any background in MADM. The proposed entropy measure is integrated with the objective attribute weighting procedure of the novel LDF-ARAS approach. The third contribution is the newly developed decision model proposed for a real-life data storage system (DSS) selection problem. Three DSS options were evaluated with respect to thirteen attributes by five skilled and well-educated experts and LDF-ARAS was performed to rank alternatives. The entropy-based LDF-ARAS approach was validated by comparing the results of six different applications. In the first three, different entropy formulas are used and in the last three, these entropy measures are integrated with LDF-TOPSIS. All six comparisons gave similar alternative rankings.

Motor Imagery Classification Using Effective Channel Selection of Multichannel EEG.

Electroencephalography (EEG) is effectively employed to describe cognitive patterns corresponding to different tasks of motor functions for brain-computer interface (BCI) implementation. Explicit information processing is necessary to reduce the computational complexity of practical BCI systems. This paper presents an entropy-based approach to select effective EEG channels for motor imagery (MI) classification in brain-computer interface (BCI) systems. The method identifies channels with higher entropy scores, which is an indication of greater information content. It discards redundant or noisy channels leading to reduced computational complexity and improved classification accuracy. High entropy means a more disordered pattern, whereas low entropy means a less disordered pattern with less information. The entropy of each channel for individual trials is calculated. The weight of each channel is represented by the mean entropy of the channel over all the trials. A set of channels with higher mean entropy are selected as effective channels for MI classification. A limited number of sub-band signals are created by decomposing the selected channels. To extract the spatial features, the common spatial pattern (CSP) is applied to each sub-band space of EEG signals. The CSP-based features are used to classify the right-hand and right-foot MI tasks using a support vector machine (SVM). The effectiveness of the proposed approach is validated using two publicly available EEG datasets, known as BCI competition III-IV(A) and BCI competition IV-I. The experimental results demonstrate that the proposed approach surpasses cutting-edge techniques.

Who dominate the information flowing between innovative and traditional financial assets? A multiscale entropy-based approach

We investigate the interactions between innovative financial assets (e.g., FinTech-related stocks, green bonds, and cryptocurrencies) and traditional ones (e.g., global equities, gold, crude oil, U.S. dollar, and government bonds) at short-, medium-, and long-time scales by adopting the multiscale entropy-based approach, through which the dominant influencers in information flowing are identified. Also, we discuss the impact of unprecedented events exemplified by the COVID-19 pandemic. Further, we propose the high-transfer-entropy trading strategy that considers the dominant influencers. The empirical results show that (i) the information exchange is heterogenous at three time scales, with the greatest intensity at short-time scale; (ii) no isolated dominant influencer is discovered, and the traditional assets are stronger influencers in long term while the innovative assets are more influential in short and medium terms; (iii) the COVID-19 pandemic alters the magnitude and direction of information transfer, during which the influence of innovative assets relative to traditional assets enhances evidently; and (iv) the high-transfer-entropy strategy can effectively provide the investor with excess returns.

Semantically redundant training data removal and deep model classification performance: A study with chest X-rays

Deep learning (DL) has demonstrated its innate capacity to independently learn hierarchical features from complex and multi-dimensional data. A common understanding is that its performance scales up with the amount of training data. However, the data must also exhibit variety to enable improved learning. In medical imaging data, semantic redundancy, which is the presence of similar or repetitive information, can occur due to the presence of multiple images that have highly similar presentations for the disease of interest. Also, the common use of augmentation methods to generate variety in DL training could limit performance when indiscriminately applied to such data. We hypothesize that semantic redundancy would therefore tend to lower performance and limit generalizability to unseen data and question its impact on classifier performance even with large data. We propose an entropy-based sample scoring approach to identify and remove semantically redundant training data and demonstrate using the publicly available NIH chest X-ray dataset that the model trained on the resulting informative subset of training data significantly outperforms the model trained on the full training set, during both internal (recall: 0.7164 vs 0.6597, p<0.05) and external testing (recall: 0.3185 vs 0.2589, p<0.05). Our findings emphasize the importance of information-oriented training sample selection as opposed to the conventional practice of using all available training data.

A combined cyclic viscoplasticity and entropy generation approach for modelling fatigue crack growth behavior of a nickel-based superalloy at high temperature

This paper developed an entropy-based approach to estimate the fatigue crack growth (FCG) behavior of GH4169 at high temperature. Firstly, systematic FCG experiments of GH4169 at 650 °C were conducted to obtain the experimental data. Then, a finite element analysis combined with Chaboche viscoplasticity and node release technology was developed to obtain cyclic stress–strain responses and entropy generation fields. Based on thermodynamic analysis, numerous discrete material elements as sub-systems were divided along the crack growth direction in simplified representative 2D middle section. Fracture process zone (FPZ) was defined equal to the size of discrete material elements (ρ) which can be determined by distribution of normal stress perpendicular to the crack plane. Subsequently, an effective cyclic entropy generation related to ρ was defined as the crack driving force. Effective cumulative entropy generation was calculated when crack increment was ρ and the results indicated that effective cumulative entropy generation float within a certain range. The average of all effective cumulative entropy generation values was defined as crack growth entropy. Finally, an entropy-based FCG rate model was established to estimate the FCG behavior of GH4169, and a good correlation between experimental results and model predictions are achieved for all the high temperature FCG tests.

Evaluation of micromobility risk management alternatives using interval-valued q-rung orthopair fuzzy interaction operators-based WISP method

Micro-mobility has become a mode of transportation since the 2010 s. Because this mode is relatively new, there are many unknowns, and the associated risks must be evaluated so they can be managed safely and effectively. This paper aims to introduce a decision support system (DSS) for ranking risk management alternatives in the micro-mobility sector under an interval-valued q-rung orthopair fuzzy (IVq-ROF) context. In this context, firstly IVq-ROF-interaction aggregation operators (AOs) are introduced for aggregating the information from IVq-ROF perspective. These AOs can provide a flexible mode to tackle realistic decision support problems within the IVq-ROF set (IVq-ROFS) context. Second, a definition of entropy and new entropies for IVq-ROFS is introduced. Third, a hybrid weight-determining formula is presented using an entropy-based weighting approach for the objective weight of criteria and a stepwise weight assessment ratio analysis (SWARA) method for the subjective weight of criteria. Fourth, the weighted integrated sum-product (WISP) approach is expanded from the IVq-ROF perspective. Further, the developed model is utilized in a case study of risk management alternative evaluation in the micro-mobility sector, which shows its applicability and feasibility under the IVq-ROF environment. Sensitivity and comparative investigations are studied to confirm the strength and stability of the obtained result. The best alternative has emerged as the state-private sector cooperation in successfully allocating the risk in the micro-mobility sector. The final results conclude that the developed methodology can provide more accurate decisions while considering multiple inducing indicators and input uncertainties.

Novel entropy generation and optimization for hybrid nanofluid flow via ribbed surface parabolic trough collector

This research paper focuses on the optimization of parameters for deformative receiver tubes to achieve prime choice of design for efficient energy absorption. The motivation of the current study lies in addressing the critical need for optimizing parameters of deformative receiver tube to obtain optimal design to enhance energy absorption efficiency in Parabolic Trough Solar Collectors (PTSC). Considering the diverse criteria involved in the design process, the application of a Multi Criteria Decision Making (MCDM) approach proves beneficial. The primary objective of this study is to identify the optimal combination of parameters (Reynolds number (Re), Relative V-rib height (Rh/ Dt), Relative streamwise pitch (PStm/Rh), and Relative spanwise pitch ratio (PSpn/Rh)) of the deformative receiver tube of PTSC) employing the multiobjective optimization based on ratio analysis (MULTIMOORA) method. Additionally, a novel Pythagorean fuzzy sets (PFSs) entropy-based approach is proposed for quantifying the weights of the criteria. The feasibility and practicality of the novel PFSs entropy and MULTIMOORA method are demonstrated through the application of the Technique for Order of Preference by Similarity to Ideal Solution method (TOPSIS), confirming the attainment of Re = 50000, Rh/ Dt=0.0135, Pstm/ Rh=56.67 and Pspn/ Rh=11.11, as optimal alternative parameters. This research offers valuable insights into optimizing deformative receiver tube parameters to improve thermal performance which benefiting researchers in the concerned field.

An entropy-based approach for assessing the directional persistence of cell migration

Cell migration, which is primarily characterized by directional persistence, is essential for the development of normal tissues and organs, as well as for numerous pathological processes. However, there is a lack of simple and efficient tools to analyze the systematic properties of persistence based on cellular trajectory data. Here, we present a novel approach, the entropy of angular distribution , which combines cellular turning dynamics and Shannon entropy to explore the statistical and time-varying properties of persistence that strongly correlate with cellular migration modes. Our results reveal the changes in the persistence of multiple cell lines that are tightly regulated by both intra- and extracellular cues, including Arpin protein, collagen gel/substrate, and physical constraints. Significantly, some previously unreported distinctive details of persistence have also been captured, helping to elucidate how directional persistence is distributed and evolves in different cell populations. The analysis suggests that the entropy of angular distribution-based approach provides a powerful metric for evaluating directional persistence and enables us to better understand the relationships between cellular behaviors and multiscale cues, which also provides some insights into the migration dynamics of cell populations, such as collective cell invasion.

Multivariate multi-scale cross-fuzzy entropy and SSA-SVM-based fault diagnosis method of gearbox

Abstract Fuzzy entropy (FuzzyEn) is widely recognized as a powerful tool for analyzing nonlinear dynamics and measuring the complexity of time series data. It has been utilized as an effective indicator to capture nonlinear fault features in gearbox vibration signals. However, FuzzyEn only measures complexity at a single scale, ignoring the valuable information contained in large-scale features of the time series. Furthermore, FuzzyEn does not account for coupling characteristics between related or synchronized time series. To address these limitations, a novel entropy-based approach called multivariate multi-scale cross-fuzzy entropy (MvMCFE) is proposed in this paper for measuring the complexity and mutual predictability of two multivariate time series. Relying on the advantages of MvMCFE in nonlinear feature extraction, a new fault diagnosis method for gearboxes is proposed based on MvMCFE and an optimized support vector machine (SVM) using the salp swarm algorithm (SSA-SVM). Ultimately, the proposed gearbox diagnostic method is employed to analyze the gearbox experimental data and a comparison with existing fault diagnosis approaches is conducted. The comparison results indicate that the proposed method can effectively extract nonlinear fault features of gearboxes and achieve the highest recognition rate compared to the other methods.

An entropy-based approach for a robust least squares spline approximation

We consider the weighted least squares spline approximation of a noisy dataset. By interpreting the weights as a probability distribution, we maximize the associated entropy subject to the constraint that the mean squared error is prescribed to a desired (small) value. Acting on this error yields a robust regression method that automatically detects and removes outliers from the data during the fitting procedure, by assigning them a very small weight. We discuss the use of both spline functions and spline curves. A number of numerical illustrations have been included to disclose the potentialities of the maximal-entropy approach in different application fields.

Optimal site selection for construction and demolition waste disposal using GIS-FAHP: a case study from Kafr El Sheikh City, Egypt.

The increasing number of building and demolition projects results in huge amounts of construction and demolition wastes (CDWs) that are illegally dumped. However, these wastes must be disposed of in appropriate legal sites to protect the environment and human health. After reviewing the literature, no prior research examined optimal site selection for dumping or recycling CDW in an Egyptian city. Furthermore, the absence of field surveys did not offer a holistic understanding of the specific criteria used in the model for this region, nor did it permit an assessment of the suitability of existing dumpsites, thereby revealing certain limitations in the final results. In this regard, this research aims to apply a multi-criteria geographic information system (GIS)-based framework to identify an optimal site for CDW disposal in Kafr El Sheikh City. The criteria affecting the site selection are identified and categorized from prior literature, which are further refined using field surveys and focus group to evaluate their applicability in the context of an Egyptian city. After conducting questionnaire surveys, the trapezoidal interval type II fuzzy analytic hierarchy process is applied to compute the weights of the identified criteria from the perspective of each group of experts. The entropy-based aggregation approach is employed to identify the compromise weights taking into account the preferences of different groups. GIS is a powerful tool for geoprocessing and analyzing spatial big data. The result is a scenario map for the optimal site locations with varying suitability scales (i.e., excellent, very good, good, average, poor, and very poor). The proposed methodology provides what-if scenarios based on a selected set of criteria. According to the results of the multi-criteria decision analysis models, the suitability varies based on the weights of the criteria. For the equal-weighted criteria model, the excellent category covers 5.96% of the study area, increasing to 6.48% for the weighted criteria model. These areas primarily lie in the northeast direction. Conversely, the majority of the study area, 41.80% under equal-weighted criteria and 32.39% under weighted criteria, falls within the average and poor suitability categories, respectively. In general, the most suitable areas are located on the outskirts of the city, and the suitability decreases near the central business district. To bridge the gap between research findings and practical applications, a land use analysis employing satellite imagery is conducted to pinpoint suitable locations for CDW disposal. Existing CDW dumpsites predominantly fall within the range of poor to very good for the equal-weighted criteria model, while the weighted criteria model categorizes them into the poor (16.66%) and average (83.33%) categories. The findings demonstrated the applicability of the proposed framework for CDW disposal management and planning.

A validation of an entropy-based artificial intelligence for ultrasound data in breast tumors

BackgroundThe application of artificial intelligence (AI) in the ultrasound (US) diagnosis of breast cancer (BCa) is increasingly prevalent. However, the impact of US-probe frequencies on the diagnostic efficacy of AI models has not been clearly established.ObjectivesTo explore the impact of using US-video of variable frequencies on the diagnostic efficacy of AI in breast US screening.MethodsThis study utilized different frequency US-probes (L14: frequency range: 3.0-14.0 MHz, central frequency 9 MHz, L9: frequency range: 2.5-9.0 MHz, central frequency 6.5 MHz and L13: frequency range: 3.6-13.5 MHz, central frequency 8 MHz, L7: frequency range: 3-7 MHz, central frequency 4.0 MHz, linear arrays) to collect breast-video and applied an entropy-based deep learning approach for evaluation. We analyzed the average two-dimensional image entropy (2-DIE) of these videos and the performance of AI models in processing videos from these different frequencies to assess how probe frequency affects AI diagnostic performance.ResultsThe study found that in testing set 1, L9 was higher than L14 in average 2-DIE; in testing set 2, L13 was higher in average 2-DIE than L7. The diagnostic efficacy of US-data, utilized in AI model analysis, varied across different frequencies (AUC: L9 > L14: 0.849 vs. 0.784; L13 > L7: 0.920 vs. 0.887).ConclusionThis study indicate that US-data acquired using probes with varying frequencies exhibit diverse average 2-DIE values, and datasets characterized by higher average 2-DIE demonstrate enhanced diagnostic outcomes in AI-driven BCa diagnosis. Unlike other studies, our research emphasizes the importance of US-probe frequency selection on AI model diagnostic performance, rather than focusing solely on the AI algorithms themselves. These insights offer a new perspective for early BCa screening and diagnosis and are of significant for future choices of US equipment and optimization of AI algorithms.