Bayesian Entropy Research Articles

Bayesian estimation under different loss functions for the case of inverse Rayleigh distribution

In this study, the best parameter estimator for the scale parameter of the inverse Rayleigh distribution was determined based on a comparison of the maximum likelihood estimator (MLE) method, the Bayesian generalized squared error loss function (SELF), the Bayesian linear exponential loss function (LINEX LF), and the Bayesian entropy loss function (ELF). The prior distribution chosen was the non-informative prior, namely the Jeffrey prior, and the informative prior using the exponential distribution. The estimator evaluation method used was based on the smallest value of the Akaike information criterion (AIC), corrected Akaike information criterion (AICc), and Bayesian information criterion (BIC). Based on simulation studies and real data, it was found that the best parameter estimator on the data for the scale parameter of the inverse Rayleigh distribution is the Bayes ELF prior exponential .

Digital twin Bayesian entropy framework for corrosion fatigue life prediction and calibration of bridge suspender

This paper proposes an intelligent digital twin framework for corrosion fatigue life prediction and calibration of suspender wires integrated with mechanism-driven, sensor-driven, and information fusion. A general probabilistic information fusion strategy is constructed to handle entropy-based external constraints and classical Bayesian updating. Statistical moment, range bound, and point data are considered to investigate the effect of various types and sequences of information. A small-time domain fatigue crack growth model is proposed to overcome the limitations of traditional cycle-based methods, which can capture the large and small cycles of random fatigue stress. The virtual sensor-based stress time-history response is obtained under different traffic flow densities through digital twin finite element model of a suspension bridge. The results show that with and without considering interval bound leads to different fatigue life prediction results, especially for statistical moment data fusion, and the maximum difference is approximately 54%. The average prediction life of suspender wires is gradually close to the actual service life as crack observations increase. The standard deviations of the corrosion fatigue life decrease by 88%, when simultaneously integrating moment, interval, and point data.

Posterior Representations for Bayesian Context Trees: Sampling, Estimation and Convergence

We revisit the Bayesian Context Trees (BCT) modelling framework for discrete time series, which was recently found to be very effective in numerous tasks including model selection, estimation and prediction. A novel representation of the induced posterior distribution on model space is derived in terms of a simple branching process, and several consequences of this are explored in theory and in practice. First, it is shown that the branching process representation leads to a simple variable-dimensional Monte Carlo sampler for the joint posterior distribution on models and parameters, which can efficiently produce independent samples. This sampler is found to be more efficient than earlier MCMC samplers for the same tasks. Then, the branching process representation is used to establish the asymptotic consistency of the BCT posterior, including the derivation of an almost-sure convergence rate. Finally, an extensive study is carried out on the performance of the induced Bayesian entropy estimator. Its utility is illustrated through both simulation experiments and real-world applications, where it is found to outperform several state-of-the-art methods.

Bayesian Entropy Methodology: A Novel Approach to Setting Anti-Islanding Protections with Enhanced Stability and Sensibility

The risk of unintentional islanding creation in distributed energy systems poses a significant security concern since unintentional islanding formation could lead to a supply of energy outside of the optimal quality limits. This constitutes a risk for users, maintenance personnel, infrastructure, and devices. To mitigate this problem, anti-islanding protections are widely used to prevent the distributed generator from feeding a portion of the radial distribution grid when a protection device trips upstream. However, the effectiveness of these protections heavily relies on properly tuning protection setting thresholds (such as time delay and pickup). This work proposes a novel approach that utilizes entropy as a model and metric of the uncertainty associated with a particular protection setting. By minimizing entropy, the proposed method aims to improve stability and sensitivity, consequently improving the overall performance of anti-islanding protection. Simulation results demonstrate that the Bayesian entropy methodology (BEM) approach achieves enhanced stability in various scenarios, including frequency transients, and demonstrates a notable reduction in the size of the dataset and computational burden, ranging between 91% and 98%, when compared to related works, with an improvement of the uncertainty achieved. The findings of this study contribute to the development of more robust and reliable anti-islanding protections.

Ordered weighted utility distance operators and their applications in group decision-making

In multiple attribute group decision-making (MAGDM), a consensus exists that the psychological factors of decision makers (DMs) generally influence their choice of optimal alternatives. To quantify well the influence of DMs’ risk attitudes on the group decision-making process, this paper develops a new approach to MAGDM by introducing DMs’ utility functions into the aggregation process. First, the ordered weighted utility distance (OWUD) operator is constructed by introducing DMs’ utility functions into the distance operator. As a novel operator, the OWUD operator not only satisfies both the basic features of distance measures and the desirable properties of aggregation operators, but also has some unique characteristics. Then, the linear risk tolerance (LRT) utility function is further used in the OWUD operator as the basic utility function, and a specific form of the OWUD operator, which is named as the ordered weighted-linear risk tolerance utility distance (OW-LRTUD) operator, is proposed. The OW-LRTUD operator can quantify well the influence of DMs’ risk attitudes on the group decision-making process. It also has an analytically tractable form and includes many well-known distance measures and aggregation operators, and thus can be applied to many different kinds of decision-making problems. Subsequently, to determine the weights of the OW-LRTUD operator, an extension of the maximum Bayesian entropy method is proposed by incorporating the prior known weight information and different risk attitudes of all individual DMs. An approach to MAGDM based on the OW-LRTUD operator and the proposed weight determination method is developed. An application for determining the optimal alternative in the investment selection is provided to show the feasibility, effectiveness and robustness of our approach in practice, and the corresponding sensitive analysis illustrates the influence of DMs’ risk attitudes on the group decision-making process.

Low-probability states, data statistics, and entropy estimation.

A fundamental problem in the analysis of complex systems is getting a reliable estimate of the entropy of their probability distributions over the state space. This is difficult because unsampled states can contribute substantially to the entropy, while they do not contribute to the maximum likelihood estimator of entropy, which replaces probabilities by the observed frequencies. Bayesian estimators overcome this obstacle by introducing a model of the low-probability tail of the probability distribution. Which statistical features of the observed data determine the model of the tail, and hence the output of such estimators, remains unclear. Here we show that well-known entropy estimators for probability distributions on discrete state spaces model the structure of the low-probability tail based largely on a few statistics of the data: the sample size, the maximum likelihood estimate, the number of coincidences among the samples, and the dispersion of the coincidences. We derive approximate analytical entropy estimators for undersampled distributions based on these statistics, and we use the results to propose an intuitive understanding of how the Bayesian entropy estimators work.

Beta-Bézier OWA operator

Determination of associated weight vectors for an ordered weighted averaging (OWA) operator is always a crucial issue, and for this purpose, several weight vector determination models have been introduced in the literature. In this article, we propose a new biparametric OWA operator, called a “beta-Bézier OWA operator”. The beta-Bézier OWA operator provides an infinite number of weight vectors for a fixed degree of orness. The orness value of this class of OWA operators is its most essential characteristic, which always depends on one of its parameters, and can be prefixed according to the decision-maker choice and also irrespective of the number of arguments aggregated. The optimum OWA weight vector can be calculated without solving a complicated optimization problem. It becomes the same as several other OWA operators at particular values of parameters. The maximum Bayesian entropy OWA operator weight vector is measured with use of the proposed OWA operator for a given level of optimism/pessimism (orness). It is also compared with many other OWA operators.

Association of latent class analysis-derived subphenotypes of acute kidney injury with mortality in critically ill patients with cardiovascular disease: a retrospective cohort study

BackgroundTo explore the potential heterogeneity of acute kidney injury (AKI) and evaluate the prognostic differences among AKI subphenotypes in critically ill patients with cardiovascular diseases.MethodsData were extracted from the Medical Information Mart for Intensive Care (MIMIC)-III database. Latent class analysis (LCA) was used to explore the potential subphenotypes of AKI in critically ill patients with cardiovascular diseases. The number of classes was identified by the Bayesian information criterion and entropy. The differences in prognostic ability among the AKI subphenotypes were evaluated by logistic regression analysis.ResultA total of 7738 AKI patients were enrolled in this study. Using LCA, AKI patients were divided into 4 heterogeneous subphenotypes, which were obviously different from the Kidney Disease: Improving Global Outcomes (KDIGO) stages. Interestingly, class 3 classified by LCA was dominated by stage 2, while the mortality rate in class 3 was significantly different from that in class 1 (15.2% vs. 1.6%, p < 0.05). After further adjustment, the mortality rate in class 3 remained higher than that in class 1, with an odds ratio of 12.31 (95% confidence interval, 8.96–16.89).ConclusionsLCA was feasible for AKI classification in critically ill patients with cardiovascular disease, and 4 distinct subphenotypes of AKI patients with different prognoses were identified. Our results highlighted the potential heterogeneity of AKI patients, which is worthy of further investigation.

Facial processing in bipolar disorder is mediated by clinical and biological aspects.

The process of detecting faces can be considered one of the initial steps in face recognition, which is essential for human interaction. We sought to investigate whether a face perception task reliably detects subtle perceptual disturbances between patients with bipolar disorder (BD) and healthy controls. In this multisite study, we examined differences between BD patients and matched healthy controls. Participants were instructed to detect the orientation (either left or right) of a face when it was presented as a face/non-face pair on a computer screen using Bayesian entropy estimation. Data analyses compared performance between the groups. Overall, BD patients exhibited more perceptual disturbances compared with controls. BD patients who took olanzapine had better performance and faster reaction times (RTs) than patients who took lithium or were medication-naive. BD patients who took lithium had better performance and faster RTs than medication-naive patients. The medication-naive BD group exhibited greater disturbances than all other groups. These findings highlight the reliability of the face perception task used herein and may be important for public health initiatives and follow-up studies that seek to understand the diverse effects of other variables that can affect sensory processing in this population.

A maximum entropy formalism model for the breakup of a droplet

A model for the prediction of the size and velocity distribution of daughter droplets created by the breakup of an unstable parent droplet is proposed. The basis of the model is the maximum entropy formalism, which states that the most probable joint probability density function (JPDF) for the daughter droplet population is the one that maximizes the Bayesian entropy conditional on the enforcement of a set of constraints, which are the conservation laws for the problem. The result is a closed-form expression for the JPDF. Validation against experimental and Direct Numerical Simulations data over the bag, multimode, and sheet-thinning breakup regimes is included. Predictions from the model show that the daughter droplet velocity distribution widens as the droplet size decreases. This result is due to a heightened sensitivity to drag force with lower droplet inertia and coincides with spray behavior. The velocity distribution is found to be near Gaussian. The model does not treat size and velocity as independently distributed, as generally assumed in the literature. In fact, marginal conditional densities derived from JPDF show that the distribution of size and velocity are interrelated.

Abstract 9592: Acute Kidney Injury Subphenotypes in Critically Ill Patients with Cardiovascular Diseases by Latent Class Analysis: A Retrospective Cohort Study

Introduction: According to the potential heterogeneity of acute kidney injury (AKI) in critically ill patients with cardiovascular diseases, we explored the subphenotypes of AKI and evaluated the prognostic differences among AKI subphenotypes. Hypothesis: We hypothesized that patients with different AKI subphenotypes would suffer different prognosis. Methods: Clinical information from the Medical Information Mart for Intensive Care (MIMIC)-III database was enrolled. Latent class analysis (LCA) was used to explore the potential subphenotypes of AKI in critically ill patients with cardiovascular diseases. The number of classes was identified using the Bayesian information criterion and entropy. The differences in prognostic ability among the AKI subphenotypes were evaluated by logistic regression analysis. Results: In this study, a total of 7738 AKI patients were enrolled and divided into 4 heterogeneous subphenotypes by using LCA. The subphenotypes showed remarkable dofference from kidney disease classified by improving Global Outcomes (KDIGO) stages. Notedly, though both dominated by stage 2, the mortality between class 4 and class 1 classified by LCA were significantly different (15.2% vs. 1.6%, p &lt;0.05). The trend remained the same after further adjustment, with an odds ratio of 12.31 (95% confidence interval, 8.96-16.89). Conclusions: LCA was feasible for AKI classification and different subphenotypes prognosis prediction in critically ill patients with cardiovascular disease. Our study revealed the potential heterogeneity of AKI patients, which is worthy of further investigation.

Coherence and Entropy of Credit Cycles across the Euro Area Candidate Countries.

The pattern of financial cycles in the European Union has direct impacts on financial stability and economic sustainability in view of adoption of the euro. The purpose of the article is to identify the degree of coherence of credit cycles in the countries potentially seeking to adopt the euro with the credit cycle inside the Eurozone. We first estimate the credit cycles in the selected countries and in the euro area (at the aggregate level) and filter the series with the Hodrick–Prescott filter for the period 1999Q1–2020Q4. Based on these values, we compute the indicators that define the credit cycle similarity and synchronicity in the selected countries and a set of entropy measures (block entropy, entropy rate, Bayesian entropy) to show the high degree of heterogeneity, noting that the manifestation of the global financial crisis has changed the credit cycle patterns in some countries. Our novel approach provides analytical tools to cope with euro adoption decisions, showing how the coherence of credit cycles can be increased among European countries and how the national macroprudential policies can be better coordinated, especially in light of changes caused by the pandemic crisis.

Bayesian-entropy gaussian process for constrained metamodeling

A novel Bayesian-Entropy Gaussian Process (BEGP) is proposed for constrained metamodeling. Gaussian Process (GP) regression is a flexible and robust tool for surrogate modeling using observation data. For many engineering problems, available information other than observations may be known, such as physical constraints, boundary conditions, and empirical knowledge. Based on the Bayesian-Entropy (BE) principle, this paper introduces a novel framework for encoding extra information in addition to point data in constructing a GP regression model. The extra information is treated as constraints on the mean prediction of GP. The BE method can rigorously incorporate extra information as constraints into classical Bayesian framework. The constraint term is added into the posterior distribution of the hyperparameters when training the GP model. BEGP serves as an information fusion tool to enhance the extrapolation behavior of the GP model by incorporating additional knowledge about the problem. The proposed methodology is demonstrated on a numerical toy example and a structural analysis example highlighting the ability to smoothly connect two local GPs and incorporate boundary conditions as extra constraints. The BEGP shows the ability of incorporating physics constraints to enhance prediction and extrapolation behaviors. Finally, conclusions and future work are drawn based on the proposed study.

A Bayesian Entropy Approach to Sectoral Systemic Risk Modeling.

We investigate the dynamics of systemic risk of European companies using an approach that merges paradigmatic risk measures such as Marginal Expected Shortfall, CoVaR, and Delta CoVaR, with a Bayesian entropy estimation method. Our purpose is to bring to light potential spillover effects of the entropy indicator for the systemic risk measures computed on the 24 sectors that compose the STOXX 600 index. Our results show that several sectors have a high proclivity for generating spillovers. In general, the largest influences are delivered by Capital Goods, Banks, Diversified Financials, Insurance, and Real Estate. We also bring detailed evidence on the sectors that are the most pregnable to spillovers and on those that represent the main contributors of spillovers.

Impairments of facial detection in tobacco use disorder: baseline data and impact of smoking duration.

Objective:Chronic tobacco consumption, classified as tobacco use disorder (TUD), has been associated with a variety of health problems. Investigations of face processing in TUD are hampered by lack of evidence. Here, we evaluated facial detection in TUD and assessed test-retest reliability for a facial detection task.Methods:Participants were instructed to detect the orientation (either left or right) of a face when it was presented with a face/non-face pair on the monitor screen, using Bayesian entropy estimation. Bland-Altman analysis and intraclass correlation coefficients were used to test the reliability of the task. The general linear model and Bayesian statistics were then used to evaluate differences between TUD (n=48) and healthy controls (n=34).Results:The reliability of the task was high for the 96 stimuli presentations. Slower reaction times (p < 0.001) and lower discrimination index (p < 0.001) were observed in the TUD group than for healthy controls. Mediation analysis indicated direct effects of smoking duration on reaction time (p < 0.001) and discrimination index (p < 0.001).Conclusions:Overall, we observed high reliability of this task and reduction of facial detection in tobacco use disorder. We conclude our findings are significant for public health initiatives and call for follow-up studies.

Meta-heuristic algorithms for resource Management in Crisis Based on OWA approach

In crisis management, Threat Evaluation (TE) and Resource Allocation (RA) are two key components. To build an automated system in this area after modelling Threat Evaluation and Resource Allocation processes, solving these models and finding the optimal solution are further important issues. In this paper, Non-dominated Sorting Genetic Algorithm-II (NSGA-II) and Strength Pareto Evolutionary Algorithms (SPEA-II) are employed to solve a multi-objective multi-stage Resource Allocation problem. These Algorithms have been compared using normalized values of the objectives by generational distance, spread, hyper-volume, cardinality and actual computational times. It is found that the non-dominated solutions obtained by SPEA-II are better than NSGA-II both in terms of convergence and diversity but at the expense of computational time. Here, the fuzzy inference systems and the decision tree have been used to conduct threat evaluation process. Finally, Ordered Weighted Averaging (OWA) with maximum Bayesian entropy method for determining the operator weights has been used to pick the final choice among optimal options. We plan to use the proposed method in this paper for crisis management in Iranian Red Crescent organization during fire fighting. Two real studies have been done and results have been presented.

Image Recognition and Analysis of Intrauterine Residues Based on Deep Learning and Semi-Supervised Learning

Residual placenta is one of the common types of postpartum complications in clinical practice. Residual placenta is also the main and most common cause of late postpartum hemorrhage. This article proposes a spatial pyramid loop module, which solves the problem that the existing network structure cannot effectively extract the semantic information and category information in the image at the same time. The spatial pyramid structure is used to effectively extract the semantic information and category information. In addition, this article proposes to use cyclic convolutional network to realize the transfer function of information at different scales, and build it in the spatial pyramid structure to further strengthen the ability to extract semantic information and category information. This article proposes a feature fusion module to solve the impact of image classification network used in the base network in the existing network structure. The attention mechanism is used to achieve the effective fusion of high-dimensional features and low-dimensional features in the base network to reduce the influence of the base network, so as to better recover the recognition and prediction results. A semantic category loss function is proposed to supervise the categories of objects in images. This article builds it on the feature layer with the smallest scale, which not only increases the intermediate supervision to make the network fully converge, but also reduces the difficulty of extracting category information, and makes full use of the information transfer function of the cyclic convolutional network. This article introduces uncertainty information into the field of image segmentation to provide the accuracy of segmentation. For the purpose of uncertainty information, this article improves the network structure. At the same time for the image segmentation task, this article improves the Bayesian cross entropy loss function. The experiment verifies the necessity of improving the Bayesian crossover function in this article and the effectiveness of the conditional random field used in this article, and also verifies the effectiveness of the proposed semi-supervised learning method.

Bayesian entropy network for fusion of different types of information

A hybrid method for information fusion combining the maximum entropy (ME) method with the classical Bayesian network is proposed as the Bayesian-Entropy Network (BEN) in this paper. The key benefit of the proposed method is the capability to handle various types of information for classification and updating, such as classical point data, abstracted statistical information, and range data. The detailed derivation of the proposed is given and special focus is on the formulation of different types of information as constraints embedded in the entropy part. The Bayesian part is used to handle classical point observation data. Next, an adaptive algorithm is proposed to mitigate the impact of wrong information constraints on the final posterior distribution estimation. Following this, several examples are used to demonstrate the proposed methodology and application to engineering problems. It is shown that the proposed method is a generalized form of classical Bayesian method, and can take advantage of the extra information. This advantage is preferable in many engineering applications especially when the number of point observations is limited. Conclusions and future work are drawn based on the current study.

Extraction of Information from Crowdsourcing: Experimental Test Employing Bayesian, Maximum Likelihood, and Maximum Entropy Methods

A crowdsourcing experiment in which viewers (the “crowd”) of a British Broadcasting Corporation (BBC) television show submitted estimates of the number of coins in a tumbler was shown in an antecedent paper (Part 1) to follow a log-normal distribution ∧(m,s2). The coin-estimation experiment is an archetype of a broad class of image analysis and object counting problems suitable for solution by crowdsourcing. The objective of the current paper (Part 2) is to determine the location and scale parameters (m,s) of ∧(m,s2) by both Bayesian and maximum likelihood (ML) methods and to compare the results. One outcome of the analysis is the resolution, by means of Jeffreys’ rule, of questions regarding the appropriate Bayesian prior. It is shown that Bayesian and ML analyses lead to the same expression for the location parameter, but different expressions for the scale parameter, which become identical in the limit of an infinite sample size. A second outcome of the analysis concerns use of the sample mean as the measure of information of the crowd in applications where the distribution of responses is not sought or known. In the coin-estimation experiment, the sample mean was found to differ widely from the mean number of coins calculated from ∧(m,s2). This discordance raises critical questions concerning whether, and under what conditions, the sample mean provides a reliable measure of the information of the crowd. This paper resolves that problem by use of the principle of maximum entropy (PME). The PME yields a set of equations for finding the most probable distribution consistent with given prior information and only that information. If there is no solution to the PME equations for a specified sample mean and sample variance, then the sample mean is an unreliable statistic, since no measure can be assigned to its uncertainty. Parts 1 and 2 together demonstrate that the information content of crowdsourcing resides in the distribution of responses (very often log-normal in form), which can be obtained empirically or by appropriate modeling.

Modeling Information Content Via Dirichlet-Multinomial Regression Analysis

ABSTRACTShannon entropy is being increasingly used in biomedical research as an index of complexity and information content in sequences of symbols, e.g. languages, amino acid sequences, DNA methylation patterns and animal vocalizations. Yet, distributional properties of information entropy as a random variable have seldom been the object of study, leading to researchers mainly using linear models or simulation-based analytical approach to assess differences in information content, when entropy is measured repeatedly in different experimental conditions. Here a method to perform inference on entropy in such conditions is proposed. Building on results coming from studies in the field of Bayesian entropy estimation, a symmetric Dirichlet-multinomial regression model, able to deal efficiently with the issue of mean entropy estimation, is formulated. Through a simulation study the model is shown to outperform linear modeling in a vast range of scenarios and to have promising statistical properties. As a practical example, the method is applied to a data set coming from a real experiment on animal communication.