Entropy Estimation Method Research Articles

Identifying Significant Cost-Influencing Factors for Sustainable Development in Construction Industry Using Structural Equation Modelling

Construction projects are typically conducted in a complex and dynamic environment in which the accumulation of many interrelated factors causes high uncertainty. Many factors may have negative effects on construction projects and thus cause project cost overruns. The aim of this study is to identify significant cost-influencing factors for sustainable development in construction. Considering the effects of economic and environmental factors on project cost is indeed a challenging task due to the shortage of appropriate methodology. This study examines the relationships between cost-influencing factors and construction project cost by using a structural equation model based on the generalized maximum entropy (GME) and Bayesian estimation methods. The advantages of GME and Bayesian methods are discussed, and results obtained from the statistical analysis are provided for illustration. The results can enhance cost performance through the identification and evaluation of the cost-influencing factors. The improved project performance is considered as an important step to transit into a sustainable development in construction. The sustainable development may greatly affect the emission reduction target of the industry.

A fuzzy entropy approach for design of hydrometric monitoring networks

This paper proposes a fuzzy entropy-based methodology for optimal design and performance assessment of a hydrometric monitoring network that provides information essential for hydrological studies. The methodology does not require choice of bin size for discretization of data to estimate entropy measures/indices. Therefore, it alleviates the uncertainty associated with bin size estimation which is a concern in analysis with conventional Shannon entropy-based methodology (SEBM). Advantage of the proposed methodology over SEBM and its related theoretical improvement EEBM (exponential entropy-based methodology) in arriving at optimal design of streamflow monitoring network is demonstrated through performance investigation on Mahanadi basin of India, which is frequently prone to floods. Following this, the methodology is used to assess the performance of streamflow monitoring networks in 16 river basins of Peninsular India and identify stream gauge deficit zones. This study is first of its kind which evaluates the effect of different entropy estimation methods and bin size estimation rules on: (i) prioritization of stream gauge stations for their importance in data collection, (ii) identification of stream gauge deficit zones, and (iii) design of optimal hydrometric network in a river basin. Proposed methodology appears promising and offers scope for application to networks monitoring various other hydrometeorological variables.

High-Efficiency Min-Entropy Estimation Based on Neural Network for Random Number Generators

Random number generator (RNG) is a fundamental and important cryptographic element, which has made an outstanding contribution to guaranteeing the network and communication security of cryptographic applications in the Internet age. In reality, if the random number used cannot provide sufficient randomness (unpredictability) as expected, these cryptographic applications are vulnerable to security threats and cause system crashes. Min-entropy is one of the approaches that are usually employed to quantify the unpredictability. The NIST Special Publication 800-90B adopts the concept of min-entropy in the design of its statistical entropy estimation methods, and the predictive model-based estimators added in the second draft of this standard effectively improve the overall capability of the test suite. However, these predictors have problems on limited application scope and high computational complexity, e.g., they have shortfalls in evaluating random numbers with long dependence and multivariate due to the huge time complexity (i.e., high-order polynomial time complexity). Fortunately, there has been increasing attention to using neural networks to model and forecast time series, and random numbers are also a type of time series. In our work, we propose several new and efficient approaches for min-entropy estimation by using neural network technologies and design a novel execution strategy for the proposed entropy estimation to make it applicable to the validation of both stationary and nonstationary sources. Compared with the 90B’s predictors officially published in 2018, the experimental results on various simulated and real-world data sources demonstrate that our predictors have a better performance on the accuracy, scope of applicability, and execution efficiency. The average execution efficiency of our predictors can be up to 10 times higher than that of the 90B’s for 10 6 sample size with different sample spaces. Furthermore, when the sample space is over 2 2 and the sample size is over 10 8 , the 90B’s predictors cannot give estimated results. Instead, our predictors can still provide accurate results. Copyright© 2019 John Wiley & Sons, Ltd.

A Trusted Resource-Based Routing Algorithm With Entropy Estimation in Integrated Space-Terrestrial Network

Integrated Space-Terrestrial Network (ISTN) can effectively improve the global communication coverage and SDN framework in ISTN has emerged as a promising solution to guarantee the Quality of Service (QoS). At present, the researches on routing algorithm are mainly focused on the QoS parameters about resource availability and do not consider the impact of the network attacks comprehensively, such as DDoS attack. Plenty of available resources will be occupied by network attacks and normal traffic may be blocked incorrectly by attack defense solutions. In this paper, we propose a Trusted Routing (TR) model based on Trusted Resource Matrix (TRM) to protect the normal traffic from network attacks in ISTN. In the process, the SDN controller constructs the characteristic matrices by collecting the state information of each routing node and the entropy estimation method which describes the changes of traffic characteristics is used to determine whether a routing node is suffering from network attacks. Corresponding trust values are allocated to the routing nodes for constructing TRM and trusted transmission paths are discovered by the TR model. Based on the comprehensive consideration of available and trusted resources, a Hybrid Routing (HR) model is proposed, taking into account the traditional QoS parameters and TRM. In the simulation experiments, the TR and HR models-based algorithms are evaluated. The experimental results show that TR and HR algorithms can effectively avoid the routing nodes suffering from network attacks and improve the communication QoS. When considering a scenario of a single attacked node under resource constraints, the average packet pass rate of HR algorithm is about 7.0% higher than that of TR algorithm and about 52.1% higher than that of traditional QoS algorithm.

Entropy Estimation of Inverse Weibull Distribution under Adaptive Type-II Progressive Hybrid Censoring Schemes

This paper discusses entropy estimations for two-parameter inverse Weibull distributions under adaptive type-II progressive hybrid censoring schemes. Estimations of entropy derived by maximum likelihood estimation method and Bayes estimation method are both considered. Different Bayes estimators using squared loss function, Linex loss function, general entropy loss function, and balanced loss function are derived. Numerical results are derived by Lindley’s approximation method. Especially, the interval estimation of entropy is derived through maximum likelihood estimation method. To test the effectiveness of the estimations, simulation studies are conducted. These entropy estimation methods are illustrated and applied to analyze a real data set.

On the Analysis and Improvement of Min-Entropy Estimation on Time-Varying Data

Widely used as fundamental security components in most cryptographic applications, random number generators (RNGs) rely mainly on randomness provided by entropy sources. If the provided randomness is less than expected, RNGs may be compromised and thus impair the security of the whole cryptographic applications. However, the common assumptions (e.g., outputs are independent and identically distributed, i.e., IID) may not always hold. For example, many entropy sources are based on some physical phenomena that are fragile and sensitive to external factors (e.g., temperature), which means the distributions of these entropy sources’ outputs are continuously changing. As important tools to measure the quality of entropy sources, existing entropy estimation methods may provide false estimations against these time-varying data, because they cannot detect the changes of data distributions. In this paper, we firstly review and analyze the existing typical entropy estimators including the NIST SP 800-90B (90B for short) estimators and the lately proposed neural network based (NN-based) estimators, especially, their limitations on the aforementioned time-varying data. Second, we propose an entropy estimation framework adopting change detection techniques to address this problem. In contrast to the NN-based estimators, the proposed estimator under this framework employs a change detection method to preprocess the tested data and adds additional distribution features to each data sample, which makes it possible to learn the distribution changes and estimate the entropy more accurately. Finally, we evaluate the performance of our estimator using various kinds of simulated data and real world data, and compare our estimator with the 90B estimators and the NN-based estimators. Extensive evaluations demonstrate that the proposed estimator provides similar or more accurate entropy estimation than the other estimators, especially for time-varying data.

On the feasibility of deriving cryptographic keys from MEMS sensors

One of the main challenges in the internet of things (IoT) will be to guarantee the security of products and services enabled by it. A fundamental assumption in any cryptosystem is that secret-key material remains securely stored and unknown to attackers. To this end, physical unclonable functions have been proposed to store cryptographic secrets without the need to use non-volatile memory. In this work, we show that microelectromechanical systems (MEMS) sensors, ubiquitous in the IoT, can be used to generate a stable nearly fully entropic bit string that can be used as a secret or private key in a cryptographic algorithm. We provide experimental evidence of the stability of our methods by analyzing data from 468 off-the-shelf 3-axis MEMS gyroscopes subjected to different temperatures in the range typically required for consumer applications and standardized aging tests. The investigations are carried out on module level so that packaging influences are considered. We derive unique fingerprints from the sensors based on their characteristics, and we show that the false rejection rate (FRR) and the false acceptance rate (FAR) are below $$1 \times 10^{-6}$$ for all applied test conditions. By adding up the values of FRR and the FAR, the highest probability for an authentication error is $$4.1 \times 10^{-6}$$. Furthermore, we extract stable keys from the fingerprints. The extracted key length lies in a range between 27 and 150 bits depending on the applied test conditions and the used entropy estimation method.

Analyzing Regional Agricultural Production Responses using a Positive Mathematical Programming Method

This study constructs regional crop production models of 8 cities in Korea using the PMP Leontief-quadratic and the generalized-CES models suggested by Mérel and Bucaram (2010) and Mérel et al. (2011). A generalized maximum entropy estimation method was used to derive the set of supply elasticity estimates allowing a complete calibration of the models. We demonstrate that those two PMP models fit the Korean regional data quite well. The impacts of two policy changes, raising the rice producer price under the rice direct payment system and raising the minimum wage for hired rural labor were simulated. Our simulation results show that the policy impacts are varying over region and crop. Moreover, it is shown that the choice of the functional specification substantially affects the direction and the magnitudes of the policy impacts. More specifically, production responses are more flexible in the CES specification rather than in the Loentief specification, and the former specification suggests that a substantial increase in rural minimum wage may result in a decline in capital and labor employment in rural areas while the latter specification suggests that the overall resource use in rural areas will not be affected by the policy.

Entropic Stabilization of Cas4 Protein SSO0001 Predicted with Popcoen.

Popcoen is a method for configurational entropy estimation of proteins based on machine-learning. Entropy is predicted with an artificial neural network which was trained on simulation trajectories of a large set of representative proteins. Popcoen is extremely fast compared to other approaches based on the sampling of a multitude of microstates. Consequently, Popcoen can be incorporated into a large class of protein software which currently neglects configurational entropy for performance reasons. Here, we apply Popcoen to various conformations of the Cas4 protein SSO0001 of Sulfolobus solfataricus, a protein that assembles to a decamer of known toroidal shape. We provide numerical evidence that the native state (NAT) of a SSO0001 monomer has a similar structure to the protomers of the oligomer, where NAT of the monomer is stabilized mainly entropically. Due to its large amount of configurational entropy, NAT has lower free energy than alternative conformations of very low enthalpy and solvation free-energy. Hence, SSO0001 serves as an example case where neglecting configurational entropy leads to incorrect conclusion. Our results imply that no refolding of the subunits is required during oligomerization which suggests that configurational entropy is employed by nature to largely enhance the rate of assembly.

Univariate and multivariate conditional entropy measures for the characterization of short-term cardiovascular complexity under physiological stress

Objective: A defining feature of physiological systems under the neuroautonomic regulation is their dynamical complexity. The most common approach to assess physiological complexity from short-term recordings, i.e. to compute the rate of entropy generation of an individual system by means of measures of conditional entropy (CE), does not consider that complexity may change when the investigated system is part of a network of physiological interactions. This study aims at extending the concept of short-term complexity towards the perspective of network physiology, defining multivariate CE measures whereby multiple physiological processes are accounted for in the computation of entropy rates. Approach: Univariate and multivariate CE measures are computed using state-of-the-art methods for entropy estimation and applied to time series of heart period (H), systolic (S) and diastolic (D) arterial pressure, and respiration (R) variability measured in healthy subjects monitored in a resting state and during conditions of postural and mental stress. Main results: Compared with the traditional univariate metric of short-term complexity, multivariate measures provide additional information with plausible physiological interpretation, such as (i) the dampening of respiratory sinus arrhythmia and activation of the baroreflex control during postural stress; (ii) the increased complexity of heart period and blood pressure variability during mental stress, reflecting the effect of respiratory influences and upper cortical centers; (iii) the strong influence of D on S, mediated by left ventricular ejection fraction and vascular properties; (iv) the role of H in reducing the complexity of D, related to cardiac run-off effects; and (v) the unidirectional role of R in influencing cardiovascular variability. Significance: Our results document the importance of employing a network perspective in the evaluation of the short-term complexity of cardiovascular and respiratory dynamics across different physiological states.

A Deterministic Sensor Deployment Method for Target Coverage

In order to monitor the gas leakage, the gas sensors are deployed conventionally in chemical industry park, with little considerations given to the gas characteristics and weather conditions, which give rise to the problems of coverage hole and coverage repetition. To solve the problems, this paper proposes a deterministic sensor deployment method with the gas diffusion models which takes into account wind speed and direction and then studies the influence of wind speed and direction on the monitoring error of gas sensors. Then, we research the deterministic deployment method of gas sensors in condition of the main wind speed and direction somewhere. Firstly, we use the CFD theory to simulate the gas diffusion situation so as to obtain the concentration value of the relevant points. Secondly, we put forward a new optimization criterion, namely, the more alarm concentration points covered by gas sensors, the coverage performance is better, and the deployment method is better. Accordingly, a new objection function is built. Thirdly, we obtain the weight values of the function using entropy estimation method. Finally, we deploy the gas sensors determinately using particle swarm optimization (PSO) algorithm. The simulation results show that the proposed method can improve the monitoring efficiency and the coverage performance of gas sensor network.

A comparison of generalised maximum entropy and ordinary least square

The generalised maximum entropy (GME) estimation method is based on the classic maximum entropy approach of Jaynes (1957). It has the ability to estimate the parameters of a regression model without imposing any constraints on the probability distribution of errors and it is robust even when we have ill-posed problems. In this paper, we simulate two sets of data from regression model with different distribution for disturbance, standard normal and Cauchy distributions respectively. For this dataset, regression coefficients are obtained by GME and OLS methods and these techniques are compared with each other for some sample sizes. Moreover, we have used some prior information on parameters to obtain GME estimators. The estimation results of GME in the case of non-normal distributed are discussed here.

Generalized Maximum Entropy Estimation of Linear Models

In this article, we describe the user-written gmentropylinear command, which implements the generalized maximum entropy estimation method for linear models. This is an information-theoretic procedure preferable to its maximum likelihood counterparts in many applications; it avoids making distributional assumptions, works well when the sample is small or covariates are highly correlated, and is more efficient than its maximum likelihood equivalent. We give a brief introduction to the generalized maximum entropy procedure, present the gmentropylinear command, and give an example using the command.

Verification of Three-Phase Dependency Analysis Bayesian Network Learning Method for Maize Carotenoid Gene Mining.

Background and Objective Mining the genes related to maize carotenoid components is important to improve the carotenoid content and the quality of maize. Methods On the basis of using the entropy estimation method with Gaussian kernel probability density estimator, we use the three-phase dependency analysis (TPDA) Bayesian network structure learning method to construct the network of maize gene and carotenoid components traits. Results In the case of using two discretization methods and setting different discretization values, we compare the learning effect and efficiency of 10 kinds of Bayesian network structure learning methods. The method is verified and analyzed on the maize dataset of global germplasm collection with 527 elite inbred lines. Conclusions The result confirmed the effectiveness of the TPDA method, which outperforms significantly another 9 kinds of Bayesian network learning methods. It is an efficient method of mining genes for maize carotenoid components traits. The parameters obtained by experiments will help carry out practical gene mining effectively in the future.

Graph Similarity through Entropic Manifold Alignment

In this paper we decouple the problem of measuring graph similarity into two sequential steps. The first step is the linearization of the quadratic assignment problem (QAP) in a low-dimensional space, given by the embedding trick. The second step is the evaluation of an information-theoretic distributional measure, which relies on deformable manifold alignment. The proposed measure is a normalized conditional entropy, which induces a positive definite kernel when symmetrized. We use bypass entropy estimation methods to compute an approximation of the normalized conditional entropy. Our approach, which is purely topological (i.e., it does not rely on node or edge attributes although it can potentially accommodate them as additional sources of information) is competitive with state-of-the-art graph matching algorithms as sources of correspondence-based graph similarity, but its complexity is linear instead of cubic (although the complexity of the similarity measure is quadratic). We also determine that the ...

Theoretical evaluation of feature selection methods based on mutual information

Feature selection methods are usually evaluated by wrapping specific classifiers and datasets in the evaluation process, resulting very often in unfair comparisons between methods. In this work, we develop a theoretical framework that allows obtaining the true feature ordering of two-dimensional sequential forward feature selection methods based on mutual information, which is independent of entropy or mutual information estimation methods, classifiers, or datasets, and leads to an undoubtful comparison of the methods. Moreover, the theoretical framework unveils problems intrinsic to some methods that are otherwise difficult to detect, namely inconsistencies in the construction of the objective function used to select the candidate features, due to various types of indeterminations and to the possibility of the entropy of continuous random variables taking null and negative values.

Estimation for the multi-way error components model with ill-conditioned panel data

Ill-posed problems resulting from limited, partial or incomplete sample information have occurred frequently in econometric practice. The traditional methods of information recovery may cause the estimates highly unstable with low precision, known as ill-conditioned problems. In this paper, we propose a dual generalized maximum entropy estimator for the multi-way error components model with ill-conditioned panel data, based on an unconstrained dual Lagrange multiplier method. The numerical results for the panel data regression model with highly correlated and endogeneous covariates are in favor of our dual generalized maximum entropy estimation method in terms of quality of estimates.

A Transfer Entropy Method to Quantify Causality in Stochastic Nonlinear Systems

In modern chemical processes, identification of the process variable connectivity and their topology is vital for maintaining the operational safety. As a general information theoretic method, transfer entropy can analyze the causality between two variables based on estimation of conditional probability density functions. Transfer entropy estimation is typically a data driven method, however, the associated high computational complexity and poor accuracy are not acceptable in real applications. Using a nonlinear stochastic state-space model in conjunction with particle filters, a novel transfer entropy estimation method is proposed. The proposed approach requires less data, is fast and accurate.

A Generalized Maximum Entropy (GME) estimation approach to fuzzy regression model

Fuzzy statistics provides useful techniques for handling real situations which are affected by vagueness and imprecision. Several fuzzy statistical techniques (e.g., fuzzy regression, fuzzy principal component analysis, fuzzy clustering) have been developed over the years. Among these, fuzzy regression can be considered an important tool for modeling the relation between a dependent variable and a set of independent variables in order to evaluate how the independent variables explain the empirical data which are modeled through the regression system. In general, the standard fuzzy least squares method has been used in these situations. However, several applicative contexts, such as for example, analysis with small samples and short and fat matrices, violation of distributional assumptions, matrices affected by multicollinearity (ill-posed problems), may show more complex situations which cannot successfully be solved by the fuzzy least squares. In all these cases, different estimation methods should instead be preferred. In this paper we address the problem of estimating fuzzy regression models characterized by ill-posed features. We introduce a novel fuzzy regression framework based on the Generalized Maximum Entropy (GME) estimation method. Finally, in order to better highlight some characteristics of the proposed method, we perform two Monte Carlo experiments and we analyze a real case study.

Feature selection method based on mutual information and class separability for dimension reduction in multidimensional time series for clinical data

Abstract In clinical medicine, multidimensional time series data can be used to find the rules of disease progress by data mining technology, such as classification and prediction. However, in multidimensional time series data mining problems, the excessive data dimension causes the inaccuracy of probability density distribution to increase the computational complexity. Besides, information redundancy and irrelevant features may lead to high computational complexity and over-fitting problems. The combination of these two factors can reduce the classification performance. To reduce computational complexity and to eliminate information redundancies and irrelevant features, we improved upon a multidimensional time series feature selection method to achieve dimension reduction. The improved method selects features through the combination of the Kozachenko–Leonenko (K–L) information entropy estimation method for feature extraction based on mutual information and the feature selection algorithm based on class separability. We performed experiments on the Electroencephalogram (EEG) dataset for verification and the non-small cell lung cancer (NSCLC) clinical dataset for application. The results show that with the comparison of CLeVer, Corona and AGV, respectively, the improved method can effectively reduce the dimensions of multidimensional time series for clinical data.