Maximum Information Entropy Research Articles

Automatic thresholding method using single information entropy under product transformation of order difference filter response

To automatically threshold images with unimodal, bimodal, multimodal or non-modal gray level distributions within a unified framework, an automatic thresholding method using single information entropy under the product transformation of order difference filter response is proposed. The proposed method first performs the product transformation of order difference filter response on an input image at different scales to obtain the product transformation image. Critical or non-critical pixels are labelled on each pixel of the binary images corresponding to different thresholds to construct a series of binary label images that are used for distinguishing critical or non-critical regions. A single information entropy is finally used for characterizing the information obtained from the product transformation image with the critical regions of different binary label images, and the threshold corresponding to maximum information entropy is selected as final threshold. The proposed method is compared with seven state-of-the-art segmentation methods. Experimental results on 12 synthetic images and 98 real-world images show that the average Matthews correlation coefficients of the proposed method reached 0.994 and 0.966 for the synthetic images and the real-world images, which outperform the second-best method by 52.4 % and 27.8 %, respectively. The proposed method has more robust segmentation adaptability to test images with different modalities, despite not offering an advantage in terms of computational efficiency.

Toward a Unified Understanding of Estimating Evapotranspiration: The Linkage Between Three Effective Parsimonious Models

AbstractThe maximum information entropy production model (MaxEnt), the relative humidity at equilibrium approach (ETRHEQ), and the Surface Flux Equilibrium model (SFE) are three recently developed models to estimate evapotranspiration. Although the connection between ETRHEQ and SFE is evident, no attempts have been made to investigate the congruence, distinctions, or potential complementarity between the two models and MaxEnt. Our mathematical analysis demonstrates that minimizing the vertical variance of RH in ETRHEQ is equivalent to minimizing the dissipation function of energy fluxes in MaxEnt, under the assumption of the same eddy diffusivity of heat and water vapor and with a specific expression for the ratio between the thermal inertia terms for H and LE. The connection between ETRHEQ, SFE, and MaxEnt is independent of Monin‐Obukhov similarity theory (MOST)’s extremum solution, and MOST's extreme solution can be viewed as equivalent to introducing a constant correction factor to account for atmospheric stability. While ETRHEQ and MaxEnt can be united within a single hydrometeorological framework, they diverge in their approaches to modeling evapotranspiration, particularly in how they address the roles of vegetation and land surface heterogeneity. More importantly, the unified framework suggests that turbulence fluxes within the atmospheric boundary layer adhere to the principles of maximum information entropy production. The way in which dissipation, along with its associated entropy production, is established using information entropy theory deviates from traditional thermodynamic entropy formulations. Exploring the connection between thermodynamic and information entropy and developing proper formulations of dissipation for energy fluxes presents an appealing avenue for prospective research.

A high-altitude geomagnetic matching area selection approach based on geomagnetic information entropy and geomagnetic direction entropy

Aiming at the obvious trend of the contours on the high-altitude geomagnetic map, this paper presents a method of high-altitude geomagnetic matching area selection combining geomagnetic information entropy and geomagnetic direction entropy based on the study of geomagnetic information entropy and geomagnetic direction entropy. The method first uses geomagnetic information entropy to select the region with rich geomagnetic information, and then determines the direction of the flight trajectory according to geomagnetic direction entropy to obtain the optimal matching trajectory. Finally, the method compares and analyzes the matching localization results of three flight trajectories in different directions on the geomagnetic map at an altitude of 30,000 m by using semi-physical simulation. The experimental results show that the flight navigation error along the trajectory with small information entropy is small, and its positioning error is 12.7% of the trajectory positioning error along the maximum information entropy direction. Selecting the flight trajectory according to the value of the geomagnetic direction entropy can greatly improve the precision and reliability of the geomagnetic matching localization. The method in this paper can provide a basis for path planning of the geomagnetic matching navigation.

Adaptive orthogonal semi-supervised feature selection with reliable label matrix learning

Feature selection (FS) can select features with high value from high-dimensional data as much as possible and reduce the dimensionality of the data to improve the performance of the machine learning model and enhance the generalization ability. In semi-supervised FS (SSFS), regression-based methods utilize the label matrix, and the quality of the label matrix can directly affect the performance of FS. Hence, the importance of the reliable label matrix is obvious. Orthogonal regression (OR) can retain more information in the subspace than least squares regression (LSR). Therefore, this paper introduces OR into SSFS, and then the label scaling technique is used to learn a reliable label matrix. Also, this study utilizes adaptive graph learning to exploit more structural information about the data. Two constraints, Frobenius-norm or maximum information entropy imposed on the similarity matrix and two adaptive orthogonal SSFS (AGLSOFS) methods with reliable label matrix learning are constructed. The impact of these two constraints on the construction of dynamic similarity graphs and FS results is discussed. Effective optimization algorithms for these two methods are based on the Alternating Direction Method of Multipliers (ADMM) and Generalized Power Iteration (GPI). Experiments are conducted on 15 benchmark datasets, and the results show that: (1) similarity graphs constructed using both original and projected data are more accurate; (2) both constraints are valid; (3) both of the methods of this paper perform well on most datasets; (4) OR performs better than LSR for FS; and (5) the scaling factor affects the convergence speed of the model.

Learning bayesian network parameters from limited data by integrating entropy and monotonicity

High-accuracy parameter learning in Bayesian Networks (BNs) is a key challenge in real-time decision support applications, particularly when the available data are limited. Prior/Expert knowledge was introduced to eliminate the drawbacks of insufficient information; however, this method is subjective. In this study, we explored the use of monotonicity constraints to control the causal relationships between the nodes and their parents-nodes in BNs and proposed a new learning algorithm called global domain monotonicity based on maximum information entropy (GDM-MIE), which was designed for parameter learning in uncertain discrete BNs with nonlinear equality constraints when only finite data are available. In the proposed algorithm, a class of monotonicity is encoded as a constraint on information entropy and the parameter learning problem is transformed into constraints among the node parameters based on a known network. Furthermore, we considered the parameters as uncertain entropy information and discussed the monotonicity among parameters in the global spatial domain, proving the accuracy of the logical relationships of the model, system reliability, and time complexity. Finally, the proposed method was validated using standard BNs, and its performance was analyzed by comparing the proposed method with the existing learning algorithms. The results showed that the proposed method is more accurate and has better Kullback–Leibler divergence. To revalidate the rationality of the proposed method, the Alarm and Asia networks were employed as special cases. The GDM-MIE was found to achieve the intended goal of observing the estimated parameters by closely approximating the original real parameters with a small sample size, indicating that the proposed algorithm can served as an efficient and feasible method for learning Bayesian parameter.

Entropy-metric estimation of the small data models with stochastic parameters

The formalization of dependencies between datasets, taking into account specific hypotheses about data properties, is a constantly relevant task, which is especially acute when it comes to small data. The aim of the study is to formalize the procedure for calculating optimal estimates of probability density functions of parameters of linear and nonlinear dynamic and static small data models, created taking into account specific hypotheses regarding the properties of the studied object. The research methodology includes probability theory and mathematical statistics, information theory, evaluation theory, and stochastic mathematical programming methods. The mathematical apparatus presented in the article is based on the principle of maximization of information entropy on sets determined as a result of a small number of censored measurements of “input" and “output" entities in the presence of noise. These data structures became the basis for the formalization of linear and nonlinear dynamic and static models of small data with stochastic parameters, which include both controlled and noise-oriented input and output measurement entities. For all variants of the above-mentioned small data models, the tasks of determining the optimal estimates of the probability density functions of the parameters were carried out. Formulated optimization problems are reduced to the forms canonical for the stochastic linear programming problem with probabilistic constraints.

Exploring the Path of Village Protection and Renewal Strategy Based on the Maximum Information Entropy Model in the Context of Rural Revitalisation

Abstract A comprehensive and systematic view and solution to the protection and renewal of traditional villages can help actively promote and implement the strategy of rural revitalization. This paper constructs a village protection and renewal evaluation system based on the maximum information entropy model, utilizes the entropy weight method to build the index weight evaluation matrix, and introduces the confidence factor to correct the index weights, realizing the combination of the system. Then, based on the linear weighting method to carry out a comprehensive assessment of village protection and renewal development, and based on the assessment results to formulate a personalized village protection and renewal strategy, and the feasibility analysis of the strategy with the village of T city. The results show that village protection and renewal in T city villages generally indicate an upward trend from 2016 to 2023, and the comprehensive evaluation index reached a peak of 0.3227 in 2023. In the case study, the linear regression equations fitted to the scattering points of the uptown residential area villages and the entire T city villages’ axes intersect with a difference in slopes of 14.5123. The angle of the regression line continues to increase in comparison with 2016, indicating that the role of traditional village protection planning is significant. Village protection planning plays an important role in verifying the effectiveness of the formulated village protection and renewal strategy, which can promote the development of village regional characteristics for the good.

Practical Teaching and Intellectualization of University Courses Based on the Maximum Information Entropy Model

Abstract Traditional teaching has become a thing of the past, the integration of technology and the classroom has brought about significant changes in education, and the role of information entropy in teaching has gradually manifested itself. In this paper, the behavioral time occupancy of the teaching process of university courses is viewed as a probability distribution event combined with the Lagrange multiplier method to solve the maximum entropy value in the distribution of practical teaching behavior. The practice depth value of classroom behavior can be calculated using the interaction behavior coefficient and interaction behavior entropy. Apply the information entropy to the practical teaching analysis of university courses, verify the practical effect of university courses from different sides, and carry out intelligent course construction according to the current situation of students’ learning behavior. The results show that the average score of class 3 in the practical analysis ability assessment is 4.021, which indicates that the practical analysis ability of students in the class meets the standard, and the students’ practical execution should be mainly cultivated in the later practical teaching. 67.1% of students believe that the intelligent classroom has improved their performance over the traditional classroom according to the intelligent course model.

Exploration and Application of Intelligent Service Mode of College Labor Union under the Spirit of the Third Plenary Session of the 20th CPC Central Committee

Abstract Under the guidance of the spirit of the Third Plenary Session of the Twentieth Central Committee of the Communist Party of China (CPC), colleges and universities have set off a boom in the construction of modernized information service systems. This paper relies on the importance and feasibility of the development of the intelligent service platform of the labor union in colleges and universities and preliminarily constructs an intelligent service platform that can meet the needs of the management of labor union work in colleges and universities. The platform adopts B/S architecture, uses the MVC pattern as a platform design plug-in, and uses the J2EE platform as the main technical platform for the development of the college union management information system. Through the information collaboration network to the union information modeling, with the help of the system’s SQL language function, through the ODBC database-driven service linking method, to ensure the safe and stable operation of the university trade union intelligent service platform. The test results show that under the IO-intensive load verification, the platform memory utilization rate is stable between 56.30~71.05%, the tuning strategy can be applied to the actual college union management scenario application, and the validity and versatility of the parameter tuning scheme have been verified. At the same time, the test results of platform information collaboration association strength show that the maximum information entropy of each associated node is 19.08, and the results can provide a basis for the construction of the information collaboration network of the university smart labor union.

Applying the Maximum Information Entropy Model to Optimize Innovative Strategies for Physical Education Management

Abstract Physical education holds a crucial role in the educational systems of schools at all levels, and innovating physical education management is essential for promoting its coordinated development. This paper uses the maximum information entropy model as the foundation and, with the goal of optimizing the quality of physical education management, establishes an evaluation index system for physical education management quality based on the generalization and summarization of traditional evaluation theory. For the established evaluation indexes, the entropy weight method in the maximum information entropy model is used to assign certain weights to the indexes at all levels in order to highlight the importance of different indexes in the evaluation system of physical education management quality. In the process of indicator screening, the mean value of the first, second, and third level indicators established in this paper is greater than or equal to 3.75, and the standard deviation and coefficient of variation also meet the design requirements of this paper, so there is no need to eliminate the indicators or other work. In the weighting analysis, the weight of the teaching process is 0.5136, which is the evaluation indicator with the highest weight among all the first-level indicators, while the weight of the professional education competence is the highest among the second-level indicators and all the third-level indicators are also given the corresponding weights. According to the results of the screened evaluation indexes and weights, this paper completes the establishment of the quality evaluation system of physical education management based on the maximum information entropy model and the formulation of evaluation standards, which provides strong support for the innovation of physical education management.

Geographical distribution and potential distribution prediction of thirteen species of Citrus L. in China.

Using DIVA-GIS software to study the spatial accumulation of Citrus species, an important economic crop in China. Draw the distribution maps of Citrus to concerning altitude and vegetation, and use DIVA-GIS' Bioclim ecological model and maximum information entropy model Maxent software to estimate the potential distribution areas of various Citrus species. The results show that the Citrus genus is located in the south of Qinling Mountains, mainly in the southwest of China and the coast of southeastern China. Sichuan and Chongqing are the most densely distributed regions of Citrus. The distribution of Citrus is closely related to the vegetation type and altitude. The vegetation types in the distribution area is evergreen broad-leaved forest, evergreen shrub, deciduous broad-leaved forest, mixed-leaf forest coverage area, deciduous coniferous forest, farmland, trees, other vegetation coverage, and evergreen coniferous forest. The current potential distribution area of Citrus is mainly in Guizhou, Hubei, Hunan, Guangdong, Guangxi, Yunnan,Sichuan, Chongqing, and other provinces and municipalities and their borders, while the potential distribution area in the future moves northward and enter Henan and southern Gansu. At the same time, climate warming changes the distribution of suitable areas of Citrus, which makes the suitable areas of C. sinensis Osbeck, C. reticulata Blanco, and other crops greatly increased. Planning the planting area will effectively improve the yield and quality in the future. Planning presents new challenges.

Informatization of Physical Education Curriculum Education and Training Management in Colleges and Universities under the Maximum Information Entropy Model

Abstract Currently, there are problems such as weak training purpose, poor training effect and backward auxiliary decision-making means in college physical education. In this paper, we first construct a college sports training management decision-making system based on Multi-Agent and adopt the blackboard model to realize the communication between Agents. The training management decision-making agent system uses the maximum information entropy model to identify the benefits and drawbacks of executing college sports training management decisions. Secondly, feature selection is carried out through the correlation filter (CFS), and the best priority search method is used to obtain the required subset of features. Finally, the reliability of the college sports training management decision-making agent system was verified by testing experiments. The results show that the inter-rater of the system is 0.92<unk>0.95, and the intra-rater is 0.94, reflecting the good reliability and practicality of the training management decision-making Agent system. This study enhances the effectiveness of sports training management to some extent, and it also serves as a valuable reference for the development of similar systems.

Application of computer geological model in geotechnical engineering investigation

The computer geological model is the approximate model of the actual complex geological phenomenon. During the modeling process, uncertainty from various sources can lead to deterioration of the quality of geological models. In order to ensure the usability of the model, it is necessary to consider the influence of spatial autocorrelation and interrelationship between strata on information transmission and uncertainty when analyzing and predicting multi-layer geological structures. The purpose of this paper is to improve the accuracy and testing breadth of computer modeling by studying geotechnical exploration activities. The calculation formula of information entropy of geological variables is given. At the same time, it introduces how to calculate the maximum information entropy of geological variables under the prior distribution according to the prior information constraints, and how to calculate the results of geological variables under the posterior distribution by using Bayesian updating. This paper introduces the calculation method of formation attribute probability field and iterates formation probability from bottom to top. The results show that the Copula method can improve the geological model in statistical sense. Finally, it is concluded that establishing 3D geological model by computer is helpful to improve the accuracy of geological exploration.

EarlGAN: An enhanced actor–critic reinforcement learning agent-driven GAN for de novo drug design

Deep generative models, such as Generative Adversarial Networks (GANs), have attracted the attention of researchers in the denovo drug design field. However, traditional GANs are typically used for image processing. Therefore, they are unsuitable for Simplified Molecular-Input Line-Entry System (SMILES) strings due to their discrete nature. Previous studies addressed this problem by combining Reinforcement Learning (RL) approaches with Monte Carlo tree search. However, for large chemical datasets, the molecule generation process is time-consuming due to the lengthy atom-by-atom sampling process with cumulative reward, an essence of the Monte Carlo tree search-based RL approaches. To address this problem, we propose an enhanced actor–critic RL agent-driven GAN, called EarlGAN, for denovo drug design. Specifically, EarlGAN’s generator acts as an actor to generate SMILES strings, and the discriminator acts as a critic to perform discrimination. EarlGAN makes autoregressive predictions at the atomic level. While the generator is based on previously generated atoms, the discriminator discriminates using a bidirectional pass over the atoms, including the current atom that is being predicted. We integrate moment, global-level discrimination rewards, and information entropy maximization. The moment rewards reduce the computation time, and the global-level rewards ensure the consistency of the molecule, whereas the information entropy maximization leads to a more diverse sample generation. Experiments and ablation studies verify the effectiveness of EarlGAN for denovo drug design on the QM9 and ZINC datasets. In addition, the visualization analysis provides insight into EarlGAN and supports our conclusion.

Simulation and analysis of red-crowned crane habitat suitability using maximum entropy and information entropy models

The red-crowned crane (Grus japonensis) is a rare waterfowl species that is sensitive to environmental changes in its habitat selection; as such it effectively reflects the variation in wetland landscapes. Therefore, analysis of the habitat suitability for the red-crowned crane can be used to determine the species requirements and to predict its potential habitat distribution, thereby providing a basis for the protection of wetlands and endangered species. In this study, the Landsat satellite remote sensing data of Zhalong Wetland from 1996 to 2019 were used to simulate the nest coordinates of red-crowned cranes using a binary logistic model and to simulate the distribution of suitable habitat via a maximum entropy model. Moreover, an information entropy model was used to examine the yearly changes in habitat suitability and the underlying mechanisms and influencing factors. These two entropy models were combined to simulate the habitat suitability distribution and variation with high accuracy, determine the contribution rates of influencing factors, and provide data for analyzing interannual changes. The following results were obtained. First, the habitat suitability was highest in the center, core area and decreased towards the periphery of Zhalong Wetland. Second, the habitat suitability increased in some regions in the south, decreased in some areas in the north, and remained unchanged in the majority of the peripheral regions. Third, habitat suitability was primarily impacted by landscape patterns, distances to rivers and ditches, and vegetation. Red-crowned cranes preferred to reside far from human disturbance in wetlands near rivers with reeds (Phragmites australis) as the dominant species.

Synergy Prediction Model of Information Entropy Based on Zone Safety Degree and Stope Roof Weighting Step Analysis

During the underground mining of coal resources, the rock pressure emerges acutely and the mine geological disasters occur frequently. It is of great significance to grasp the manifestation law of rock pressure in time to guide the safety production and operation in the pit. In this research, the calculation equations and concept of information entropy based on zone safety degree are primarily defined, and the synergetic theory of the maximum information entropy principle is combined to put forward the synergy prediction model of information entropy based on zone safety degree. In the meantime, the synergy prediction model of information entropy based on zone safety degree is employed to calculate and predict the first weighting step and the periodic weighting step of the main roof of the 9203 working face of Hengsheng Coal Mine in China’s Shanxi Province, as well as verifying the validity and reliability of the synergy prediction model of information entropy based on zone safety degree by the comparison of similar simulation test results, which has presented a scientific basis for the effective control of rock pressure and roof management.

What constrains food webs? A maximum entropy framework for predicting their structure with minimal biases.

Food webs are complex ecological networks whose structure is both ecologically and statistically constrained, with many network properties being correlated with each other. Despite the recognition of these invariable relationships in food webs, the use of the principle of maximum entropy (MaxEnt) in network ecology is still rare. This is surprising considering that MaxEnt is a statistical tool precisely designed for understanding and predicting many types of constrained systems. This principle asserts that the least-biased probability distribution of a system's property, constrained by prior knowledge about that system, is the one with maximum information entropy. MaxEnt has been proven useful in many ecological modeling problems, but its application in food webs and other ecological networks is limited. Here we show how MaxEnt can be used to derive many food-web properties both analytically and heuristically. First, we show how the joint degree distribution (the joint probability distribution of the numbers of prey and predators for each species in the network) can be derived analytically using the number of species and the number of interactions in food webs. Second, we present a heuristic and flexible approach of finding a network's adjacency matrix (the network's representation in matrix format) based on simulated annealing and SVD entropy. We built two heuristic models using the connectance and the joint degree sequence as statistical constraints, respectively. We compared both models' predictions against corresponding null and neutral models commonly used in network ecology using open access data of terrestrial and aquatic food webs sampled globally (N = 257). We found that the heuristic model constrained by the joint degree sequence was a good predictor of many measures of food-web structure, especially the nestedness and motifs distribution. Specifically, our results suggest that the structure of terrestrial and aquatic food webs is mainly driven by their joint degree distribution.

Synergetic Theory of Information Entropy Based on Failure Approach Index for Stability Analysis of Surrounding Rock System.

It is generally acknowledged that the stability evaluation of surrounding rock denotes nonlinear complex system engineering. In order to accurately and quantitatively assess the safety states of surrounding rock and provide a scientific basis for the prevention and control of surrounding rock stability, the analysis method of the synergetic theory of information entropy using the failure approach index has been proposed. By means of deriving the general relationship between the total two-dimensional plastic shear strain and the total three-dimensional plastic shear strain and obtaining the numerical limit analysis step of the plastic shear strain, the threshold value of the ultimate plastic shear strain can be determined, which has provided the key criterion for the calculation of the information entropy based on the failure approach index. In addition, combining with the synergetic theory of the principle of maximum information entropy, the evolution equation of the excavation step and information entropy based on the failure approach index of the surrounding rock system in underground mining space are established, and the equations of the general solution and particular solution as well as the expression of the destabilizing excavation step are given. To account for this, the method is applied to analyze the failure states of the floor surrounding rock after the mining of the 71 coal seam in Xutuan Coal Mine and involve the disturbance effect and stability control method of the underlying 72 coal seam roof from the macroscopic and microscopic aspects. Consequently, the validity of the analysis method of synergetic theory of information entropy based on the failure approach index has been verified, which presents an updated approach for the stability evaluation of surrounding rock systems that is of satisfactory capability and value in engineering applications.

Sparse Compression-Based Image Encryption using Data Encryption Standards RC5

In this work, we have used the standard symmetric key block cipher data encryption algorithm RC5(32,16,8) for the encryption of digital greyscale images. For this, we have only considered the nonzero elements of the sparse matrix of the images which have been generated using the sparse representation technique. The security strength of the proposed technique is verified through different quality parameters, e.g. information entropy, correlation coefficients, NPCR, UACI, and NIST test. Our proposed algorithm achieved the maximum information entropy value of 7.9977 for the cameraman image and nearly zero correlation coefficient values establish the robustness of the cryptosystem. The highest NPCR and UACI values obtained in our work are 99.6916 and 33.9605, which are closer to the maximum theoretical values. The proposed technique is also compared with some of the contemporary works to validate the credibility of our proposed algorithm.

Region-Aware Hierarchical Latent Feature Representation Learning-Guided Clustering for Hyperspectral Band Selection.

Hyperspectral band selection aims to identify an optimal subset of bands for hyperspectral images (HSIs). For most existing clustering-based band selection methods, they directly stretch each band into a single feature vector and employ the pixelwise features to address band redundancy. In this way, they do not take full consideration of the spatial information and deal with the importance of different regions in HSIs, which leads to a nonoptimal selection. To address these issues, a region-aware hierarchical latent feature representation learning-guided clustering (HLFC) method is proposed. Specifically, in order to fully preserve the spatial information of HSIs, the superpixel segmentation algorithm is adopted to segment HSIs into multiple regions first. For each segmented region, the similarity graph is constructed to reflect the bands-wise similarity, and its corresponding Laplacian matrix is generated for learning low-dimensional latent features in a hierarchical way. All latent features are then fused to form a unified feature representation of HSIs. Finally, k -means clustering is utilized on the unified feature representation matrix to generate multiple clusters from which the band with maximum information entropy is selected to form the final subset of bands. Extensive experimental results demonstrate that the proposed clustering method can achieve superior performance than the state-of-the-art representative methods on the band selection. The demo code of this work is publicly available at https://github.com/WangJun2023/HLFC.