Concept Of Information Entropy Research Articles

Entropy-based syntactic tree analysis for text classification: a novel approach to distinguishing between original and translated Chinese texts

Abstract This research focuses on classifying translated and non-translated Chinese texts by analyzing syntactic rule features, using an integrated approach of machine learning and entropy analysis. The methodology employs information entropy to gauge the complexity of syntactic rules in both text types. The methodology is based on the concept of information entropy, which serves as a quantitative measure for the complexity inherent in syntactic rules as manifested from tree-based annotations. The goal of the study is to explore whether translated Chinese texts demonstrate syntactic characteristics that are significantly different from those of non-translated texts, thereby permitting a reliable classification between the two. To do this, the research calculates information entropy values for syntactic rules in two comparable corpora, one of translated and the other of non-translated Chinese texts. Then, various machine learning models are applied to these entropy metrics to identify any significant differences between the two groups. The results show significant differences in the syntactic structures. Translated texts have a higher degree of entropy, indicating more complex syntactic constructs compared to non-translated texts. These findings contribute to our understanding of the effect of translation on language syntax, with implications for text classification and translation studies.

Leveraging the concept of information entropy to improve a multi-fidelity design framework for early-stage design exploration of complex vessels

Early-stage design exploration is crucial since most of the major design decision are locked-in and only small design modifications are possible at later stages. To assess the performance of the various design candidates while performing design exploration, there are available methods and tools of various fidelities. These methods can be combined to form a multi-fidelity (MF) framework that guarantees accuracy through the high-fidelity model and achieves faster computational speeds through low-fidelity models. The present study proposes the adoption of information-theoretic entropy to improve a MF design framework based on Gaussian Processes (GPs). Entropy quantifies the uncertainty associated with the prediction of the design space. We propose using this uncertainty metric both as a criterion to determine whether further designs should be sampled to construct a reliable approximation of the design space and as a criterion to establish in which optimization step the optimization of the covariance matrix for the MF-GPs should be performed. The approach was tested to benchmark analytical functions and to a ship design problem of an AXEfrigate. The approach holds potential in practical applications, as it aids in the determination of whether additional resources should be allocated for high-fidelity analysis to support early-stage exploration.

Research on a Framework for Chinese Argot Recognition and Interpretation by Integrating Improved MECT Models.

In underground industries, practitioners frequently employ argots to communicate discreetly and evade surveillance by investigative agencies. Proposing an innovative approach using word vectors and large language models, we aim to decipher and understand the myriad of argots in these industries, providing crucial technical support for law enforcement to detect and combat illicit activities. Specifically, positional differences in semantic space distinguish argots, and pre-trained language models' corpora are crucial for interpreting them. Expanding on these concepts, the article assesses the semantic coherence of word vectors in the semantic space based on the concept of information entropy. Simultaneously, we devised a labeled argot dataset, MNGG, and developed an argot recognition framework named CSRMECT, along with an argot interpretation framework called LLMResolve. These frameworks leverage the MECT model, the large language model, prompt engineering, and the DBSCAN clustering algorithm. Experimental results demonstrate that the CSRMECT framework outperforms the current optimal model by 10% in terms of the F1 value for argot recognition on the MNGG dataset, while the LLMResolve framework achieves a 4% higher accuracy in interpretation compared to the current optimal model.The related experiments undertaken also indicate a potential correlation between vector information entropy and model performance.

Discrete Entropies of Chebyshev Polynomials

Because of its flexibility and multiple meanings, the concept of information entropy in its continuous or discrete form has proven to be very relevant in numerous scientific branches. For example, it is used as a measure of disorder in thermodynamics, as a measure of uncertainty in statistical mechanics as well as in classical and quantum information science, as a measure of diversity in ecological structures and as a criterion for the classification of races and species in population dynamics. Orthogonal polynomials are a useful tool in solving and interpreting differential equations. Lately, this subject has been intensively studied in many areas. For example, in statistics, by using orthogonal polynomials to fit the desired model to the data, we are able to eliminate collinearity and to seek the same information as simple polynomials. In this paper, we consider the Tsallis, Kaniadakis and Varma entropies of Chebyshev polynomials of the first kind and obtain asymptotic expansions. In the particular case of quadratic entropies, there are given concrete computations.

Disentangling Administration Errors From Scale Development Errors in Survey Research

In a recent manuscript, Friesner, et al. (2023) used the concept of information entropy to assess the quantity of information in survey responses. They demonstrate how assessments of the quantity of information can be used to identify possible errors in a survey’s administration. A major limitation of their methodology is that it assumes the survey items used to elicit consumer preferences were created appropriately and contained a meaningful quantity of information. The current study addresses this limitation by incorporating a methodology developed by Friesner et al. (2021) into the Friesner et al. (2023) methodology. The combined methodology is applied to the same data studied in both Friesner et al. (2021) and Friesner et al. (2023), which allows for a direct comparison of the quantity of information gained/lost from survey administration versus scale development. The results indicate that the survey used in the empirical application exhibits flaws in both scale design and survey administration.

Variable Gap MAG Welding Penetration Control Using Rough Set Knowledge Acquisition

The establishment of the correspondence between the important features of the topside weld pool and the penetration state is a key link in penetration control. This article presents a scheme of variable gap welding penetration control based on rough-fuzzy modeling. By utilizing the basic concepts of variable precision rough sets and conditional information entropy, relevant algorithms for attribute reduction and rule extraction in decision information systems are addressed. Raw data for modeling are obtained through current–gap combination experiments, and then rough-set-based knowledge acquisition algorithms are used to generate classification rules for the penetration state. Based on the correspondence between the important features of the topside pool and the penetration state, a fuzzy control model is established. Welding experiments under variable gaps are conducted to examine the proposed control scheme, and the results indicate that the weld has good penetration and meets the requirements of welding specifications.

Analysis of Controllability in Cyber–Physical Power Systems under a Novel Load-Capacity Model

In cyber–physical power systems (CPPSs), system collapse can occur as a result of a failure in a particular component. In this paper, an approach is presented to build the load-capacity model of CPPSs using the concept of electrical betweenness and information entropy, which takes into account real-time node loads and the allocation of power and information flows within CPPSs. By introducing an innovative load redistribution strategy and comparing it with conventional load distribution strategies, the superior effectiveness of the proposed strategy in minimizing system failures and averting system collapses has been demonstrated. The controllability of the system after cascading failures under different coupling strategies and capacity parameters is investigated through the analysis of different information network topologies and network parameters. It was observed that CPPSs constructed using small-world networks, which couple high-degree nodes from the information network to high-betweenness nodes from the power grid, exhibit improved resilience. Furthermore, increasing the capacity parameter of the power network yields more favorable results compared to increasing the capacity parameter of the information network. In addition, our research results are validated using the IEEE 39-node system and the Chinese 132-node system.

Uncertainty Reduction and Environmental Justice in Air Pollution Epidemiology: The Importance of Minority Representation.

Ambient air pollution is an increasing threat to society, with rising numbers of adverse outcomes and exposure inequalities worldwide. Reducing uncertainty in health outcomes models and exposure disparity studies is therefore essential to develop policies effective in protecting the most affected places and populations. This study uses the concept of information entropy to study tradeoffs in mortality uncertainty reduction from increasing input data of air pollution versus health outcomes. We study a case scenario for short-term mortality from particulate matter (PM2.5) in North Carolina for 2001-2016, employing a case-crossover design with inputs from an individual-level mortality data set and high-resolution gridded data sets of PM2.5 and weather covariates. We find a significant association between mortality and PM2.5, and the information tradeoffs indicate that a 10% increase in mortality information reduces model uncertainty three times more than increased resolution of the air pollution model from 12 to 1km. We also find that Non-Hispanic Black (NHB) residents tend to live in relatively more polluted census tracts, and that the mean PM2.5 for NHB cases in the mortality model is significantly higher than that of Non-Hispanic White cases. The distinct distribution of PM2.5 for NHB cases results in a relatively higher information value, and therefore faster uncertainty reduction, for new NHB cases introduced into the mortality model. This newfound influence of exposure disparities in the rate of uncertainty reduction highlights the importance of minority representation in environmental research as a quantitative advantage to produce more confident estimates of the true effects of environmental pollution.

Synergy Prediction Model of Information Entropy Based on Zone Safety Degree and Stope Roof Weighting Step Analysis

During the underground mining of coal resources, the rock pressure emerges acutely and the mine geological disasters occur frequently. It is of great significance to grasp the manifestation law of rock pressure in time to guide the safety production and operation in the pit. In this research, the calculation equations and concept of information entropy based on zone safety degree are primarily defined, and the synergetic theory of the maximum information entropy principle is combined to put forward the synergy prediction model of information entropy based on zone safety degree. In the meantime, the synergy prediction model of information entropy based on zone safety degree is employed to calculate and predict the first weighting step and the periodic weighting step of the main roof of the 9203 working face of Hengsheng Coal Mine in China’s Shanxi Province, as well as verifying the validity and reliability of the synergy prediction model of information entropy based on zone safety degree by the comparison of similar simulation test results, which has presented a scientific basis for the effective control of rock pressure and roof management.

Visualization of Cell Membrane Tension Regulated by the Microfilaments as a "Shock Absorber" in Micropatterned Cells.

The extracellular stress signal transmits along the cell membrane-cytoskeleton-focal adhesions (FAs) complex, regulating the cell function through membrane tension. However, the mechanism of the complex regulating membrane tension is still unclear. This study designed polydimethylsiloxane stamps with specific shapes to change the actin filaments' arrangement and FAs' distribution artificially in live cells, visualized the membrane tension in real time, and introduced the concept of information entropy to describe the order degree of the actin filaments and plasma membrane tension. The results showed that the actin filaments' arrangement and FAs' distribution in the patterned cells were changed significantly. The hypertonic solution resulted in the plasma membrane tension of the pattern cell changing more evenly and slowly in the zone rich in cytoskeletal filaments than in the zone lacking filaments. In addition, the membrane tension changed less in the adhesive area than in the non-adhesive area when destroying the cytoskeletal microfilaments. This suggested that patterned cells accumulated more actin filaments in the zone where FAs were difficult to generate to maintain the stability of the overall membrane tension. The actin filaments act as shock absorbers to cushion the alternation in membrane tension without changing the final value of membrane tension.

Use of the entropy approach in water resource monitoring systems

Effective management of water resources is possible only with an effectively organized monitoring system. After the emergence and development of information theory, the concept of information entropy found its place in the field of the development of water monitoring systems. The purpose of this work is to review research related to the construction of water monitoring systems and networks that applied the entropy theory in the design process. Methodology. Entropy terms used in the construction of water monitoring systems are summarized. Recent applications of the entropy concept for water monitoring system designs classified by precipitation are reviewed; flow and water level; water quality; soil moisture and groundwater. The integrated method of designing multifactorial monitoring systems is also highlighted. Results. The review analyzes studies and their implementation in the design of water monitoring networks based on entropy. The use of various methods of information theory and their adaptation for use in the design of monitoring networks is demonstrated, with the goal of network design methods being the selection of stations that provide the most information for the monitoring network, while being independent of each other. Through extensive testing, information theory has proven to be a reliable tool for evaluating and designing an optimal water monitoring network. Scientific novelty. This review focuses on studies that have applied information theory or information entropy to construct monitoring networks and systems. Information theory was developed by Shannon in the middle of the last century to measure the information content of a data set and was subsequently applied to solving water resources problems. To date, there are no review studies regarding the design of water monitoring networks based on the concept that entropy will be able to characterize the information specific to the monitoring station or monitoring networks. The main goal is to have the maximum amount of information. Practical significance. The optimal design of the monitoring network can be built based on the specified design criteria; however, the practical application of a new optimal monitoring network is rarely evaluated in a hydrological or other model. It is also important to identify the benefits of entropy-based network design to convince decision-makers of the importance of entropy-based approaches. The optimal network can be subjective, based on the choices made during the entropy calculation and the design method chosen, especially when additional objective functions are considered in the design. This applies to the method chosen to construct the optimal monitoring network, whether it is found using an iterative method where one station is added at a time, or a collection of stations that are added simultaneously. Research has also shown that data length, catchment scale, and the order can affect optimal network design. when using discrete entropy, it was shown that the binning method affects the final network design. Therefore, when selecting options based on the intended application of the monitoring network, a clear understanding and further research is needed to provide recommendations specific to water monitoring networks. In particular, more work is needed on the spatial and temporal scaling of the entropy calculation data to provide robust recommendations for decision-makers.

Attribute reduction based on fusion information entropy

The attribute reduction algorithm based on fuzzy rough set theory is a commonly used method for dealing with uncertain information. However, many existing attribute reduction algorithms do not consider the information provided by the upper approximation. In fact, the upper approximation contains some certain decision information and decision risks brought by uncertain information. Therefore, we propose an attribute reduction algorithm based on fusion information entropy (FIE) to make proper use of the upper approximation. Firstly, we define the concepts of kernel decision degree and general decision degree, and flexibly combine them using the precision decision index. This method overcomes the limitations of existing attribute reduction methods in considering the upper approximation information. Then, we introduce the concept of fusion information entropy and measure the importance of attribute subsets by analyzing the changes in fusion information entropy. Finally, we design the corresponding attribute reduction algorithm. Extensive comparative experiments on public datasets show that the FIE algorithm based on fusion information entropy achieves higher classification accuracy while retaining fewer attributes compared with other state-of-the-art reduction algorithms. These results confirm the effectiveness and superiority of the FIE method proposed in this paper.

Merging experimental design and structural identification around the concept of modified Constitutive Relation Error in low-frequency dynamics for enhanced structural monitoring

This paper presents a novel Optimal Sensor Placement (OSP) strategy that is dedicated to model updating problems based on the modified Constitutive Relation Error (mCRE) functional in low-frequency dynamics. The mCRE is a credible alternative to model updating functionals that stands out by searching structural parameters alongside mechanical fields as the best trade-off between all available information from measured data, without any further a priori assumption. Considering damage detection problems, due to possible discrepancies in terms of parameters sensitivity with respect to mCRE, sensor locations provided by standard OSP algorithms may be irrelevant. The proposed approach uses the concept of Information Entropy by formulating a modified Fisher information matrix, in which the sensitivity of the mCRE mechanical fields with respect to the updated parameters is involved. The approach is legitimated by the strong connection between mCRE and Bayesian inference. A proof-of-concept involving an earthquake engineering inspired academic case study, where accelerometers are positioned on a two-story frame structure subjected to random ground motion, permits to illustrate the soundness and efficiency of the proposed methodology compared to other classical OSP techniques. The influence of critical mCRE parameters is shown, as well as the benefits of taking multiple scenarios into account so as to get an OSP that is relevant for a wider range of possible damage occurrences.

Structural k-means (S k-means) and clustering uncertainty evaluation framework (CUEF) for mining climate data

Abstract. Dramatic increases in climate data underlie a gradual paradigm shift in knowledge acquisition methods from physically based models to data-based mining approaches. One of the most popular data clustering/mining techniques is k-means, and it has been used to detect hidden patterns in climate systems; k-means is established based on distance metrics for pattern recognition, which is relatively ineffective when dealing with “structured” data, that is, data in time and space domains, which are dominant in climate science. Here, we propose (i) a novel structural-similarity-recognition-based k-means algorithm called structural k-means or S k-means for climate data mining and (ii) a new clustering uncertainty representation/evaluation framework based on the information entropy concept. We demonstrate that the novel S k-means could provide higher-quality clustering outcomes in terms of general silhouette analysis, although it requires higher computational resources compared with conventional algorithms. The results are consistent with different demonstration problem settings using different types of input data, including two-dimensional weather patterns, historical climate change in terms of time series, and tropical cyclone paths. Additionally, by quantifying the uncertainty underlying the clustering outcomes we, for the first time, evaluated the “meaningfulness” of applying a given clustering algorithm for a given dataset. We expect that this study will constitute a new standard of k-means clustering with “structural” input data, as well as a new framework for uncertainty representation/evaluation of clustering algorithms for (but not limited to) climate science.

Implementation of the concept of information entropy when creating water monitoring networks

Implementation of the concept of information entropy when creating water monitoring networks

Principal component analysis (PCA) based sparrow search algorithm (SSA) for optimal learning vector quantized (LVQ) neural network for mechanical fault diagnosis of high voltage circuit breakers

An essential piece of equipment for the protection and management of the electricity grid is a high voltage(HV) circuit breaker. The reliability and stability of the electricity grid are directly affected by the condition of operation of a high voltage circuit breaker. This study proposes a PCA-SSA-LVQ-based high-voltage circuit breaker diagnosis method to identify mechanical faults of high-voltage circuit breakers more quickly and effectively. Laboratory simulation of vibration signals for different fault states of high voltage circuit breakers. Finite intrinsic mode functions(IMF) are obtained by empirical modal decomposition(EMD) of the simulated vibration signal. The envelope spectrum of the IMF is extracted using the Hilbert envelope transform, and the concept of information entropy is used to determine the entropy value of the envelope as the characteristic quantity of the signal. Using principal component analysis, one can decrease the eigenvectors’ dimension. The high voltage circuit breaker LVQ neural network fault diagnosis model is made, and the initial weight of the LVQ neural network is optimized using the SSA optimization algorithm. The accuracy was improved in the end. The optimized and non-optimized LVQ neural networks were compared. The results show that the PCA-SSA-LVQ neural network has a higher diagnostic rate and faster training convergence. The main contribution of this paper is to reduce the redundancy of feature vectors using the PCA algorithm, optimize LVQ neural network by the SSA algorithm, and creatively use these three algorithms together. The training speed and the accuracy of state recognition are improved.

A probabilistic model on streamwise velocity profile in open channels using Tsallis relative entropy theory

The present study uses the concept of information entropy to derive a velocity distribution model in open channel flow. The Tsallis relative entropy theory is explored for that purpose, where the prior probability density function (PDF) is chosen from the maximum Tsallis entropy distribution. Both the cases of the fixed and varying entropy index are considered from the literature. The velocity equations are derived analytically using the homotopy analysis method (HAM) together with the Padé approximation technique in a general framework. The approximations and the HAM-based series solution are verified against laboratory and field data sets. Also, the prediction accuracies of the proposed models are assessed through the measures of statistical errors. It is seen that the model corresponding to the varying index is superior to the other model. The entropy index of Tsallis relative entropy is considered a variable, and the non-unity values of this index justify the applicability of the entropy function. Moreover, it is observed that one of the Lagrange multipliers corresponding to the velocity model with a varying index becomes negligible subject to the data sets considered, and hence, it simplifies the model development mathematically. The proposed approach can be further extended to develop models for estimating the suspended sediment concentration and shear stress distribution.

Quantitative Analysis on the Proportion of Renewable Energy Generation Based on Broadband Feature Extraction

With the massive access of distributed renewable energy sources, many uncertain renewable energy power components have been added to the low-voltage lines in substations in addition to the loads of definite classification. From the perspective of economy and cleanliness, it is necessary to quantitatively analyze the renewable energy share among them and improve the power quality level of users. For the power quality information at low-voltage feeders, this paper proposes a quantitative analysis algorithm based on improved wavelet energy entropy and LSTM neural network. The method is based on wavelet transform, based on sym8 wavelet basis function; it divides the long time sequence into equal-length small time sequences, calculates each feature component obtained from wavelet transform decomposition separately, then borrows the concept of information entropy to find its energy entropy. After obtaining the energy entropy sequence of each feature component, it then borrows the concept of kurtosis to weighted differentiation of each energy entropy sequence to highlight the feature information and finally, uses the LSTM neural network to classify the power quality information of different renewable energy sources to determine to which interval segment they belong. By building a simulation model to simulate the actual data in the field, the percentage of renewable energy can be quantitatively analyzed efficiently and accurately.

Analysis of the Information Entropy on Traffic Flows

This paper aims to reveal the uncertainty of traffic flow by introducing a new quantity based on the concept of information entropy (IE). We discover the existence and analyze the properties of IE of traffic flows. It is revealed by both real-world trajectory data and simulation data that the IE of traffic flows can be clearly measured and observed. More importantly, the relationships between IE and other key quantities, those are space mean speed and density, in traffic flow analysis can be described by linear and parabolic functions. We also discover that these relationships are not sensitive to traffic volume. With the inspiration from IE, another new quantity termed speed entropy (SE) is then proposed. Tests with aggregated traffic data from Performance Measurement System (PeMS) show that the pattern of the relationship between SE and flow-weighted average speed illustrates different traffic conditions. In general, a key achievement of the IE analysis is that it gives us a new pathway to better capture the intricate traffic flows from the dimension of uncertainty, thus it has the potential to enhance existing models for traffic data analysis.

Modelling and visualizing information entropy in Python

This paper examines the specifics of structural evolution in large systems and diverse distribution of resource in the context of the traditional Information Entropy concepts. An original and comprehensive methodology for modelling and visualizing the production of maximal and minimal levels of entropy is developed in Python/Pycharm environment.